StableDiffusion

This is an automated archive made by the Lemmit Bot.

The original was posted on /r/stablediffusion by /u/Won3wan32 on 2024-09-05 05:31:22+00:00.

TL;DR: we propose an end-to-end audio-only conditioned video diffusion model named Loopy. Specifically, we designed an inter- and intra-clip temporal module and an audio-to-latents module, enabling the model to leverage long-term motion information from the data to learn natural motion patterns and improving audio-portrait movement correlation. This method removes the need for manually specified spatial motion templates used in existing methods to constrain motion during inference, delivering more lifelike and high-quality results across various scenarios.