In today’s blog post, we are venturing into the exciting world of ComfyUI as they unveil a collection of groundbreaking Custom Nodes. Think of these nodes as the secret sauce, akin to what Automatic 1111 brings to Stable Diffusion. Our focus today is one of these transformative tools – the ‘Animate Diff Evolved’ Custom Node.
Introducing ‘AnimateDiff-Evolved’
Before we dive into the intricacies of ‘Animate Diff Evolved,’ some of you may recall our previous tutorial on another ‘Animate Diff’ feature within ComfyUI. ‘Animate Diff Evolved’ shares similarities with its predecessor but introduces an array of additional settings for generating animation frames.
Simplified Installation Process
Let’s get started with the installation process, which is remarkably straightforward. Navigate to the ComfyUI Manager, locate ‘Animate Diff Evolved,’ and with a simple click on ‘Install,’ give your Web UI a quick restart. Additionally, you’ll need to download the ‘Animate Diff Motion’ models from Hugging Face. For this demonstration, I’ve chosen the latest ‘mm sd v15 vs’ model, but you’ll find other camera panning motion models available to suit your needs.
ComfyUI Custom Node In This Tutorial : https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved
Motioin Models : https://huggingface.co/guoyww/animatediff/tree/main
Navigating ComfyUI with Ease
Now, let’s immerse ourselves in the ComfyUI web interface. But first, for those of you curious about the hardware configuration necessary for running Stable Diffusion locally, fret not, as we’ll cover that right here.
Confirmation of Installation
Perfect, it appears that ‘Animate Diff Evolved’ has installed seamlessly. Within ComfyUI, head to the ‘Install Custom Nodes’ section within the ComfyUI Manager. Search for ‘Animate Diff Evolved’ and proceed to download it.
Loading Custom Workflow
Now, we’ve loaded a text-to-animation workflow. You can find a selection of these workflows on the Animate Diff GitHub page. When you drag and drop your workflow file into ComfyUI, watch out for any nodes marked in red; they signify missing custom nodes. But don’t worry, rectifying this is a breeze.
Installing Missing Custom Nodes
To resolve this, simply reopen the ComfyUI Manager and select ‘Install Missing Custom Nodes.’ The system will automatically identify the required nodes and handle the installation for you. Following a quick ComfyUI restart, you’re all set to delve into the magic of Stable Diffusion.
Exploring the Workflow
With the red flags cleared, let’s take a closer look at the Batch Prompt Schedule. This schedule commences with the ‘Checkpoint loader’ and ‘VAE loader.’ Subsequently, two yellow nodes, courtesy of ‘Animate Diff Evolved,’ make their entrance, forming the core of the image-to-animation process in this workflow.
Fine-Tuning with Custom Nodes
Leveraging the motion model downloaded earlier, we’ll employ the linear schedule method. The default settings for ‘Uniform Context Options’ suffice for most scenarios, and a context length of 16 typically yields optimal results for this type of animation. The ‘Latent Count’ parameter dictates how your images will be arranged in the animation output.
Creating Prompts
Now, let’s experiment with a prompt. For Batch Prompt text formatting, follow this pattern: enclose the frame number in double quotes and separate each transition with commas. Remember, refrain from adding a comma after the last prompt to avoid error messages.
Exporting Your Masterpiece
Before we hit the ‘Go’ button, keep in mind that you have the option to save your animations as either GIFs or MP4s. Opting for MP4? Ensure that you have FFmpeg installed in your environment.
Witness the Speed
Get ready for the results! As you can see, the outcome closely aligns with what was showcased on the GitHub page. Do note that even with identical Seed numbers, the animation may exhibit slight variations.
A World of Possibilities
Explore further with other available workflows. ComfyUI offers the convenience of saving your workflow file, allowing you to share your JSON or image files with others. You can even drag and drop the workflow file to queue prompts for instant generation.
Experience Unparalleled Speed
Let’s run this prompt once more, this time with a randomized Seed number, and witness ComfyUI’s incredible speed. We’re talking about generating animations in a minute or less.
Performance Beyond Compare
When it comes to creating animations with text or images, ComfyUI surpasses the competition, carving a niche of its own.
A Glimpse of Realism
In this post, I’ve also demonstrated how this animation custom node seamlessly integrates with Realism style’s checkpoint models. Stay tuned for the stunning visuals showcased at the beginning and end of this article.
What Lies Ahead
In a previous tutorial, we delved into Stable Diffusion with Pika Labs for crafting story animations. Next, I’ll be back with a tutorial solely dedicated to using Stable Diffusion to craft captivating story animation videos.
Stay Tuned for More
So, stay tuned for more captivating Stable Diffusion animation tutorials, and let your creative spirit soar!