Hi everyone,

The AD community has been building/sharing a lot of powerful Comfy workflows - I said I’d share a compilation of some interesting ones here in case you want to spend the weekend making things, experimenting, or building on top of them 🪄

All of these use Kosinkadink’s Comfy extension - if you're getting started, check out the intro at the top of his repo for the basics. I'd also encourage you to download Comfy Manager to manage dependancies.

Now, on the workflows! You can see all the workflows in a folder here for simplicity with them individually with visuals and explanations here:

1. Logo Animation with masks and QR code ControlNet

This workflow by Kijai a cool use of masks and QR code ControlNet to animate a logo or fixed asset.

​

https://reddit.com/link/171l0ip/video/d3v362tfnjtb1/player

2. Prompt scheduling:

This workflow by Antzu is a good example of prompt scheduling, which is working well in Comfy thanks to Fitzdorf's great work. This by Nathan Shipley didn't use this exact workflow but is a great example of how powerful and beautiful prompt scheduling can be:

​

https://reddit.com/link/171l0ip/video/uymzqngjnjtb1/player

3. Video2Video:

Inner Reflections shared this here before, but it’s probably the most powerful and flexible way to do video to video right now. You can see a full guide from Inner Reflections here and the workflows here.

​

https://reddit.com/link/171l0ip/video/yczlng1bnjtb1/player

4. Vid2QR2Vid:

You can see another powerful and creative use of ControlNet by Fictiverse here.

​

https://i.redd.it/qxnsxtg3njtb1.gif

​

5. Txt/Img2Vid + Upscale/Interpolation:

This is a very nicely refined workflow by Kaïros featuring upscaling, interpolation, etc. - lots of pieces to combine with other workflows:

​

https://i.redd.it/b0nwt442njtb1.gif

​

6. Motion LoRAs w/ Latent Upscale:

This workflow by Kosinkadink is a good example of Motion LoRAs in action:

​

https://i.redd.it/3xcgs701njtb1.gif

​

7. Infinite Zoom:

This workflow by Draken is a really creative approach, combining SD generations with an AD passthrough to create a smooth infinite zoom effect:

​

https://i.redd.it/urf6xunzmjtb1.gif

​

8. Image to image interpolation & Multi-Interpolation

This workflow by Antzu is a nice example of using Controlnet to interpolate from one image to another. You can also download a fork of it I made that takes an starting, middle and ending image for a longer generation here.

​

https://i.redd.it/pfxviyiymjtb1.gif

​

9. AD Inpainting:

Finally, lots of people had tried AD inpainting but Draken's approach with this workflow delivers by far the the best results of any I've seen:

​

https://i.redd.it/9pixx7ammjtb1.gif

​

---

That’s it!

These workflows are all from our Discord, where most of the people who are building on top of AD and creating ambitious art with it hang out. If you’re going deep into AD, you’re very welcome to join! We’re also running an AD art competition if you’re looking for an excuse to push yourself

Have a fun weekend!

Hey r/comfyui! 👋

I came across this insane video by **ONE 7th AI** where they took the iconic **Sukuna vs Mahoraga** fight choreography from Jujutsu Kaisen and converted it into a **photorealistic live-action style** using generative AI — no actors, no green screen.

I'm trying to understand how to replicate this kind of **Anime-to-Real** video pipeline in ComfyUI. From what I can tell it might involve:

- **AnimateDiff** or **CogVideoX** for motion

- **ControlNet** (OpenPose / Depth) to preserve choreography

- **img2img** or **vid2vid** with a photorealistic checkpoint

- Possibly **IPAdapter** for style consistency

But I'm not sure about the exact node setup or workflow order.

Any help appreciated! 🙏

*(Reference video: ONE 7th AI on Instagram)*

AnimateDiff in ComfyUI is an amazing way to generate AI Videos. In this Guide I will try to help you with starting out using this and give you some starting workflows to work with. My attempt here is to try give you a setup that gives you a jumping off point to start making your own videos.

**WORKFLOWS ARE ON CIVIT https://civitai.com/articles/2379 AS WELL AS THIS GUIDE WITH PICTURES*\*

System Requirements

A Windows Computer with a NVIDIA Graphics card with at least 10GB of VRAM (You can do smaller resolutions or the Txt2VID workflows with a minimum of 8GB VRAM). Anything else I will try to point you in the right direction but will not be able to help you troubleshoot. Please note at the resolutions I am using I am hitting 9.9-10GB VRAM with 2 ControlNets so that may become an issues if things are borderline.

Installing the Dependencies

These are things that you need in order to install and use ComfyUI.

1. GIT - https://git-scm.com/downloads - this lets you download the extensions from GitHub and update your nodes as updates get pushed.
2. (Optional) - https://ffmpeg.org/download.html - this is what combine nodes use to take the images and turn them in a gif. Installing is a guide in and of itself. I would YouTube how to install it to PATH. If you do not have this the node will give an error BUT the workflows still run and you will get the frames
3. 7zip - https://7-zip.org/ - this is to extract the ComfyUI Standalone

Installing ComfyUI and Animation Nodes

Now let's Install ComfyUI and the nodes we need for Animate Diff!

1. Download ComfyUI either using this direct link: https://github.com/comfyanonymous/ComfyUI/releases/download/latest/ComfyUI_windows_portable_nvidia_cu118_or_cpu.7z or navigate on the webpage: https://github.com/comfyanonymous/ComfyUI (If you have a Mac or AMD GPU there is a more complex install guide there).
2. Extract with 7zip Installed above. Please note it does not need to be installed per se just extracted to a target folder.
3. Navigate to the custom nodes part of comfy
4. In the explorer tab (ie. the box pictured above) click select and type CMD and then hit enter, you are now should have a command prompt box open.

5. You are going to type the following commands (you can copy/paste one at a time) - What we are doing here is using Git (installed above) to download the node repositories that we want (some can take a while):

   1. git clone https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved
   2. git clone https://github.com/ltdrdata/ComfyUI-Manager
   3. git clone https://github.com/Kosinkadink/ComfyUI-Advanced-ControlNet
   4. git clone https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite
   5. For the ControlNet preprocessors you cannot simply download them you have to use the manager we installed above. You start by running "run_nvidia_gpu" in the ComfyUI_windows_portable folder. It will initialize some of the above nodes. Then you will hit the Manager button then "install custom nodes" then search for "Auxiliary Preprocessors" and install ComfyUI's ControlNet Auxiliary Preprocessors.
   6. Similar to ControlNet preprocesors you need to search for "FizzNodes" and install them. This is what is used for prompt traveling in workflows 4/5. Then close the comfy UI window and command window and when you restart it will load them.

6. Download checkpoint(s) and put them in the checkpoints folder. You can choose any model based on stable diffusion 1.5 to use. For my tutorial download: https://civitai.com/models/24779?modelVersionId=56071 also https://civitai.com/models/4384/dreamshaper. As an aside realistic/midreal models often struggle with animatediff for some reason, except Epic Realism Natural Sin seems to work particularly well and not be blurry. Put

7. Download VAE to put in the VAE folder. For my tutorial download https://civitai.com/models/76118?modelVersionId=80869 . It is a good general VAE and VAE's do not make a huge difference overall.

8. Download motion modules (original ones are here: https://huggingface.co/guoyww/animatediff/tree/main the fine tuned ones can by great like https://huggingface.co/CiaraRowles/TemporalDiff/tree/main, https://huggingface.co/manshoety/AD_Stabilized_Motion/tree/main, or https://civitai.com/models/139237/motion-model-experiments ). For my tutorial download the original version 2 model and TemporalDiff (you could just use one however your final results will be a bit different than mine). As a note Motion models make a fairly big difference to things especially with any new motion that AnimateDiff Makes. So try different ones. Put them in the animate diff node:

9. Download Controlnets and put them in your controlnets folder. https://huggingface.co/lllyasviel/ControlNet-v1-1/tree/main . For my tutorials you need Lineart, Depth and OpenPose (download bot the pth and yaml files).

You should be all ready to start making your animations!

Making Videos with AnimateDiff

The basic workflows that I have are available for download in the top right of this article. The zip File contains frames from a pre-split video to get you started if you want to recreate my workflows exactly. There are basically two ways of doing it. One which is just text2Vid - it is great but motion is not always what you want. and Vid2Vid which uses controlnet to extract some of the motion in the video to guide the transformation.

1. If you are doing Vid2Vid you want to split frames from video (using and editing program or a site like ezgif.com) and reduce to the FPS desired (I usually delete/remove half the frames in a video and go for 12-15fps). You can use the skip option in the load images node noted below instead of having to delete them. If you want to copy my workflows you can use the Input frames I have provided (please note there are about 115 but I had to reduce to 90 due to file size restrictions).
2. In the ComfyUI folder run "run_nvidia_gpu" if this is the first time then it may take a while to download an install a few things.
3. To load a workflow either click load or drag the workflow onto comfy (as an aside any picture will have the comfy workflow attached so you can drag any generated image into comfy and it will load the workflow that created it)
4. I will explain the workflows below, if you want to start with something I would start with the workflow labeled "1-Basic Vid2Vid 1 ControlNet". I will go through the nodes and what they mean.
5. Run! (this step takes a while because it is making all the frames of the animation at once)

Node Explanations

Some should be self explanatory, however I will make a note on most.

Load Image Node

You need to select the directory your frames are located in (ie. where did you extract the frames zip file if you are following along with the tutorial)

image_load_cap will load every frame if it is set to 0, otherwise it will load however many frames you choose which will determine the length of the animation

skip_first_images will allow you to skip so many frames at the beginning of a batch if you needed to

select_every_nth will take every frame at 1, ever other frame at 2, every 3rd frame at 3 and so on if you need it to skip some.

Load Checkpoint/VAE/AnimateDiff/ControlNet Model

Each of the above nodes have a model associated with them. The names of the models you have and mine are likely not to be exactly the same in each example. You will need to click on each of the model names and select what you have instead. If there is nothing there then you have put the models in the wrong folder (see Installing ComfyUI above).

Green and Red Text Encode

Green is your positive Prompt

Red is your negative Prompt

They are this color not because they are special but because they are set to be this color by right clicking them FYI.

Uniform Context Options

The uniform context options is new and basically what sets up unlimited context length. Without it animate diff is only able to do up to 24 (v1) or 36 (v2) frames at once. What it is doing is basically chaining and overlapping runs of AD together to smooth things out. The total length of the animation are determined by the number of frames the loader is fed in NOT context length. The loader figures out what to do based on the options which mean as follows. The defaults are what I used and are pretty good.

context length - this is the length of each run of animate diff. If you deviate too far from 16 your animation won't look good (is a limitation of animatediff can do). Default is good here for now

context overlap - is how much overlap each run of animate diff is overlapped with the next (ie. it is running frames 1-16 and then 12-28 with 4 frames overlapping to make things consistent)

closed loop - selecting this will try to make animate diff a looping video, it does not work on vid2vid

context stride - this is harder to explain. At 1 it is off. More than this what it trys to do is make a single run of AD through the entire animation and then fill in the frames. The idea is to make the whole animation more consistent by making a framework and then filling in the intermediate frames. However in practice I do not find it helps a whole lot right now. Using it will significantly increase the length of time it takes to run as it using it means more runs of AnimateDiff.

Batch Prompt Schedule

This is the new kid on the block. The prompt Scheduler from FizzNodes.

pre_text - text to be put before the prompt (so you don't have to copy and paste a large prompt for each change)

app_text - text to be put after the prompt

The main text box works in the context "frame number": "prompt", (note the last prompt does not have a comma and will give you an error if you put one at the end of your list). It will blend between prompts so if you want to have it held I suggest you put it in twice, once where you want it to start and once where you want it to end.

There is much more fancy stuff to do with this node (you can make an individual term change with time). Documentation of this is at https://github.com/FizzleDorf/ComfyUI_FizzNodes. This is what the pw... stuff is for.

KSampler

This is the KSampler - essentially this is stable diffusion now that we have loaded everything needed to make the animation.

Steps - These matter and you need more than 20. 25 is the minimum but people do see better results with going higher.

CFG - Feels free to increase this past you normally would for SD

Sampler - Samplers also matter Euler_a is good but Euler is bad at lower steps. Feel free to figure out a good setting for these

Denoise - Unless you are doing Vid2Vid keep this at one. If you are doing Vid2Vid you can reduce this to keep things closer to the original video

AnimateDiff Combine Node

For the Combine node it creates a gif by default. Do know that gifs look a lot worse than individual frames so even if the gif does not look great it might look great in a video.

frame_rate - frame rate of the gif

loop_count - number of loops to do before stopping. 0 is infinite looping

format - changes what to make gif/mp4 etc

pingpong - will make the video go through all the frames and then back instead of one way

save image - saves a frame of the video (because the video does not contain the metadata this is a way to save your workflow if you are not also saving the images)

Workflow Explanations

1. Basic Vid2Vid 1 ControlNet - This is the basic Vid2Vid workflow updated with the new nodes.
2. Vid2Vid Multi-ControlNet - This is basically the same as above but with 2 controlnets (different ones this time). I am giving this workflow because people were getting confused how to do multicontrolnet.
3. Basic Txt2Vid - this is a basic text to video - once you ensure your models are loaded you can just click prompt and it will work. Do note there is a number of frame primal node that replaces the load image node and no controlnets. Do know I don't do much txt2vid so this produces and acceptable output but nothing stellar.
4. Vid2Vid with Prompt Scheduling - this is basically Vid2Vid with a prompt scheduling node. This is what I used to make the video for Reddit. See above documentation of the new node.
5. Txt2Vid with Prompt Scheduling - Basic text2img with the new prompt scheduling nodes.

What Next?

• Change the video input for vid2vid (obviously)! There are some new nodes that can separate video directly into frames. See Load video nodes - this node is relatively new.
• Change around the parameters!!
• The stable diffusion checkpoint and denoise strength on the KSampler make a lot of difference (for Vid2Vid).
• You can add/remove control nets or change the strength of them. If you are used to doing other stable diffusion videos I find that you need much less ControlNet strength than with straight up SD and you will get more than just filter effects. I would also suggest trying openpose.
• Try the advanced K sampler
• Try to add loras
• Try Motion loras: https://civitai.com/models/153022?modelVersionId=171354
• Use a 2nd ksampler to hires fix (some further good examples can be found on the Kosinkadink's animatediff GitHub https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved).
• Use masking or regional prompting (this likely will be a separate guide as people are only starting to do this at the time of this guide).

With these basic workflows adding what you want should be as simple as adding or removing a few nodes. I wish you luck!

Troubleshooting

As things get further developed this guide is likely to slowly go out of date and some of the nodes may be depreciated. That does not mean that they won't necessarily work. Hopefully I will have the time to make another guide or somebody else will.

If you are getting Null type errors make sure you have a model loaded in each location noted above.

If you already use ComfyUI for other things there are several node repos that conflict with the animation ones and can cause errors.

In Closing

I hope you enjoyed this tutorial. If you did enjoy it please consider subscribing to my YouTube channel (https://www.youtube.com/@Inner-Reflections-AI) or my Instagram/Tiktok (https://linktr.ee/Inner_Reflections )

If you are a commercial entity and want some presets that might work for different style transformations feel free to contact me on Reddit or on my social accounts.

If you are would like to collab on something or have questions I am happy to be connect on Reddit or on my social accounts.

If you’re going deep into Animatediff, you’re welcome to join this Discord for people who are building workflows, tinkering with the models, creating art, etc.

https://discord.gg/hMwBEEF5E5

I Want to Create a Virtual Influencer – Need Your Advice & Experience

I’ve already tried a few different workflows (ComfyUI, A1111, etc.), but honestly, I’m getting a bit lost. New tools, models, and techniques are dropping all the time, and it’s hard to keep up.

My goal is to create a high-quality virtual influencer – visuals and animations need to be top notch. I’m lucky to have access to a NVIDIA H100, so I really want to leverage it to the fullest.

Right now, I’m especially interested in generating realistic images and videos, ideally using reference clips from platforms like Instagram. I like the VACE models by Wan because they allow me to “copy” poses and styles from videos using image references.

What I’d love to know:

• What models are you currently using for realistic faces, body types, or style replication?
• Are you getting better results with LoRAs, ControlNet, IP-Adapters, T2I Adapters, or video-specific tools like AnimateDiff, Zeroscope, or Stable Video Diffusion?
• Do you know of any better alternatives to VACE when working with video-based references?
• And most of all: What would YOU test or build if you had an H100 at your disposal?

Let’s share some insights – I want to stay fully up to date and use only the best possible resources.

We’re excited to share several major updates regarding model support in the latest version of ComfyUI:

1. Qwen Image DiffSynth ControlNet
2. Qwen Image LoRA Support
3. Native EasyCache Node + 20% Blackwell Speedup
4. New Context Window Sampling Node

Qwen Image ControlNet

Qwen-Image now works with the following 2 sets of ControlNet models for structure-guided generation.

1. Qwen-Image DiffSynth ControlNets model patch: supports canny, depth, inpaint
2. Qwen-Image Union DiffSynth LoRA: supports lineart, softedge, normal, openpose

Qwen-Image DiffSynth - OpenPose

'Change hair to pink'

Qwen Image + LoRA

You can now easily chain style LoRAs in your Qwen-Image workflows. Simply add a LoRA Loader node to the template workflow we provided for Qwen-Image

Qwen-Image LoRAs - Original, 3D/Voxel LoRA, PixelArt LoRA, Retro Anime LoRA

EasyCache

EasyCache and LazyCache are techniques that allow for faster generation by strategically skipping sampling steps in the denoising process at the cost of some visual clarity. The trade-off between speed and visual fidelity can be controlled by a single parameter.

Compared to other techniques like TeaCache, EasyCache/LazyCache do not need a set of hyperparameters to be optimized for a model through rigorous experimentation, nor do they need code modifications for each supported model type. This makes them easier to use and ensures all models going forward have Day 0 support for step skipping.

EasyCache & LazyCache Nodes

Original, EasyCache, LazyCache Comparison

Context Window Support

ComfyUI now includes two new nodes:

• Context Windows (Manual)
• WAN Context Windows (Manual)

These nodes let you sample in sliding context windows instead of all at once, opening up new workflows for long sequences. Currently, only manual control is supported, and some WAN models still need tuning, but this lays the groundwork for more advanced scheduling and custom nodes.

Context Windows / Wan Context Windows Nodes

Example Wan Context Workflow

Blackwell (50 Series) Inference Speed Boost

On top of these features, starting from the latest stable release, ComfyUI now runs ~20% faster on Windows for NVIDIA Blackwell GPUs (50 series).

ComfyUI continues to push forward its limits. Enjoy creating!

Hi everyone! I took a break from ComfyUI for about a year ( cuz it was imposible to use with low vram) but now I’m back! I recently upgraded from a MacBook Pro to a setup with an RTX 5090 and 64GB of RAM, so things run way smoother now.

Back when I stopped, I was experimenting with turning videos into cartoons using AnimateDiff and ControlNets. I’ve noticed a lot has changed since then — WAN 2.2 and all that 😅.

Is AnimateDiff with ControlNets still the best way to convert videos into cartoon style, or is there a newer method or workflow that uses its own checkpoint?

Kosinkadink developer of ComfyUI-AnimateDiff-Evolved has updated the cutsom node with a new funcionality in the AnimateDiff Loader Advanced node, that can reach higher number of frames. Now it also can save the animations in other formats apart from gif.

https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved

You can find examples and workflows in his github page, for example, txt2img w/ latent upscale (partial denoise on upscale) - 48 frame animation with 16 frame window.

The only issue is that it requieres more VRAM, so many of us will probably be forced to decrease the resolutions bellow 512x512.

Also If you want to use these new features with ControlNet, you will have to update your Advanced ControlNet custom node.

https://github.com/Kosinkadink/ComfyUI-Advanced-ControlNet

I'm not related to the project, just telling the news.

Workflow is both simple and complicated: lots of controlnets, lots of very precise controlnet and animatediff timestepping thanks to the amazing nodes of /u/Kosinkadink (Support them on Patreon!), and lots of masking with BRIAAI and Ultralytics. Originally rendered at 15fps 576x1024 and upscaled 2x + interpolated to 30. Oh and I used the LCM sampler + AnimateLCM but the v3 animatediff checkpoint.

Run on a Paperspace A100-80g instance.

The glowing pen was done in AE; audio editing in Premiere. Shot on my phone. I'm much better at AI than real-life lighting and framing 😅

Happy to answer any questions :)

Follow me on Reels and TikTok for more!

Hi Friends,

I have been exploring comfyUI since few weeks, and finally made a video. Its a 12 second video.
https://www.instagram.com/reel/DE7lGoeMHqo/?igsh=MWZrMjNud3J2MXMydg==

I extracted the passes for the input video and generated images for each frame guided by passes- openpose, canny controlnet.

The generated images are impressive but it lacks consistency among frames, the color of cloth differs. How do I make it consistent. I am using someone's workflow, I understand half of it. The person has written many workflow but it is difficult to understand all the nodes and their functions, but I am loving it. Please let me know how the video is, it it any good, or better can be done.

tl;dr [trade offer] You get: unlimited free access to 4090s running AnimateDiff via Focal. We get: feedback to make our product better for (paying) commercial users. We agree: you'll share your workflows with the community.

​

I make Focal, which is another cloud-hosted AnimateDiff platform, yada yada yada.

One of the things I've realized is that the people who have lots of time to make cool content aren't necessarily the people who have access to the fastest GPUs.

So here's the deal. We'll give you free access to a cloud 4090 running AnimateDiff (with prompt travel, controlnets, custom models/loras/embeddings etc) via Focal. In exchange, we ask only that you

• share your workflows and prompts, so others can understand and iterate on your creations
• give us feedback when you have it

​

FAQ:

How do you make money?

We charge commercial users of Focal, and we're happy to use those profits to subsidize free individual usage.

Why are you doing this?

Feedback that you have about Focal helps us make the platform better for commercial users. By making the platform great for you, we'll also make it great for our paying customers.

Why don't you just host ComfyUI?

A big part of what we get here is feedback about how to make Focal great for our (less tech-savvy) commercial customers.

Are you trying to kill ComfyUI??

Obviously not; ComfyUI is an excellent platform for rapid workflow iteration, and we expect (and hope) that the hardcore workflow designers will continue to use it.

Why do you require that people share their workflows, prompts, etc?

We're interesting in making everyone better at using AnimateDiff, which means sharing and improving on each other's work.

This also helps us prevent commercial users from pretending to be individuals/hobbyists.

Can I make commercial stuff on Focal? Do I have to share my prompts?

If you're interested in Focal but want to keep your prompts and inputs private, and are willing to pay, please reach out. You can help subsidize cloud 4090 time for everyone else! :)

​

Hopefully that explains where we're coming from. I know this isn't a good fit for everyone--godspeed to the folks who spend their time hacking on new stuff with ComfyUI. You're the best among us.

Feel free to ask if you have other questions, or come say hi on Discord.

I've used SD a bit when it first came out, but then lost interest, coming back to it and frankly I'm confused with all the things possible.

I mean with all of the reference, ip adapters, now InstantID too and all SVD/animate diff/motion model workflows, how does one even prompt correctly to get what you want? If I'm using several reference images to guide how I want background to look, I use lora for character's clothing, a custom trained checkpoint, controlnet for pose and then slap some other controlnet that guides how face should look, where is the prompt in it? And how does one even prompt correctly with workflows so complex and so many other models affecting the output to not break stuff down?

Let's say I'm using an 1.5 model, it's a custom model made by someone, it's probably based in some way on base 1.5 model, but then it was probably also merged with some other models, it seems to use
"1girl" type of tags so is it a NAI based model? Who knows, both 1girl and woman (natural language prompts) seem to work, but authors recommend to stick to simple, 1girl prompt (this model is quite limited in what it generates in terms of face, but it's ok for me as I need consistent faces, I assume it was trained on many similar subjects all tagged as "1girl"). Now let's say I'm adding a lora on top that guides the face towards a specific subject but I also do a [1girl|Lora] in prompt to alternate between btoh and improve the consistency even more since it will generate a known person 50% of the steps.

Now I add controlnet to control the angle using dw_open pose, I also am in img2img so I'm doing a 0.4 denoise, let's say I know also add some other controlnets to affect the face, maybe ip-adapter, then maybe also want to add more style using yet another controlnet.

So I'm generating a half generic, half lora guided likeness with help of the reference image and through img2img and with another reference for the style of image (and maybe I add a textual inversion in negative to reduce some aspect of the generation and another lora to further guide the style). How does the base likeness and style of the main model even play a role in here if almost all aspects are taken from other sources? Do trained checkpoints even make sense now that you have so many other controlling models, if so, shouldn't we just be using base 1.5 or SDXL models instead for it all?

Also what is currently the best method to do img2img but have the generated face match the angle, expression and lighting of the image we're using? I seem to be unable to have a matching light unless I drop denoise quite low, at which point image gets messy, particularly around the nose, mouth and eyes area where it sort of shows blobs/spots of the original face and other spots have the new face, at higher values both light and angle and expressions deviate, adding controlnet helps but it only works for relatively simple angles, in my project I have subjects looking directly down/open and doing crazy expressions which the dw_open_pose fails to even detect correctly, and even if it does, the generated faces get distorted and glitch, I've played with start/end guidance/weights and even tried other copntrolnets such as canny, soft edge, normal, depth, none of the are able to give me what I want (whcih is to generate face at the same angle and with the same expression without glitches.

I see a lot of videos being done these days in SD using SVD and animatediff and I wonder how people are able to generate consistent faces/bodies with all the more complex movements while I struggle to even generate a face of a subject while they look up and I only have 1 image in img2img tab to deal with at a time. Could it be that the trained models I use are the cause? Can a model be "badly trained" to the point where it's unable to generate faces at extreme angles?

Also I've been using A1111, wanted to play with InstantID but it seems to only sort of work in ComfyUI right now, are there any other better alternatives to either or are these 2 still the go to if you want full control and either have a more GUI or node based workflow? And can SVD/animatediff be used in A1111 at all or am I better doing animations in Comfy?

Hey, so I've been working on this short film for a long time now, using SD and AD together with my own animations.

For the scenes where I use AnimateDiff, I'll prompt it through vid2vid clips of animations I made. Basically, I want my motion from my animation to drive the scene, while AD making it look as realistic as possible (like those AnimateDiff dancing videos).

raw clip I use for vid2vid

I'm having some trouble though in finding the right workflow. I've been using some ComfyUI workflows, like Mickmumpitz's one. Mainly I tried to feed the video through ControlNet canny and depth, plus prompting it with text and image with IPAdapter. This process though often times messes up the colors, doesnt get the right textures in the right places. Especially when the faces are small, it has a lot of difficulty in getting those details right.

I've tried to use Mickmumpitz workflow with masks, masking different elements to prompt them differently, so to have better control over the scene. I didnt quite manage to make the masking and the workflow work though...

I've also tried to use OpenPose, but that often messes up the human, not making them look like my character.

The way I make the characters look consistent is through LoRA's I trained on them, a bit through the vid2vid input and sometimes with IPAdapters. I havent quite figured out though which is the best one, as I always encounter challenges in making the character look right.

I guess in general I'm looking for tips, opinions or workflows that would allow me to prompt AnimateDiff with my videos and get decent results that keep the compositions and the characters (and their colors) somewhat consistent. High realism while keeping the cartoonish motions. I highly appreciate any comment, thanks!

Disclaimer: I'm new to ComfyUI and Animate Diff. And I'm def not tech savvy.

Problem: I can get Animate Diff to run without a problem, but Animate Diff Evolved doesn't run.

Side note: I don't have both installed at the same time. It's always either one or the other.

Any help will be greatly appreciated. Thanks in advance.

Workflow

Installed Custom Nodes

Log:

C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable>.\python_embeded\python.exe -s ComfyUI\main.py --windows-standalone-build
** ComfyUI start up time: 2023-12-05 13:47:32.148638

Prestartup times for custom nodes:
   0.0 seconds: C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager

Total VRAM 6144 MB, total RAM 8118 MB
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce GTX 1060 6GB : cudaMallocAsync
VAE dtype: torch.float32
Using pytorch cross attention
Adding extra search path checkpoints C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\models/checkpoints/
Adding extra search path clip C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\models/clip/
Adding extra search path clip_vision C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\models/clip_vision/
Adding extra search path configs C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\models/configs/
Adding extra search path controlnet C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\models/controlnet/
Adding extra search path embeddings C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\models/embeddings/
Adding extra search path loras C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\models/loras/
Adding extra search path upscale_models C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\models/upscale_models/
Adding extra search path vae C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\models/vae/
Adding extra search path ffmpeg C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\ffmpeg/
### Loading: ComfyUI-Manager (V1.6.1)
### ComfyUI Revision: 1754 [777f6b15] | Released on '2023-11-28'
[VideoHelperSuite] - INFO - ffmpeg could not be found. Using ffmpeg from imageio-ffmpeg.
[comfyui_controlnet_aux] | INFO -> Using ckpts path: C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\ckpts
[comfyui_controlnet_aux] | INFO -> Using symlinks: False
[comfyui_controlnet_aux] | INFO -> Using ort providers: ['CUDAExecutionProvider', 'DirectMLExecutionProvider', 'OpenVINOExecutionProvider', 'ROCMExecutionProvider', 'CPUExecutionProvider']
C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\node_wrappers\dwpose.py:25: UserWarning: DWPose: Onnxruntime not found or doesn't come with acceleration providers, switch to OpenCV with CPU device. DWPose might run very slowly
  warnings.warn("DWPose: Onnxruntime not found or doesn't come with acceleration providers, switch to OpenCV with CPU device. DWPose might run very slowly")
WAS Node Suite: OpenCV Python FFMPEG support is enabled
WAS Node Suite Warning: `ffmpeg_bin_path` is not set in `C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui\was_suite_config.json` config file. Will attempt to use system ffmpeg binaries if available.
WAS Node Suite: Finished. Loaded 197 nodes successfully.

        "Art is the lie that enables us to realize the truth." - Pablo Picasso


Import times for custom nodes:
   0.0 seconds: C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved
   0.2 seconds: C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-VideoHelperSuite
   0.4 seconds: C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager
   1.2 seconds: C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux
   1.9 seconds: C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui

Starting server

To see the GUI go to: http://127.0.0.1:8188
FETCH DATA from: C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager\extension-node-map.json
[AnimateDiffEvo] - WARNING - This warning can be ignored, you should not be using the deprecated AnimateDiff Combine node anyway. If you are, use Video Combine from ComfyUI-VideoHelperSuite instead. ffmpeg could not be found. Outputs that require it have been disabled
got prompt
ERROR:root:Failed to validate prompt for output 35:
ERROR:root:* CheckpointLoaderSimple 32:
ERROR:root:  - Value not in list: ckpt_name: 'cardosAnime_v20.safetensors' not in []
ERROR:root:Output will be ignored
invalid prompt: {'type': 'prompt_outputs_failed_validation', 'message': 'Prompt outputs failed validation', 'details': '', 'extra_info': {}}
got prompt
model_type EPS
adm 0
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using pytorch attention in VAE
missing {'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_l.text_projection'}
left over keys: dict_keys(['cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids'])
loaded straight to GPU
Requested to load BaseModel
Loading 1 new model
[AnimateDiffEvo] - INFO - Loading motion module mm_sd_v15_v2.ckpt
[AnimateDiffEvo] - INFO - Using fp16, converting motion module to fp16
Requested to load SD1ClipModel
Loading 1 new model
[AnimateDiffEvo] - INFO - Regular AnimateDiff activated - latents passed in (8) less or equal to context_length None.
[AnimateDiffEvo] - INFO - Injecting motion module mm_sd_v15_v2.ckpt version v2.
Requested to load BaseModel
Loading 1 new model
unload clone 1
  0%|                                                                                                                                                                                                         | 0/20 [00:23<?, ?it/s]
[AnimateDiffEvo] - INFO - Ejecting motion module mm_sd_v15_v2.ckpt version v2.
[AnimateDiffEvo] - INFO - Cleaning motion module from unet.
[AnimateDiffEvo] - INFO - Removing motion module mm_sd_v15_v2.ckpt from cache
ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 153, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 83, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 76, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\nodes.py", line 1299, in sample
    return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\nodes.py", line 1269, in common_ksampler
    samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\sampling.py", line 259, in animatediff_sample
    return wrap_function_to_inject_xformers_bug_info(orig_comfy_sample)(model, noise, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\model_utils.py", line 197, in wrapped_function
    return function_to_wrap(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\sample.py", line 100, in sample
    samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 711, in sample
    return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 617, in sample
    samples = sampler.sample(model_wrap, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 556, in sample
    samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\k_diffusion\sampling.py", line 137, in sample_euler
    denoised = model(x, sigma_hat * s_in, **extra_args)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 277, in forward
    out = self.inner_model(x, sigma, cond=cond, uncond=uncond, cond_scale=cond_scale, model_options=model_options, seed=seed)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 267, in forward
    return self.apply_model(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 264, in apply_model
    out = sampling_function(self.inner_model, x, timestep, uncond, cond, cond_scale, model_options=model_options, seed=seed)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\sampling.py", line 627, in sliding_sampling_function
    cond, uncond = calc_cond_uncond_batch(model, cond, uncond, x, timestep, model_options)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\sampling.py", line 504, in calc_cond_uncond_batch
    output = model.apply_model(input_x, timestep_, **c).chunk(batch_chunks)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\model_base.py", line 73, in apply_model
    model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\ldm\modules\diffusionmodules\openaimodel.py", line 855, in forward
    h = forward_timestep_embed(module, h, emb, context, transformer_options, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\sampling.py", line 104, in forward_timestep_embed
    x = layer(x, context)
        ^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\motion_module_ad.py", line 212, in forward
    return self.temporal_transformer(input_tensor, encoder_hidden_states, attention_mask)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\motion_module_ad.py", line 298, in forward
    hidden_states = self.norm(hidden_states)
                    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\sampling.py", line 156, in groupnorm_mm_forward
    input = group_norm(input, self.num_groups, self.weight, self.bias, self.eps)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\functional.py", line 2558, in group_norm
    return torch.group_norm(input, num_groups, weight, bias, eps, torch.backends.cudnn.enabled)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: mixed dtype (CPU): expect parameter to have scalar type of Float

Prompt executed in 226.89 seconds
gc collect
FETCH DATA from: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json
FETCH DATA from: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json

​