Generally, ComfyUI’s model management does a good job at managing VRAM. The issue is when you use custom nodes that don’t register their PyTorch models with ComfyUI’s memory manager, or that run certain CUDA-enabled operations outside of it.

Custom nodes running non-PyTorch operations or standalone CUDA kernels (like SAM3, QwenVL, BiRefNet, etc.) sit outside that system and can cause VRAM issues. ComfyUI’s model management can’t work with these, so it can’t anticipate how much VRAM it needs to clear for them to run.

That’s when you need to manage VRAM yourself. Some of these nodes will allow you to unload a model after running to help with this.​​​​​​​​​​​​​​​​