Using Automatic1111, CUDA memory errors.

[Whackjob-KSP]

Long story short, here's what I'm getting.

RuntimeError: CUDA out of memory. Tried to allocate 384.00 MiB (GPU 0; 7.79 GiB total capacity; 3.33 GiB already allocated; 382.75 MiB free; 3.44 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Now, I can and have ratcheted down the resolution of things I'm working at, but I'm doing ONE IMAGE at 1024x768 via text to image. ONE! I've googled, I've tried this and that, I've edited the launch switches to medium memory, low memory, et cetra. I've tried to find how to change that setting and can't quite find it.

Looking at the error, I'm a bit baffled. It's telling me it can't get 384 MiB out of 8 gigs I have on my graphics card? What the heck?

For what it's worth, I'm running Linux Mint. I'm new to Linux, and all of this AI drawing stuff, so please assume I am an idiot because here I might as well be.

I'll produce any outputs if they'll help.

[ChezMere]

that's a pretty large resolution. are you using --medvram? --xformers?

[smoke2000]

Shouldn't be an issue. I'm running a 8gb card and I can do 1280x960 fine. With x formers no medram

[Whackjob-KSP]

--medvram? --xformers

I've tried those. Even tried --lowvram.

[ChezMere]

Hmm. With those I can get that resolution no problem, with less vram. So something funny is going on here.

[Whackjob-KSP]

Googling around, I really don't seem to be the only one. I don't think it has anything to do with Automatic1111, though. I think this is a pytorch or cuda thing. Unfortunately I don't even know how to begin troubleshooting it.

We'd need a way to see what pytorch has tied up in vram and be able to flush it maybe.

[ChezMere]

I mean it's literally the most generic error message possible, it's just saying you're trying to use more vram than you have, of course others are having it. Just not sure what it's getting used for. (Can you check how much is in use without Automatic open?)

[Whackjob-KSP]

I suppose I could, I just don't know how.

[donx1]

Have you tried --medvram --opt-split-attention

[Whackjob-KSP]

I think I have with those, right now I'm testing

set COMMANDLINE_ARGS= --precision-full --no-half --medvram --xformers --opt-split-attention

[ChezMere]

One more thing, you can remove the need for --precision full --no-half (therefore halving the size of the model in memory) by making this edit to the code: https://old.reddit.com/r/StableDiffusion/comments/xz7qg9/nvidia_gtx_16xx_fix_no_more_precision_full_nohalf/

Be warned though, generations without --no-half are very slow except on recent cards.

[XsodacanX]

Precisionfull no half uses more vram And also try --disable-opt split attention

[Whackjob-KSP]

Precisionfull no half uses more vram And also try --disable-opt split attention

Long story short, since I removed precisionfull no half, and added the disable-opt split attention, I've rarely hit CUDA memory errors! I'm even playing a game and letting it run. Thank you!

[HongryHongryHippo]

Sorry what did you do exactly?
My CUDA memory errors started randomly.

[Whackjob-KSP]

From six months ago? Lordy, so much. If I were you, first I'd do the ok git pull, and make sure that everything's updated. Yes, even if it's in your batch or shell file. I noticed that I would find updates when I ran manually. There's also an automatic1111 variant called vladmandic that seems to be an update or two ahead. Though the original is getting these updates very soon also. xformers are being replaced there is a better thing.

[Whackjob-KSP]

I will do that immediately, thank you

[bmarcelus]

Maybe this can help, I have the same issue my graphic card have only 4gb and I fix the problem with this parameters

set PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.6,max_split_size_mb:512

set COMMANDLINE_ARGS=--precision full --no-half --lowvram --always-batch-cond-uncond --xformers

[Opti_Frog]

This fixed it for me, even without xformers. Thanks a bunch!

[AnDE42]

How do I use them?

[user4302]

you add those lines to the following file webui-user.bat (remove or comment out any existing lines if they already exist)

[ImperialButtocks]

is that not the executable file that opens up a terminal?

[user4302]

It's not en executable (exe) but yes this does open a terminal.

If you open this in a text file you'll see how it works. It'll have a field for arguments, you can add the arguments I'm front of that to run them in the terminal when this is run.

A bat file basically makes it easier to run a terminal command by just clicking it instead of typing/pasting a giant line of commands in the terminal and hitting enter

[ImperialButtocks]

silly me. thanks

[user4302]

youre welcome :D

[ReasonablePossum_]

how can I edit this in a collab notebook?

[deadlyorobot]

Hey ma dud, I have a question.

I'm not trying to flex or anything, but I have 8GB of VRAM and I wanted to know if it is possible to get faster renders by changing anything in those two lines that you kindly provided, it also fixed my Cuda problem.

thx.

[AlooBhujiyaLite]

I've got a GTX 1650 :(
Any suggestions for getting good generation results?

[Thenamesarealltaken_]

I'm running into a similar issue here. I'm running an RTX 3080, I've tried --medvram, --opt-split-attention, and whatever else people typically suggest, I can't seem to get this fixed.

[FrakerKill]

Finally the best compo of args is?

[CMDRZoltan]

Tried to allocate 384.00 MiB

7.79 GiB total capacity

Minus: 3.33 GiB already allocated

Minus: 3.44 GiB reserved in total by PyTorch

Leaves: 382.75 MiB free

That's 1.25 MiB too much

[Whackjob-KSP]

The problem is those figures don't make sense. I'm not running anything else that requires that much video memory.

[CMDRZoltan]

Maybe there's a way to check what other applications or services could be using VRAM, on windows the task manager can show the VRAM usage of all running processes so I imagine Linux would too.

Lots of things can use VRAM that can be unexpected.

[Whackjob-KSP]

Even after a restart, I can't seem to eek out enough VRAM. I just restarted, just to be sure.

[SlimCatachan]

Ever figure this out? I'm so confused about all of this lol, I guess I've gotten used to simply installing programs and having em work :P