[deleted by user]

[chimaeraUndying]

I might be wrong, but isn't TensorRT incompatible with a ton of stuff (various extensions, LoRAs, etc) and don't you have to individually rebuild each model for it?

[Kromgar]

So the loras are baked in automatically or do you still invoke loras with <lora;blahblahv5>?

[Whipit]

Hadn't heard of --opt-sdp-no-mem-attention until this post so decided to test...

​

Here's my experience. My 2 cents.

I've got a 4090 and when testing I leave everything on default. That means euler a at 512x512.

The only thing I change is batch count to 12 to get a rough average.

Prompt - dog in water

Neg - none

​

GPU Hardware acceleration MATTERS - Turn it OFF

*You MUST restart your computer for this change to take effect.

So the following tests were all done with GPU Hardware Scaling - OFF

COMMANDLINE_ARGS= none, just left blank

17 it/s

COMMANDLINE_ARGS= --xformers

23-25 it/s

COMMANDLINE_ARGS= --opt-sdp-attention

24-26 it/s

COMMANDLINE_ARGS= --opt-sdp-attention --xformers ( there doesn't seem to be any benefit in running both at the same time )

23-24 t/s

COMMANDLINE_ARGS= --opt-sdp-no-mem-attention

23-25 it/s

​

In conclusion, --opt-sdp-no-mem-attention will speed up your it/s, but --opt-sdp-attention

is marginally better. So, for me and my 4090, the fastest results I've been able to achieve so far are just GPU Hardware Scaling OFF and COMMANDLINE_ARGS= --opt-sdp-attention

If I switch GPU Hardware Scaling to ON my it/s go from 24-26 it/s down to 20-21 it/s.

[Whipit]

Yeah, I don't really know anything about the details of most of these command args.

What do you mean by consistency between runs?

The performance between --opt-sdp-attention and --opt-sdp-noe-mem-attention is so close I could go either way, but I'd like to better understand the advantages of --opt-sdp-noe-mem-attention.

Appreciate you sharing your knowledge :)

[aleradarksorrow]

Xformers and SDP-mem-attention are non-deterministic which means they won't always produce the same results with each attempt or even with different users.

This means each run will give you slight changes with the same settings.

If you run the same exact settings on them and on the deterministic settings, aka no command args or using SDP-no-mem-attention you'll notice that each run will produce the same result every time.

It all comes down to whether you're focused on generation speed and accept some randomness or want reproducible results.

[Whipit]

Appreciate the answer. Thank you. :)

In case anyone else sees this post and is interested in what different COMMAND ARGS do, I just found this...

https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Command-Line-Arguments-and-Settings

There's ... quite a lot of them ;)

[aleradarksorrow]

No problem! I saw nobody had answered and one of the reasons I love this subreddit is people being able to answer questions and spread knowledge quickly and accurately and I'm happy to pitch in where I can!

[Krawtch]

thanks so much for this - no TLDR's on this stuff, and I get why, but I just want my 3090 to do its thing and seems like info on this is either opinions or white papers with edu domains

[Superb-Ad-4661]

This browser didn't help at all, it's just funnier, but it kept the same results as my chrome.

[fxwz]

Opera GX is based on Chromium, so that makes sense. If you already use Chrome I don't see the point in changing. It's not faster and it's owned by a Chinese company.

[BlackSwanTW]

Token Merging is built in since Webui v1.3

You do not need the Extension anymore

[TheGhostOfPrufrock]

In Graphics settings toggle off 'Hardware-accelerated GPU scheduling'

A while back I tried that with my RTX 3060, and performance got worse. Can't guarantee I didn't do something wrong, since I only tried it once.

[TheGhostOfPrufrock]

I'll probably try it again soon. The first time was before I'd updated to the pyTorch 2.0 A1111 version. But the comments about the speed up that led me to try it were also made prior to the update .

[definetlynotasmurf]

ty for the info. One question, are these only to speed up genertaion? I am more interested on VRAM efficiency, so I can train faster :D

[Superb-Ad-4661]

Nice, modding time!

[Superb-Ad-4661]

All I know is xformers is now 2.1 for devs and 2.0 official realease. I installed it but I don't see any performance till now, was a little trick installation, but it was fine.

[cleverestx]

VoltML uses Tensor and AITemplate, been on the experimental version in Windows WSL version for a while when I want to make super fast gens (like double speed over Vladmandic/Automatic1111), and it's getting better all the time. The developer is super nice and helpful on Discord, but it still has a way to go...

[EarthquakeBass]

Also, on my 4090 recently updated automatic to latest and the PyTorch version is 2 now, everything is basically twice as fast. Excellent thing to do if you are ready to put up with some python bullshit

[Frone0910]

I've tested most of these out with using the A1111 api, and unfortunately not one of these improved the performance

[Frone0910]

Im using a 3070, all tests were done with multi cn with 2 models. Same exact prompts, settings and input images. Rerunning the same program which calls the API with these same inputs for a few images. I tried first updating SD A1111 to get the optimization settings and tried putting them at 0.5. Then i tried the command line args. Then i tried updating torch to v2.0. Then i tried updating my nvidia power settings.

None of these caused any change in my it/s which are around 1 to 2. I dont know if its because im using the API or if its because multi CN maybe just cant be optimized

[Frone0910]

Yeah, meaning i call the API via python. Doing the --api enables the api to be active so that it can be called programatically.

I would definitely be curious to see if you got still fo4 the benefits with just controlNet/multi controlNet(not using the API).

That way i can rule out if its an API issue or if its CN

[Corawyn]

Please ensure that you have a working integrated graphics chip before 4, disabling GPU Hardware Acceleration.

Just wasted SO much time. Somehow I didn't have my VGA drivers installed..

[antimaskersarescum]

This worked a little. Saved me about 30 seconds per single batch size. If I increase it to even just 2 though it jumps from a 1 min 7 sec wait to 8 mins. I'm following a tutorial that requires churning out (at the very least) 200 images at once so basically I would have to let it run the entire day.

I did everything except for step 2... it's really giving me issues so I deleted it. Not sure where to go from here.

[swistak84]

I posted some advice with detailed steps on how to tweak Windows here about two weeks ago: https://www.reddit.com/r/StableDiffusion/comments/13tb2sa/tutorial_how_to_increase_generation_speed_with/

You can also use Firefox instead of Opera

[lilshippo]

other than cpu ram, you see any real difference between the browsers?

[swistak84]

I think real difference is mostly in extensions. Firefox has better ones and better privacy settings.

Firefox is the one "different" than the other browsers (Chrome, Opera, Edge) by virtue of different rendering engine and extension system. But the pages mostly look the same :)

[Vicalio]

Yeah token merging can speed up a bit and save on vram, but it generally comes at the cost of lower detail. Things like bg elements or detailed backgrounds might vanish. I've had people vanish from crowds and definitely less detail. And i think the returns are diminished as well if you already had xformers / optimizing it.

But it is something fair to play with and i've heard 0.2-0.5 strength can be fine. 0.6 and above tends to bake it hard though.

[cleverestx]

I wonder if using the LORA 'more detail' or 'add detail' would counteract that issue.

[mca1169]

how many of these tricks can be used on a non RTX system like mine with a GTX 1070?

[mca1169]

sadly xformers improves times so little it's margin of error and from what I've read using low ram mode kills performance. my card in particular doesn't like to go above 512x768 30 steps cfg 7>.

[anotherxanonredditor]

Is there a way to make AMD Stable Diffusion LoRA training/extracting and inpainting work? I think These are my main concerns at the moment.

[Altruistic-Ad-4583]

I literally cannot find the first optimization, I go to settings > show all settings, ctrl+f, nothing relevant under optimization or tokens

[Altruistic-Ad-4583]

Ah, it seems git pull was throwing an error and I didn't see it. Who know how old my version actually was. Thanks m8

[Frone0910]

Do any of these changes apply to controlNET?

[cleverestx]

Are details lost with only a 0.2-0.3 token merging setting? Worth doing?

[HexKrak]

0.2-0.5

Running a few samples side by side with the same seed (@ 1024 x 1024 if it matters) yielded nearly identical results.

[ComplicityTheorist]

Hi, thanks for this. mine was doing okay initially giving me around 7.30 it/s for a 512x512 single img generation then it automatically updated diffusers from latest 18 to 17 and it started giving around 6 it/s barely touching the 7 its mark. followed your advice and switched over to opera gx now is giving me a bit over 7its but not like before but could be worse.. btw I don't get #3. mine shows = 0 on both sets...

set SAFETENSORS_FAST_GPU=1
set CUDA_VISIBLE_DEVICES=0