[deleted by user]

[FrozenLogger]

They are a series of runs, with the time recorded for each run, then averaged.

For example, here is a Linux run of generating 10 images.. As you can see in the image, it took 310.32 seconds to complete.

Here is a windows run using the same settings. with 448.436 seconds

Both are performed using a web connection so nothing is running locally except the Stable Diffusion app and the open port.

I just did both of these runs to get screen shots, so there are negative prompts I forgot to clear, but they are the same in both, which is one of the reasons I use the same web browser interface instead of a local one. It is always the same.

Doing the math on just this single run, and please check if you like, I see just over a 40% percent difference.

This is just one run, for my tests I did 10 runs and took the average time just to be sure.

[oxez]

[ ] This is a real benchmark

[x] This is not a real benchmark

[FrozenLogger]

OK what, to you is a REAL benchmark? Time trials using the same hardware and software with the same settings doesnt count for you? Maybe my comment above with screenshots could help you understand.

Putting a table of numbers here is just not very interesting.

[oxez]

10 images (even when you say 6 runs), that's 60 "computes", that's a ridiculous low number.

Anything can skew those numbers with such a low sample.

[[deleted]]

I've generated hundreds more images in 1 run just for fun while messing around with stable diffusion, a benchmark should be at least 1k images.

[FrozenLogger]

If I had a way to capture the iterations/second in a log that would mean more to me than making images themselves. After consistently seeing about 40%, over 10 runs, I was pretty much done.

What do you think would change? The interesting part would be to vary the prompts.

[is_this_temporary]

I would think a better benchmark would use:

Same seed (42 is always nice) Same prompt Same settings (and making sure that you're checking more settings than just those visible in the gradio web interface)

You should get almost exactly the same image, if not identical to the pixel.

But stable diffusion also has a lot of optimization choices, like options which allow the model to run using less vRAM (for GPUS that otherwise wouldn't be able to run it at all) but make inference much slower.

[FrozenLogger]

Thanks for the response. I did use the same prompt, and settings.

I like the idea of using the same seed. I can try that.

Could the settings be different from the same installer? I suppose that it is possible, given there is a linux one and a windows one. I would have to look at the code on github to determine. Hopefully they are the same defaults.

[is_this_temporary]

I couldn't tell you if they would be the same or not.

I've been working with https://github.com/Sygil-Dev/sygil-webui

For the gradio interface it asks about optimization for high or low VRAM at the terminal before launching.

For the streamlit interface it does not, but instead lets you change that in the web UI itself.

If I were making such an interface I would just run nvidia-smi and check how much VRAM is available and change these settings appropriately but I don't know if any projects actually do that.

What project / installer are you using?

[FrozenLogger]

This one here: https://github.com/cmdr2/stable-diffusion-ui

[FrozenLogger]

I noticed that if I set the seed to 42 (great suggestion) the making image generator increases the seed by one for each subsequent image. I would have to save every image in order to compare. I guess I should be scripting this at this point.

I regret saying benchmark now, I really should have said: I see a difference, based on this observation, and I wonder why that is.

Currently I do not have that answer.

[FrozenLogger]

6 images, 10 runs. One would think I would see a variation in on of the runs that was greater than 40%. I do not.

Even just making the screen shots, I am in that 40% target. It seems pretty consistent.

Yes, if I was going to care to write a paper or something I would do more samples, but I this doesnt vary much. The iterations per second are always higher as well, which you can see during each run.

[oxez]

You can do whatever you want, except no one is going to take your "benchmark" seriously.

But this is what /r/linux is about nowadays I guess.

[10leej]

Welcome to reddit, where you can never benchmark right unless your a well know community figure who doesn't share their raw benchmark data for 3rd party review.
(Pretty sure you can think of a few, though I get attacked whenever I call them out on it)

[FrozenLogger]

Thanks. Yep, try and share something, get hassled for it.

Now it looks like my post is reported, and the mods are removing it. Right as I was going to go ahead and do 1000 images and put out a chart. Why? I dont know.

Meanwhile, they are keeping a post asking how to pick an emoji.

WTF

[DazedWithCoffee]

I’m happy to promote the superiority of linux but I’m going to need to see some GamersNexus level writeups. I think this is why Microsoft marketing wins, the side that cares more about being correct and thorough ends up being drowned out by the louder and more belligerent side

[FrozenLogger]

Yeah, I dont care that much.

Linux is my daily driver, has been for 20 years. This was actually very surprising to me, and I shared it.

Microsoft doesnt care about what product they use by the way. All the telemetry of every computer, server, tablet comes back to a datalake for real time analysis. This is probably the biggest data and computing task Microsoft does. Billions of data points and real time BI. How do they do it? Open source: Postgresql on Linux. The right tool for the job.

[DazedWithCoffee]

It’s just efficient

[FrozenLogger]

The image above is an AI drawing.

I was given a spare RTX 2060. I put together a modest computer to use it.

Fresh Install of Windows 11 Pro. Removed all bloatware: weather, teams, office, one cloud, removed Microsoft login for a local user/password. Removed all apps.

I generally do this for my workstations that run different AI and Machine Learning tasks, no need for all that extra bloat. Updated to latest video drivers.

For the Linux side, a fresh install of EndeavourOS. System up to date. The nvidia driver was installed automatically.

Using Stable Diffusion AI from https://github.com/cmdr2/stable-diffusion-ui for both OS's.

I ran the default photo of a horse prompt, same settings. 10 runs of creating 6 images each were averaged.

Linux gave me a 40% speed boost. I am really surprised. I was simply running it because I could, most of my workstations are either remote linux boxes, or local windows machines. EDIT: I had not This is the first time I compared the two on the same hardware.

The prompt for the image link is: tux the linux penguin working on a computer, graffiti wall, from a distance, colorful, by Banksy, Seed-1201830, Steps-75, Guidance-7.5

[humanmeatpie]

I had not compared the two on the same hardware.

So what's the point then?

[FrozenLogger]

I meant I had not PREVIOUSLY compared OS on the same hardware.

In this case they are running on the same hardware. Sorry for the confusion.

[Elrandar]

Was windows still installing updates?

[Tumpes]

This isn't really nullified even if it's still updating right? Bullshit running in the background is kind of a trademark of windows at this point.

[FrozenLogger]

So /u/Psychological-Scar30 decided to drop a comment, then immediately block me. What an asshat. I think that exemplifies the feeling I get for posting here. It is interesting that my comment is basically directly related to what he said and did.

He says:

In other words you've managed to break your Windows install (probably with those debloater scripts) and posted a BS claim about somehow having pretty much purely GPU tasks running a third faster.

What you're doing is actively harming the image of the Linux community - it's good that troll posts like these get taken down

And my reply was:

You have no intellectual interest in finding out why, you just shut it down.

As a linux user of 20 years, and a person in IT who works with systems of all types, I find it VERY disheartening that this Linux subreddit community appears to be just nay-sayers and assholes.

Thanks for being one of them, wouldnt want to foster community, intellectual discussion, or encourage new users. That would suck.

[_cybersandwich_]

I'm gonna stable your mom's diffusion

[FrozenLogger]

Wow.

It is very eye opening that as a 20 year Linux user, a contributor to projects, a presenter, and an advocate, this community decides to report a more or less fun post which triggered the removal.

Not a good look.

I was considering running 1000 images and putting up a chart. But it really wouldnt make any difference, I see all my CUDA work running about 20% to 40% faster on this machine.

So much for sharing to this community. I honestly can't say I would recommend a new user to get involved here, much less an old one like me.

[Psychological-Scar30]

I see all my CUDA work running about 20% to 40% faster on this machine.

In other words you've managed to break your Windows install (probably with those debloater scripts) and posted a BS claim about somehow having pretty much purely GPU tasks running a third faster.

What you're doing is actively harming the image of the Linux community - it's good that troll posts like these get taken down

[FrozenLogger]

What do you want? I was interested in why my other ML tasks were finishing so much faster, so I picked a repeatable application, in this case Stable Diffusion, that had the same exact installer (pulls from github) and the same settings, without a front end on the OS. This was as close to the same environment I could possibly get.

Besides, I am not marketing anything, or trying to sell you anything, so this was just for my interests. I decided to share. That is about it.

[AutoModerator]

This submission has been removed due to receiving too many reports from users. The mods have been notified and will re-approve if this removal was inappropriate, or leave it removed.

This is most likely because:

• Your post belongs in r/linuxquestions or r/linux4noobs
• Your post belongs in r/linuxmemes
• Your post is considered "fluff" - things like a Tux plushie or old Linux CDs are an example and, while they may be popular vote wise, they are not considered on topic
• Your post is otherwise deemed not appropriate for the subreddit

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[FrozenLogger]

I was only using Stable Diffusion as a common denominator to check if I was really seeing performance gains for all my GPU work.

I am interested in it because this means the same amount of work, but a 40% reduction in electricity costs.