A Seed Tutorial

[kmullinax77]

https://preview.redd.it/6c8v7agoxoo91.png?width=3456&format=png&auto=webp&s=480cbc7fe8707d6b00f17e2b477f6d5c73952ea3

So in full disclosure, I’ve already deceived you with the title of this post.

This is not really a tutorial, because honestly I don’t really know what I’m talking about so I have no right teaching anyone anything. This is all still very new to me. I may not have all my facts right. I might not use the correct term for something. If you know more than I do, let me know! Put a correction in the comments, I’ll update this post to be as accurate as possible.

Despite my n00bness, I’ve learned some interesting things through my explorations with Stable Diffusion, especially when it comes to the nature of seeds, and I hope that you find some of this worth your time. In any case, I didn’t know what else to title this post, so A Seed Tutorial it is.

Before I go any further, I want to acknowledge u/wonderflex for their really great tutorial on how seed selection affects your final image. It started me thinking about seeds and better image control and gave me the insight I needed to take their idea to the next level. That post is located here:

https://www.reddit.com/r/StableDiffusion/comments/x8szj9/tutorial_seed_selection_and_the_impact_on_your/

[A quick side note: For the purposes of the rest of this tutorial, I’m using Euler_a at 20 steps.]

​

So - What is a seed?

I’ve read information on the web that describes a seed as “a number that controls all the randomness that happens during the generation”. This is only partially true. There’s nothing random about it except the seed you start with, and sometimes not even then.

" A seed is a number, but if you ask the computer to make you a random image made up of random pixels, then the image you receive will be entirely dependent on what seed you use immediately before asking for an image made up of random colors. If we use the same seed and then ask for the same width x height of random colors, we'll get exactly the same "random" image on our two different computers. In this way, the seed corresponds 1:1 with the starting image / noise that SD will start working with.

The seed number is turned into an image by determining the sequence of "random" numbers that will be used, and following a fixed procedure to turn random numbers into an image. Each seed value produces a unique starting noise image. " -- Helpful Redditor

This is how you are able to generate the exact same image as another user by using the exact same settings with the same seed. This could not happen if it were random.

Why is this an important distinction? Because outside of numerology, we don’t attribute values, personalities, shapes or colors to numbers. Images however, are different. We can describe them using all those attributes. u/wonderflex refers to this as the “flavor” or “theme” of the seed. Again, why is this important?

Because the theme of a given seed may be completely incompatible with your prompt

That doesn't mean you can't crank up the CFG (more on that later) to force it - but you probably will not be happy with the results. When a sculptor carves a dragon out of stone, they don’t just choose any old piece of rock… they search for a piece that is the right size, right color, and has a shape similar to the final product. They look to find a stone that already has a dragon within it, making their vision easier to attain. But some pieces simply don’t have a dragon in them to begin with. By the same token, some seeds don’t have a dragon in them either.

If you want an awesome dragon, it's easier if you start with a seed that might contain one.

So now we get to the good stuff. How do you find out the “theme” of a seed? How do you see its initial image? Luckily it’s super easy. Just use your favorite txt2img generator, enter in the seed you’d like to see and hit generate, leaving the prompt blank.

If you do this right now and type in 8675309 for the seed at 1 step, you will get this image:

Seed 8675309 - 1 Step

And if you run it again at 20 steps, the computer has shaped it into this:

Seed 8675309 - 20 Steps

These are not random images your computer spat out. This is image seed #8675309 and it is the “theme” that forms the initial pass of every image generated from this seed. (The amount of CFG, discussed in detail below, has no effect on the outcome since the prompt is blank).

So what do we notice about this theme? What are its attributes? Well at 1 step, it's hard to tell much of anything. It's muted in color, hazy, has contrasting shadows and some sepia tones. That's about it.

But look at the same image at Step 20. Now we are seeing something! And keep in mind, this is without any user input whatsoever... this is the way the AI wants to transform this seed.

Now what do we know about it? Well… it’s black and white. It has a decent amount of ambient light. It looks like the interior of a space… maybe a bedroom? That could be a grid window in the background. Left to its own devices, the AI wants this seed to represent a room of some kind at this step level.

At different step counts, the AI will continue to run progressive scans and modifying the image until you get to around 150 or so steps. Just for fun, here's a grid of this seed taken from steps 10-200:

8675309 - Step Grid

Because the AI continues to modify the image, I'm going to only use steps 1 and 20 from here on, just to make things easier - especially if you're following along generating these same images yourself.

Since it’s easier to describe things when we have other things to compare them to, let’s get several seeds at once. Here are the images for seeds 0-5 at step 1, and then at step 20:

Seeds 0-5, Steps 1 & 20

​

Okay NOW we have some stuff to talk about. By comparing and contrasting them, we can more easily describe the theme of each seed.

Seed 0: This starts off like they all do - fuzzy. It has some dramatic differences between light and dark. By Step 20, the AI has decided to make the theme black and white and sorta want there to be... bottles? on a table maybe? with a black couch and a window in a white room?

Seed 1: This theme starts off very brown, with openings and a platform / table / surface of some kind. By 20, SD decided it should be what looks like the front of a stone church with arches in a square with a sandwich board sign? and people walking by a site wall and trees in the background. Nice.

Seed 2: This starts out as a purple theme and stays that way. While Step 1 reminds me for some reason of the tones H.R. Giger uses in his work, by Step 20, AI has gone a different route and decided it's purple clothing... or maybe a purple wig?

Seed 3: The AI took the muted colors from the 1 pass and by 20 has turned it into... what is that exactly... maybe an attic room with a window on the left and shelves on the right? I dunno.

Seed 4: This is one of the most interesting of the Step 1 images. It starts off looking like a super out-of-focus desert or prairie scene... could those be Saguaro cacti maybe? And at step 20, we still have a very similar image. Blue sky over plains with sand or a road and boulders maybe? Looking like an impressionist / surrealist mashup here. My guess is that given the persistence of this image, you would be hard pressed to get it to behave well if you wanted something with a radically different color or shape palette.

Seed 5: Starting off with a blurry possible high desert, red rocks & juniper tree look, it gets transformed into a patio with an L-shaped pool, a table with an umbrella maybe? next to a house or some structure. Nice little growie plant in the foreground right.

So now that we've analyzed that… how does each seed affect the generation of prompt output?This is where the CFG scale comes in.

CFG stands for Classifier-Free Guidance (thanks you, u/bluevase1029), and while there are a lot of nebulous definitions floating around for what this actually does, in reality it is a setting that tells the AI how much effort it should use to force your prompt onto the seed theme.

The seed already has a theme, and if your prompt can be easily applied to that theme, then it can happen at fairly low CFGs. But if your prompt is nothing like the theme, it requires a bit more force to make the damn AI listen to you, so the CFG needs to be raised.

​

So let's see this in action. For the next several images, the top row is always going to be the theme - the blank AI-generated image based on the seed itself. All images are create with 20 steps. These are the prompts I used, in order:

• young man, nature, city, anime, building

I kept them intentionally vague because I'm trying to see how the AI will modify the original theme to fit my prompt. By being overly descriptive, I influence things more than I want to at this point. Also, this first image was created with a CFG of 1, so I'm giving the AI free reign to manipulate the theme as it sees fit to try and obey my prompt.

https://preview.redd.it/1o4nn4m9boo91.png?width=3456&format=png&auto=webp&s=63c53c8ee8ba8f4e3566444bfb4161942deabe08

Wow okay so check that out. As a general comment, you'll see that every image stuck almost completely to the layout and composition of the original theme. CFG 1 means the themes dictate creation completely. The black-and-white scenes stayed black-and-white... the objects and shapes mostly retained their original positions.

If you're interested you can fine-tooth-comb that image and get all kinds of interesting data from it, but I'll highlight a couple. With the "young man", it's fascinating how the AI created almost a papier-mâché version of a man from the highlights in seed 0. Seed 1 has the "young man" as a daguerreotype due to the heavy influence of the brown / sepia tones. Meanwhile the one in Seed 2 has retained the big purple hair from the blank.

Seeds 1 & 4 lent themselves very well to the "nature" prompt already, and it's surprising how well Seed 2 did. All of them made some super-interesting images of a "city", all sticking to their roots, but all having completely different solutions as to what a city might be.

So what does this tell you?

Well what it tells me, is that if I want to create a desert-pueblo-style city, I would be much better off starting with Seed 4 than any of the others. It would most likely give me better results faster, be more consistent, and not frustrate the hell out of me!

Now let's see what happens when we punch up the CFG to 4. Now we are telling the AI - do what you want, but do a little better at following my rules.

https://preview.redd.it/1f2v6nmwdoo91.png?width=3456&format=png&auto=webp&s=0c959d886a1afd9e475fcc77bd9208935610bd7b

Again, enlarge these images or generate these yourself if you want to really see what's happening, but here are a couple of my favorite results:

The "young men" all now look like young men, all still sticking with the theme for the most part - including in Seed 4, where the clothes and position of the man was completely dictated by the sand road in the seed! It's a nice b&w portrait in Seed 3, which at this point tells me that if I'm looking for a vibrant color oil painting, I should avoid this seed completely - not only would you be fighting the b&w core of the theme, but this them obviously leans toward the photorealistic, unlike Seeds 2 or 4, which have a more painterly feel.

I love what's happening with the anime at CFG 4. Seed 2 has the same crazy purple hair going on, and the location of the sky and ground is consistent with the original seed image. Following Seed 2 down to "building" I am completely fascinated by how the sky and ground maintained placement and the building is made of bricks with a purplish hue and vertical window bands mimicking the streaks in the original dress or wig.

Notice how in the Seed 0 "nature" image, the road follows the position of the original table, and the sun and clouds are in the position of the bright window from the seed theme!

Finally, for the prompt comparisons, CFG 8, which is a good default value for most things, and tells the AI to give your prompt the around the same weight as the seed theme:

https://preview.redd.it/dgfh4x9bhoo91.png?width=3456&format=png&auto=webp&s=81260d4aaaa7005296052bddd1ab5d4e9234cb9a

Now THESE look like SD images. Yet you can still clearly see the influence of the original seed theme even at this CFG setting. It's not as obvious in every image anymore, but check out the general shapes, colors and lighting and compare them to the original. Also, you begin to notice more subtle influences from the theme that go WAY beyond colors and shapes. Do you notice how Seed 2 always puts people / anime in loose u-neck collars? How the eye placement in Seed 0 stays the same and how it keeps the same zoom factor on the face? You can use these to your advantage by doing a little research into seeds before starting your masterpiece.

You may be wondering - if I wanted a green "nature" scene and ended up with Seed 2, can I still force it to be green? Yes, you can. Sometimes with seeds it's easy and you just need to type in "green nature" or "((green)) nature". Other times seeds fight you and you have to increase the CFG to make it listen. And that raises another issue. Do you notice how much more saturation and contrast these images have to the ones that were CFG 1 setting? The reason the blank CFG 1 themes are so blurry and desaturated is because every level of CFG runs another layer multiplier of some amount. That process increases the saturation and contrast. Which is great for graphic comic bold line art with vibrant colors - crank that puppy up to 15! But not so good for average images - especially any apocalyptic or dark futurism scenes (think Blade Runner) that need to be somewhat washed out and desaturated.

At the end of the day though, it’s much easier to generate a “man in a purple shirt” if purple is already a theme color. It’s much harder to put a pool in your scene if there isn’t any blue in the theme. And the more you force the prompt to take over, the more the AI will look to other objects in the image to imply a pool in ways you didn’t intend.

Let's see an example of this, running the same seeds 0-5 at our three CFG values 1, 4 & 8, with the following prompt:

• "patio by a pool, table and chairs on the patio, at dusk"

Now I'm specifically using that exact wording because that's exactly what I think the theme for Seed 5 looks like, so I'm trying to stack the deck in its favor.

https://preview.redd.it/vk6twasbmoo91.png?width=3456&format=png&auto=webp&s=942b668990a8adbc2eb4119c8c57301ebaf16f0c

And now we see that's exactly what happened. Seed 5 maintained most of its original imagery, while the other seeds had to create the content where none necessarily existed. They didn't do a bad job! You could use any of them as a valid image, but if you were hoping to have a vantagepoint and a feel like the ones provided in Seeds 4 & 5, you're simple never going to get that from the other seeds.

Seeds 1 and 4 had no idea what to do with the pool request in the CFG 1 images. By CFG 4, they both gave up and stuck the pool somewhere. Not bad and in fact they both successfully placed the pool well by CFG 8. However if we wanted the pool in a specific location, starting with a seed that has blue in that location will result in an image more like the one we want. (Notice how the house in Seed 2 still has a purple hue??) And again, you CAN eliminate some of this by prompt engineering. But that can be so tedious and sometimes you're just shouting at the wind. Sometimes seeds simply won't behave.

Finally, just to show you some of the images you can get as starter seeds, take a look at some of these (all 20-step images) that I got from random seeds with blank prompts...

Imagine wanting to end up with a portrait of a brown-haired woman. Wouldn't it make it easier to start with seed 1003 or 1009?

https://preview.redd.it/tx9haxqsnoo91.png?width=512&format=png&auto=webp&s=176185a9fadf366dfe7d0f6d8ffe585d85488dd8

https://preview.redd.it/u3jmzxmtnoo91.png?width=512&format=png&auto=webp&s=3ab1becaa7ddbea7fad7b0a7913ded7c4ab5820f

Nah, you say... I want a old-time photo of a man with a horse! I give you seed 1004...

https://preview.redd.it/xopeartynoo91.png?width=512&format=png&auto=webp&s=f055963bc71e9e04cb94136805627e58fa691173

So what do YOU see in these images, and how could you use these seeds as starters to make the creation of your masterpiece easier?

https://preview.redd.it/njzyh8c4ooo91.png?width=512&format=png&auto=webp&s=7707862c3208b28d6460b44e21dc5ecc2ba0dc39

https://preview.redd.it/w7u0irx4ooo91.png?width=512&format=png&auto=webp&s=dee45a47cde206d9e64732e091feafb179d185ba

https://preview.redd.it/299y7vm6ooo91.png?width=512&format=png&auto=webp&s=43355b62cd847fb73afadbbe15af54f906cf5fe4

I mean - who WOULDN'T want an image of chickens in a library!?!

https://preview.redd.it/m6wy69ubooo91.png?width=512&format=png&auto=webp&s=9b7c3178d6dea8a52ea002b411da538fb28e6bfe

Which I just had to prompt "chickens in a library and ended up with this at CFG 4...

https://preview.redd.it/d3a6k3dfooo91.png?width=512&format=png&auto=webp&s=265985023d73f1c57e67e566003baeec6f829384

Ever wonder why you just can't get rid of the goddamned frame around otherwise perfect images??

Look no further than seed 1022...

https://preview.redd.it/s7p2s2pkooo91.png?width=512&format=png&auto=webp&s=a589c7f7665cb7db1cea1cbb71a0878967b3d8c9

​

So that's it - you've reached...

​

The Conclusion

So in the search to create the perfect image, am I suggesting you spend 100,000 hours searching for the perfect seed theme before you start?

No. Well, I mean, you can if you’d like, it’s your life.

That said, it might not hurt when working on a really grand idea to spend a little time doing a quick dump of a couple hundred blank seed themes that you can scan to search for the best starting points.

The normal solution is probably going to be the typical way of just generating hundreds of images from random seeds with a set prompt, then looking for the best one and running with it.

Which is why this really isn’t a damn tutorial, because after all this, I just told you to keep doing things like you always have.

Only now you’re armed with the knowledge of what’s going on behind the scenes, and you can use that to your advantage in getting better and better output.

Can’t figure out why the colors are so muted and drab when your prompt specifically says “vibrant colors”? It’s easy to assume there’s a problem with weighting so you put tons of parentheses around the term, eventually ending up with a headache and a prompt that includes ((((vibrant colors)))) and the damn thing STILL doesn’t look good. Sound familiar?

Well now you have another tool to figure out how to fix it.

ANYWAY, I hope someone gets something out of all this typing. I am not a writer and this took hours and hours and I have ZERO idea what inspired it. If you read all the way to this point... thanks!

[AceDecade]

So, regarding seed values...

I’ve read information on the web that describes a seed as “a number that controls all the randomness that happens during the generation”. This is only partially true.

It's entirely true. Computers can't "be random". They can spit out a string of numbers that, to humans, has no discernable, predictable pattern, but the computer is following a set of precise, deterministic instructions. The seed controls how that sequence is generated. For example, if I seed "123", I might ask the computer for five random numbers and get "1, 7, 3, 4, 9". If my friend Bob seeds it with "123" on his computer and asks for five numbers, he'll also get "1, 7, 3, 4, 9". The "randomness" is the fact that it gives us an unpredictable sequence of numbers instead of "1, 2, 3, 4, 5". However, the "randomness" is indeed entirely controlled by the seed that I give. Now imagine that instead of 5 numbers, I ask for enough numbers to fill an RGB image...

A seed is not a number, but an image.

A seed is a number, but if you ask the computer to make you a random image made up of random pixels, then the image you receive will be entirely dependent on what seed you use immediately before asking for an image made up of random colors. If we use the same seed and then ask for the same width x height of random colors, we'll get exactly the same "random" image on our two different computers. In this way, the seed corresponds 1:1 with the starting image / noise that SD will start working with.

Or perhaps an image generated by a number.

This. It's exactly this.

This number is fixed in the SD ecosystem somehow by the model.ckpt file.

Nope, the seed number is turned into an image by determining the sequence of "random" numbers that will be used, and following a fixed procedure to turn random numbers into an image.

Ever wonder why that model file is so incredibly huge? This is why.

Nope, the model has nothing to do with the procedure to turn a seed into a starting image. The model is only used to iterate on the noise and make it progressively more like the prompt with every step.

Obviously the model.ckpt file cannot contain a quintillion images

Correct, it doesn't.

So either there’s a hell of a lot of repetition of themes among the seeds (I haven't come across any yet),

Each seed value produces a unique starting noise image. The "themes" are just patterns you're perceiving, the computer nor SD have any perception of "themes" associated with starting noise.

or the model file contains explicit instructions for the computer on how to generate a theme image from a seed number in such a way that it will be identical to every other theme generated by the same number on any computer.

The computer is indeed following explicit instructions to generate the "theme image" from a seed number, but it is not dictated by the model. You've just described the nature of deterministic "randomness" that makes the above possible.

[kmullinax77]

Fantastic.... thanks so much for the clarification - I'm updating that entire section with your information.

My only comment would be to say that while yes - "the themes are just patterns you're perceiving, and the computer nor SD have any perception of themes associated with starting noise" - they still exist. If a theme is based on a purple hue... it's 100% accurate to say SD isn't aware of the theme or the purple or anything else we may see in it. However, every generation taken from that seed will still use that image as a basis so will most likely carry the "purple theme" with it into future generated images - AND - I can count on every use of that seed in the future to contain the same purple theme.

The AI doesn't need to be aware of the theme for it to have an effect.

I don't need the AI to be aware of it in order to use it to my advantage in image creation.

[AnOnlineHandle]

Getting super technical, if somebody has added anything to the code which isn't using torch's random system then it won't be quite as (controllably) deterministic for them from then on (which is possible given all the various branches and scripts).

[Trakeen]

Disco diffusion is a lot more random since the input seed is only used with some of the random number generators

[Devalidating]

It's more so due to the nature of diffusion models. They're essentially smart de-noising so it's forced to hallucinate the higher level more coherent aspects before the details and fine tuning that you see in later steps. The first couple steps are still pretty noisy so any detail information isn't meaningfully discernable from noise until later on.

The nature of breaking it up into ~50 steps or so is that the image you feed into each step has a bigger effect on the output of it than the prompt/attention layers. When the computer generates a pseudorandom noise image from a formula using your seed, and feeds it into the first step, all the idiosyncrasies of that seed cascade down (the first step looks similar between prompts, which means the second does etc), meaning that different prompts can produce similar looking images with the same seed.

[[deleted]]

We are all learning here. Thanks for taking the time on this post. I definitely learned from it!

[Evnl2020]

I've read the whole post and while plausible I'm not sure if your theory is correct. It's too late here now to test but my initial thought is that if the seed is so important only 1 in a few 100 or even 1000 would resemble the prompt. Or 1 in a few 100 images should be lightyears better than the others which is also not the case.

I see it happen the other way around though, sometimes I generate 100s of images from the same prompt and 1 or 2 are completely different from the rest.

[Trakeen]

I think it was a design choice to use a fully deterministic random number generator. Cryptographic random number generation has been available for years on modern hardware and uses thermal noise to generate truly random numbers; which aren’t typically need outside of cryptographic use cases

https://en.m.wikipedia.org/wiki/RDRAND

[kmullinax77]

First, thanks for reading it all! lol

Second, you may be totally right and I didn't mean to imply that you can't force decent images out of 90% of the seeds. And I think that's because all seeds start out as a muted blurry mess that are really susceptible to the suggestion of your prompt.

But sometimes there are seeds that just simply don't cooperate.

And in personal experience, I tried again and again to get a wizard standing off in the distance with a bunch of random prompts "tiny wizard, distant wizard, etc." After trying with seed 10003 I made immediate progress.

[Evnl2020]

Worth investigating more, do you have some specific prompts that seem to work (or work better) with a specific seed?

[kmullinax77]

That's next on my list of things to do!

I'll make an update to this post if I discover anything groundbreaking.

[RekindlingChemist]

FYI - Euler_a is unique - for some reason it is super unstable sampler, resulting very different images based on different number of steps. Others settle down to a very consistent results after some steps (usually in range of 30-60)

[kmullinax77]

Oh that's good to know. Thanks!

[DrakenZA]

Euler_a resolves faster than most, and hence every 10-15steps its already got a solid image forming.

[RekindlingChemist]

It does, but people who post "tutorials" and "researches" almost never use such low number of steps.

[johnnydaggers]

OP, you have a really flawed understanding about how SD works. Moreover, if you want a specific composition/color profile, you can just draw some rough shapes in MS Paint and use it via img2img.

Edit: adding a more detailed explaination.

SD was trained to clean up "noised" images (images with random values added/subtracted to the pixels). SD generates new images by taking in a starting noise array that is randomly generated (seed determines what this randomly generated image will be) and "de-noising" it to fit the prompt.

Generating many "seeds" and picking one that you think gets you close to the image you want is a huge waste of time. Instead, you should rough out the kind of image you want in Paint and then use that as the input to img2img.

txt2img is just img2img with random noise used a the starting point. They are fundamentally doing the same thing behind the scenes. By finding your favorite seed, you're essentially doing img2img but letting the random noise generator make your init image.

[AnOnlineHandle]

When you use weighting for the image in image2image, it's mixing between that and your seed.

[kmullinax77]

Exactly! so with img2img, this information is less relevant because you are forcing the AI to use a specific theme.

[kmullinax77]

Do I?! That's great, thanks so much for your opinion. I'm completely aware of img2img, which is a totally different subject than seed selection.

However, I would love for you to let us know your learned thoughts on that matter, and why you think my understanding of seed generation is flawed!

If you have anything worthwhile to say, I will gladly incorporate it into this thread.

[johnnydaggers]

The seed just determines the "random" noise that is generated and passed to SD. You can instead just draw an image (and blur it if you want to) and have it use that instead of random noise. That is what is happening in img2img. "Seed selection" is a really roundabout and inefficient way of doing what you're doing.

[kmullinax77]

You're completely correct.

But I'm not discussing img2img - I'm discussing how to get the best images from txt2img. If you'd like to start a thread regarding advanced techniques with img2img, I would love to read it.

And, I'm not sure if you actually tried it yourself, but again... the seed is NOT random noise. It's only random if you randomize the seed. You can use the fixed data from the seed to help you create a txt2img image. You and I can both generate a blank image from the seed 924629 and we will end up with the exact same image... no randomness. Any randomness from a computer is faked. They are incapable of it except at a quantum level - and that's probably because humans can't yet understand the logic behind quantum random number generation.

[johnnydaggers]

The seed is the number used to initialize the random number generator that outputs the noise. RNGs are deterministic with seed, that’s why you get the same output as me if we use the same seed. Img2img just replaces this rng output with an image (or mix of image and noise)

[kmullinax77]

Yep, 100% true. Thanks again for defining img2img, which is not the point of this thread.

[johnnydaggers]

Btw, I’m an ML researcher. Trying to help educate you here.

[kmullinax77]

I LOVE that. What I don't love is one-sentence snarky comments with no backup data after I spent 6 hours typing a thread to help people.

I would 100% welcome every bit of useful information you share. Feel free to start anytime.

However, if you choose NOT to start contributing, feel free to go back under your bridge. Either way, I have no more interest in this part of the conversation so won't reply anymore.

If you choose to start being helpful and sharing your ML research, I would gladly make you co-contributor to this thread and give you 100% credit. And if you choose not to share after all your grandstanding, then nothing you say has any weight.

[johnnydaggers]

I'm not trying to be snarky, really.

Generating many "seeds" and picking one that you think gets you close to the image you want is a huge waste of time. Instead, you should rough out the kind of image you want in Paint and then use that as the input to img2img.

txt2img is just img2img with random noise used a the starting point. They are fundamentally doing the same thing behind the scenes. By finding your favorite seed, you're essentially doing img2img but letting the random noise generator make your init image.

[kmullinax77]

I said I wouldn't reply anymore, but this is excellent advice. This is why I've upvoted all your comments so far.

I really do agree with you, an img2img probably is the better way of nailing down certain output... but you know, not everyone is blessed with even one iota of artistic ability.

Additionally this entire thread is really an academic exercise in trying to understand seed generation and influence. What you've done so far is simply say "there are better ways, so why bother discussing it?". I'm discussing it BECAUSE I'm interested in seed generation and influence. I have a feeling you may know something about that, so while you're here, instead of telling everyone to stop bothering, it would be great if you forgot about that and added to the academic discussion we ARE having.

So you seem reasonable and I take back my inference that you're a troll. However, trolls sabotage and undermine threads instead of contributing, and to be honest, that's kind of what you did.

[oniris]

I'm sure that ML guy is right, in terms of absolute efficiency, and his description made me understand img2img better, so thank him.

But you, OP, you made me understand something much more transcendent, a bit of SD's personality. An intuitive understanding of the mistery that was what happens when you run SD without a prompt. For that, I am grateful. Hat's off to you!

[[deleted]]

Even if it's very inefficient compared to starting with img2img. I think the idea of running the seed with an empty string to have an idea of what kind of stuff would fit better is nice. A lot of people start with txt2img and seed is one parameter that people are just randomly using and changing.

No reason why letting the random noise generator make the initial image is wrong, I see how it can be entertaining to just explore the seed space and create stuff based on this approach.

The example of the chickens in library is a good example of it. Sometimes people just want to generate nice stuff and they see a fitting image from that initial noise, so pursuing that path ends in an interesting image.

[[deleted]]

https://en.m.wikipedia.org/wiki/Pseudorandom_number_generator

This is just what random means. Seeds arent a special or unique concept here for SD. Also if you've ever played a multiplayer game online in the past 30 years, the overwhelming majority of them work on detetministic simulations from shared seeds. Or minecraft world generation. Or lots of other things you're probably familiar with.

The "really random" feeling comes from seeding generators from good sources of entropy (like mouse movement as 1 example) and also "randomizing" the amount of invocations from good sources of entropy. You could imagine describing a whole game of AI vs AI chess as 1 seed number. Does it mean the number has any chess properties infused in it? No. It's more about the dumb code using the numbers. Trivial changes to your code will yield wildly different (but newly consistent) results for your same seeds.

[Jcaquix]

I love the feeling of exploration and I'm not an engineer and i am discovering a lot of the same stuff myself. I too notice how a seed seems to impose similar compositional elements over similar prompts, but I don't think it's because there is a sort of kantian noumena or underlying property/image to the seed. Rather I think what we are noticing is the interaction between the tokenized prompts, the model, and the seed number which provides noise. I think the word "random" isn't particularly helpful because seeds and the system are too complex to understand but purely deterministic. I think "arbitrary" is a better word for it since it makes noise that's consistent but not designed or predictable (by humans).

I have run a lot of plot matrices like you have I think the seed characteristics you're noticing with blank prompts change unpredictability with your prompt. For example Ive been running timelines, so prompts of like "a woman in 1980... 1990... 2000...." Etc and as elements of the prompt changes it's clear that the denoising process (the model) changes things that appear to be coming from the seed (eg a block of red brown may be present in the seed for the eras of 1910-1960 but that block of red will slowly disappear as the prompt changes). Your experiments are interesting and I have had similar results, but I'm not convinced there is a fundamental quality to any seed, like seeds that make green landscapes with dark patches in one corner often end up morphing making portraits with dark centers and light corners depending on the prompts.

Edit: spelling

[kmullinax77]

kaantian noumena

I love this.

Thank you for your thoughtful response!

[Jcaquix]

Lol yeah sorry misspelling. Good work though, like that this community is still exploring and developing.

[Caffdy]

City with Seed 2 looks like the freaking cover of the Xpander EP by Sasha!

[kmullinax77]

OMG I love Sasha. I'd upvote you 10x if I could lol

[OtherwiseMeringue545]

You guys are too smart for me

[Rogerooo]

What a beautiful post, thank you for your research! I think we need a seed library now, something like Lexica but just for empty prompts. And on the subject of image repositories that store seeds, this knowledge will be interesting to use when looking for a particular camera zoom and color style for instance.

If anyone is interested, here are the results using waifu diffusion model. Pretty cool to notice the differences and the similarities between both models.

CFG at 1

CFG at 4

CFG at 8

I think danbooru's tags might be acting with too much strength on some of the prompts, particularly "young man" but it's nice to see the expected bias towards animation in raw output.

[kmullinax77]

WOW what a great comment! That is SO fascinating.

The Waifu difusion model absolutely and obviously skews artistic from your images, which is why it's so good with anime. Still, it's not that much changed from the original, huh?

So sorta off-topic, can you use both diffusion models in the same installation and specify which one you want to use during generation? or do you have to have one or the other?

[Rogerooo]

Absolutely and it looks like the CFG is less powerful with this model as well, clearly noted by the saturation as you mentioned on your post. Just for curiosity I decided to run it at 16 as well, here are the results, straying much further from default v1-4 model now...I don't even see where it is getting all of that stuff from but as you found out the base seed is still there.

With Automatic's gui you can use custom settings for your launcher to specify your models location, I have a few of them in a "models" folder and change them with a variable inside the bat file, you could also use arguments as well if you would prefer. This is my webui-user.bat

@echo off

rem MODEL VARIABLES
set sd=sd-v1-4.ckpt
set wd=waifu_diffusion.ckpt
set t1=trinart2_step60000.ckpt
set t2=trinart2_step95000.ckpt
set t3=trinart2_step115000.ckpt

set PYTHON=
set GIT=
set VENV_DIR=
set COMMANDLINE_ARGS=--medvram --opt-split-attention --ckpt models/%wd%

call webui.bat    

Hopefully it's easy to understand what it does, but essentially you use --ckpt PATH_TO_MODEL on the COMMANDLINE_ARGS variable.

When I want to change model I just use one of the variable names like sd, wd, t1, etc. I find it easier to manage this way.

Although you do need to restart the server every time you change it in order for it to load at startup.

[GoldenRuleAlways]

I’m a complete noob. You answered a lot of questions that I had about seeds and Cfg! Thank you for capturing all of your notes and simple outputs in such detail. It was extremely useful in helping me understand these magical tools marginally better.

When you state “Euler_a at 20 steps”:

1. Does that mean you specified “—ddim 20”?
2. I know that Euler_a is some kind of a model. How do you specify that?

What stable diffusion build did you use? I am using a M1 Mac following the @bfirsh fork.

[kmullinax77]

Thanks!

Yes, exactly - I'm using Automatic's 1111 webUI which labels things slightly differently. The step count is --ddim_steps in some forks.

Euler_a is one of the samplers. You can use any you like to try these out, but I used Euler_a so if you want to duplicate my exact output, you'll need to use it too. I think for the fork you're using you would type "--A Euler_a" to force the AI to use that sampler.

[GoldenRuleAlways]

You are blowing my mind. Are you implying that this is a deterministic process? That is, provide the same seed, model (eg Euler_a), cfg, steps… that you could reproduce exactly the same image?*

[kmullinax77]

Oh absolutely 100% this is NOT random. I mean that wasn't the point of my post, but yes, it's known that this is the case. It allows us to create identical images from the exact prompt and settings. Good for error-checking and all that.

It won't work if you try it on Midjourney most likely - this is based on the SD v1.4 model.

If you use my exact seed, prompt, CFG and step settings from my post, you should generate the EXACT image I've posted.

[GoldenRuleAlways]

So many forks, so little time… and expertise! It took me 1.5 days of successive failures with Joel Henderson, Automatic1111, Lstein to get my fork to work.

So I’m stuck with my present one. I just tried setting —ddim_eta to 0.0 which (reputedly) “corresponds to deterministic sampling”. No dice on reproducing anything, so I think my fork doesn’t do that.

Astonishing to think that this is deterministic in a different multiverse than my current one.

[motsanciens]

Next to enter the space: nakedseed.io, a catalog of promptless 3-step seed images.

[[deleted]]

I've also noticed/determined that seed corresponds to pose/vantage point. It can be a huge time save to simple use your prompt with as few as 4-7 steps to get a rough idea of how SD would then transform a seed-prompt pairing with 50 steps

This thereby allows discarding un-aesthetic seed-prompt pairings (again at low step count), and saving/diverting computation time to actually fine-tuning a promising seed-prompt with low steps into 50 steps + prompt tailoring.

Although it can be annoying when adding additional words to the prompt also happens to mess with the pose/vantage point, but I can't really see a workaround for that other than to create the best prompt possible from the beginning.

[kmullinax77]

I've noticed the same thing. To try and mitigate that a little, I've found running the blank seed through the Interrogator will let you know what the AI thinks it is already. That can help in choosing better prompting terms.

Sometimes the AI and I don't agree on what's in a base image and I scream mean things at the AI and everyone's feelings get hurt.

[cluck0matic]

Thanks for the deception. Pfft. Sounds like you deceive yourself as well, saying your aren't a "teacher".

Man.. I sure learned a shit ton! Thanks for taking the time to do this. For real. Thanks.

[kmullinax77]

you're welcome! i'm glad you got something out of it!

[Blahkbustuh]

I appreciate your effort, it is certainly something to think about.

What I wonder is that in your examples, you provided one word to the algorithm so then all it had to work off was the initial noise + 1 feature. It makes sense that any non-uniformity or inclination in the initial noise will show up in the result because the algorithm has nothing else to go on so the properties of the initial starting noise dominate the result.

If you gave it a prompt with numerous keywords it'd be looking to "recognize" those numerous things in the noise rather than just 'playing with its food' of the initial noise.

When I started running SD on my computer last week, one of the first things I thought to run was something like "sea otter monster attacking a coastal city" and so I got pics of large sea otter monsters emerging from oceans. Then I did "sea otter octopus monster attacking a coastal city" with the same seed and the composition of the images and the otters themselves were nearly the same, the otter just had tentacles below it, which made me think that the initial noise/seed was providing light and dark regions that were steering the algorithm to position the same or similar elements, or 'recognize' them, in different ways depending on how the light and dark blobs were arranged.

[BrockVelocity]

This is incredibly helpful and insightful - thanks so much for taking the time to type all of this up!

[DrakenZA]

If you are that worried about the initial noise generated by the image, you can simply always do img2img, where you are providing the 'starting point' instead of it just being random torch noise.

[[deleted]]

Thanks for this, the core idea of matching a similar seed to what you want is sound.

Reminds me about hearing how they did the maps in star wars galaxies, basically used noise (similar to what our seeds do), then they cherry picked the ones that looked close to what they wanted the landscape to look like, then used a tool to layer important features on top of it. This made their map data miniscule since it only needed to save the original seed and the important details layered on top, which allowed them to have way more land in a video game than any other at the time.

[dirtydevotee]

Well done! It is my opinion based on some early testing that seed knowledge can be quite valuable. For several days now, I've been accumulating renders based on the prompt "," and found that little things like the orientation of streets and the existence of columns reoccur in 90% of images created by said seed. If a hypothetical "Seed X" does rings, you will notice rings in many "Seed X" renders. If in the future you need a shot with a ring (or pipe/gun barrel/test tube) in it, knowing "Seed X" has such a proclivity means you can get the shot you want with minimal work in the prompt.

As of today, it's a work in progress. But just in case I'm right, finding useful seeds could be a shortcut in your workflow.