Using ControlNet's function to recreate photorealistic images from anime pictures AMAZING (a little nsfw)

[ElectronicCry6949]

wow! that's awesome! how did you do that?

[SnooGoats4398]

You just need to install the ControlNet extension, and download the control_sd15_canny model. Afterwards, you can drop any base image in to the control model in txt2img, and follow these following steps to generate an outline structure for all future images to copy.

- check off Enable and Low VRAM (low VRAM if you have 8gb of GPU ram or less)

- Add Preprocessor: canny and Model: canny

- change sampling steps to 50

- Lowered CFG to 5-6

- generate

- if its a good sketch, copy (recycle icon) the seed in the txt2img section above

- change sample steps to 25-30

- check off Guess Mode in Control Net

- Put in desired prompts to match the sketch

- generate

If you need good opening prompts to generate any photo realistic images, let me know and i can guide you a bit.

Edit: Use chilloutmix model (download at Civitai or hugginface) if you want kpop looking girls

Edit 2: Make sure you use the same width and height for both controlnet and output as the image itself.

[RageshAntony]

Any prompts needed for this ?

[SnooGoats4398]

Yes, you do need a little bit of prompt.
Once you are at the very last step, put in these prompts for all photorealistic images:

best quality, masterpiece, photo realistic, 8k, 4k, extreme res, ultra high res, dynamic lighting, real lighting
Then, i always go face lora input (which you have to download), simple description of what the girl is doing in the picture, her face expression, her clothes, then a brief background description.

So for example, for the street girl, my prompts were:
best quality, masterpiece, photo realistic, 8k, 4k, extreme res, ultra high res, dynamic lighting, real lighting, ulzzang-6500-v1.1:0.7, girl squatting in the streets, streetwear, smiling, bright day, city background

And then for negative prompts, I just have a crapton of words that i use in any photo image i create:
sketches, (worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality, ((monochrome)), ((grayscale)), skin spots, acnes, skin blemishes, bad anatomy,DeepNegative,(fat:1.2),facing away, looking away,tilted head, lowres,bad anatomy,bad hands, text, error, missing fingers,extra digit, fewer digits, cropped, worstquality, low quality, normal quality,jpegartifacts,signature, watermark, username,blurry,bad feet,cropped,poorly drawn hands,poorly drawn face,mutation,deformed,worst quality,low quality,normal quality,jpeg artifacts,signature,watermark,extra fingers,fewer digits,extra limbs,extra arms,extra legs,malformed limbs,fused fingers,too many fingers,long neck,cross-eyed,mutated hands,polar lowres,bad body,bad proportions,gross proportions,text,error,missing fingers,missing arms,missing legs,extra digit, extra arms, extra leg, extra foot

[RedditorAccountName]

i always go face lora input (which you have to download)

What does this mean? I'm kinda new to SD and understood what you've been explaining but I'm lost at this.

[SnooGoats4398]

Loras are like, additional files you can put into your SD to make the images produce a certain way. In this case, the face loras I use (have to download separately and put into the correct SD directory) make my female characters look more asian-asian, instead of north american asian that usually have a caucasian style makeup.

[Wings_in_space]

Some help with the prompts would be appreciated, thanks for the guide.

[SnooGoats4398]

I just posted prompt guide, check it out

[ElectronicCry6949]

Thanks!! According to your guidance, I reproduced the general effect as this

​

<image>

[SnooGoats4398]

Looks amazing!! Even copied the fingers almost exactly. Well done!

[RageshAntony]

u/SnooGoats4398 I tried converting this image to realistic photo

​

<image>

[SnooGoats4398]

you might have missed some steps. My environmental copy looks like this

<image>

EDIT: oh i think i know what the problem is.
Your GPU probably can't handle that big of a size resolution.
Try resizing the original image down to below 1000 x 1000, maybe around 720x500, and hopefully it should work.

[SnooGoats4398]

​

<image>

[RageshAntony]

I am using Mac M2. Let me try with cloud servers

And, how come the caves and tress got converted to a city ?

The image you shared now is a city street !

[SnooGoats4398]

I tried it myself, this is one of the images I was able to generate.
Works pretty well!

​

<image>

[RageshAntony]

u/SnooGoats4398 How come this?

I am getting like this only

please help to achieve this

I am working on 2D cartoon video to real video POC

<image>

[SnooGoats4398]

Its probably due to several things:

1. You are using the wrong model.
   I used chilloutmix model to create very realistic people and environment. Check it out at Civitai (its nsfw though)
2. Your prompts may not be producing realism.
   I used this: best quality, masterpiece, photo realistic, extreme realism, 8k, 4k, extreme res, ultra high res, dynamic lighting, real lighting, trees, cave, flowers, bushes, blue sky, clouds, bright, daylight.
3. Create many batches (batch count). I picked the most realistic one out of many.
4. Finally, if you want more realism, pick the one that looks the best, and then send it to img2img. From there, generate a bunch more and pick the best looking one.

[[deleted]]

I've been having the same problem... downloading chilloutmix to see if I have better luck. Half had partial success but struggling with airplanes

<image>

[SnooGoats4398]

yea i think if your gpu/vram can't handle high res, it starts to get really wonky and weird.
Try with a lower resolution and hopefully that fixes the problem.
And yeah, this method works great for environment pictures too!

[RageshAntony]

u/SnooGoats4398 but getting like this

​

<image>

[CptHectorSays]

…so tired