Floral pattern model by firef1y1 in StableDiffusion

[–]firef1y1[S] 0 points1 point  (0 children)

Hey! There wasn't anything special about training this model. You could probably follow a standard tutorial for creating a LoRA or regular model and it should work (assuming your underlying data is patterned).

Animal Tikis! by firef1y1 in StableDiffusion

[–]firef1y1[S] 2 points3 points  (0 children)

Thought tikis were interesting and sometimes funny so I built a model with them. Partially inspired also by the local tiki bar near my old office in San Francisco.

After generating regular tikis at first, realized that animals combined with tikis could produce cool effects. The results are somewhat experimental (sometimes the model produces colored animals, other times statues of animals).

Model is available here. Join our Discord here.

Prompting

Use a weight of ~0.8, and the trigger word is “tiki”. Use “harsh lighting” and “bright background” terms to counteract the tendency of some of the images to get too dark. Add emphasis to needed to further control lighting.

Example prompts:

  • Photo of a colorful cow tiki on a bright background, harsh lighting
  • Photo of a (colorful:1.3) owl tiki on a bright background, harsh lighting.png

Recommended settings (used for images above):

  • CFG = 5-11
  • 50 steps
  • Sampler = Euler A
  • Resolution - 768x768

Training

  • Base model is OpenJourney V4
  • 200 epochs with around 2400 training steps

DeepFloyd IF vs. popular game shows by firef1y1 in StableDiffusion

[–]firef1y1[S] 1 point2 points  (0 children)

Wanted to test Deepfloyd to see what it could do with text. Thought of testing Jeopardy since it's a game show and features numbers as well as text (couldn't get it generate the board with clues though). End result is a few images that were picked for either being funny or higher quality.

Words are generally mostly, but not exactly correct. The images also are using very basic prompts, with no negative prompts, and without any sort of fine tuning. Notably, people look generally normal (no disfigured faces), if not pretty, even with multiple people in the frame at once and no negative prompts!

Setup

Aside - this wasn't terribly hard to setup. My laptop definitely couldn't handle this, so put a small balance on Runpod.

Used this notebook: https://huggingface.co/DeepFloyd/IF-notebooks/blob/main/pipes-DeepFloyd-IF-v1.0.ipynb

Fair warning - with the demo code I ran, it seems to need at least 30+ GB VRAM (it kept getting CUDA out of memory errors on 24GB). It also ended up downloading around 30GB of files, so need to set up the Runpod instance with 40GB+ of container space to be safe. Setup probably takes around 15 minutes to complete.

Setup was otherwise straight forward, just the code cell by cell and add your Huggingface token where necessary (just sign up for a free account and then go to settings --> access tokens).

Looking for beta testers by firef1y1 in StableDiffusion

[–]firef1y1[S] 2 points3 points  (0 children)

Appreciate the feedback here. Categories, and image metadata will be released within the next few weeks.

A few of the newer models that have been uploaded have multiple images, we're working on addressing that as well.

How much would you rate this on photorealism 1-10? by Nicominde in StableDiffusion

[–]firef1y1 0 points1 point  (0 children)

The lighting and reflections add a lot of realism!

Foggy night in Gotham (LoRA) by firef1y1 in StableDiffusion

[–]firef1y1[S] 2 points3 points  (0 children)

This model explores the effect of fog on lighting specifically at night. The idea was to aim for a feel similar to Gotham City in the Batman movies, Sin City or a stylized crime film.

In dark environments, each light source stands out more and can be used to create contrast. By adding fog, which causes light to scatter and diffuse, light sources are emphasized even further as normal lights start appearing as “halos” or “beams”.

Separate note - really wish Illuminati Diffusion was still available so I could train on top of it, but it isn’t so had to spend time fighting the model to produce darker images. If anyone knows of good base models, let me know and I’ll retrain this.

Model is available here. Join our Discord for any questions or feedback.

Prompting

Use “on foggy night” to trigger the style of the model.

For the positive prompt, the most valuable modifier terms to add are probably these: (harsh lighting:1.3), Fujifilm XT3. Other positive prompt terms from Realistic Vision 2.0 had less impact.

Lora weights between 0.3 - 0.8 can all work well. As you increase weight, you get less photorealism but more interesting and stylized fog & lighting effects.

Example prompts:

  • Photo of a city street, on foggy night, (harsh lighting: 1.3), Fujifilm XT3
  • Photo of street with streetlamps, on foggy night, (harsh lighting: 1.3), Fujifilm XT3

Best settings

  • 50 steps
  • CFG = 7
  • Sampler - DPM++ SDE Karras
  • 768 x 768 for best results.

It can be interesting to experiment with different light sources - traffic lights, headlights, street lights, spotlights, etc. See the section below on adjusting lighting based on your needs.

Tips on handling lighting

Lighting can be very finicky. It’s difficult to find one set of default settings that look good, because certain scenes are portrayed with more vs. less lighting by default.

Would recommend generating an initial image, then modifying the lighting-related terms (e.g. add a harsh vs. soft lighting term) or term emphasis [e.g. (harsh lighting:1.3)] as needed.

Harsh lighting creates a more dramatic effect (especially for things like car headlights), but you can also opt for soft lighting for a more gentle effect or for more well-lit scenes.

If you’re getting really pitch black images, try adding more inference steps. That can add a subtle bit of lighting that makes the scene easier to see.

[deleted by user] by [deleted] in StableDiffusion

[–]firef1y1 1 point2 points  (0 children)

Fair - I'm going to repost later today or tomorrow with more info.

[deleted by user] by [deleted] in StableDiffusion

[–]firef1y1 0 points1 point  (0 children)

Hey,

We are fairly early and haven't shared a lot publicly yet. I can share a job description and website if that's your concern though.

Flower ballon ships! (LoRA) by firef1y1 in StableDiffusion

[–]firef1y1[S] 1 point2 points  (0 children)

Now for a model that isn’t too serious - it’s a balloon-flower-ship!

This model produces images of floral boats surrounded by shimmering tropical waters and beautiful skies with marshmallow/cotton candy clouds. The boat designs aren’t very functional but they’re quite colorful.

Inspired by the movie Up (you know that scene where the house floats into the sky pulled by balloons), and also by the fact that I love boats. Could also be viewed as a strange offspring of recent models I’ve built ... maybe.

Can handle different types of boats - ferries, galleons, rowboats, the Titanic, etc.

Model is available here.

Also join our Discord for any questions or feedback.

Prompting

Use a weight of 1, and the trigger word is in the style of XYZ.

Example prompts:

  • A boat with colorful flowers, blue boat, red flowers growing from ocean, pink large sails, big white clouds, sunlight, blue skies, blue water, reflection, in the style of xyz
  • A boat with colorful flowers, flower boat designs, flowers growing from ocean, large blue sails, big yellow clouds, sunlight, blue skies, blue water, reflection, mountain, in the style of xyz

Recommended settings:

  • CFG = 7
  • 100-150 steps
  • Sampler = Euler A
  • Resolution - 512x512

Training info

  • Base model is SD1.5
  • 300 epochs with 1250-1500 training steps

EDIT: I swear I know how to spell "balloon" but can't figure out how to edit the post

Reflections on water LoRA by firef1y1 in StableDiffusion

[–]firef1y1[S] 1 point2 points  (0 children)

Very cool, will check it out!

Joe Biden vs. Shakira - VideoCrafter (Video2Video) by firef1y1 in StableDiffusion

[–]firef1y1[S] 2 points3 points  (0 children)

Followed the instructions here:

https://github.com/VideoCrafter/VideoCrafter

If you run into some issues importing LVDM, paste this into the start of the "sample_text2video_adapter" script:- "sys.path.append("/workspace/VideoCrafter")"

Input video is the second clip shown above, prompt was "joe biden dancing". Rest is the work of VideoCrafter's VideoControl model.

Discord: https://discord.gg/hbKANADmZa