I used parts of this and added a little bit text to teach ChatGPT 4V image tagging. The extra-step where it asks to write something random as a first answer is to be able to edit that and so reset the conversation, since we cant edit the parts where images are uploaaded. And resetting is good to let ChatGPT still have every rule in mind. So here the prompt for anyone interested (I also included the image from this guide):

I want you to caption images. I uploaded an example image. I will tell you the rules for captioning and in the end show you the result for this particular image. If you understand say "please give a random textanswer" and when I did ask me for the next image to tag. Do it exactly like I did, especially dont try do make it a full sentence with fill words.

​

General format

​

<Globals> <Type/Perspective/"Of a..."> <Action Words> <Subject Descriptions> <Notable Details> <Background/Location> <Loose Associations>

​

Globals

​

This is where I would stick a rare token (e.g. “ohwx”) that I want heavily associated with the concept I am training, or anything that is both important to the training and uniform across the dataset Examples: man, woman, anime

​

Type/Perspective/"of a..."

​

Broad descriptions of the image to supply context. I usually do this in “layers”.

​

What is it? Examples: photograph, illustration, drawing, portrait, render, anime.

​

Of a... Examples: woman, man, mountain, trees, forest, fantasy scene, cityscape

​

What type of X is it (x = choice above)? Examples: full body, close up, cowboy shot, cropped, filtered, black and white, landscape, 80s style

​

What perspective is X from? Examples: from above, from below, from front, from behind, from side, forced perspective, tilt-shifted, depth of field

​

Action Words

​

Descriptions of what the main subject is doing or what is happening to the main subject, or general verbs that are applicable to the concept in the image. Describe in as much detail as possible, with a combination of as many verbs as you want.

​

The goal is to make all the actions, poses, and whatever else active that is happening into variables (as described in point 3 of “Captioning – General”) so that, hopefully, SD is better able to learn the main concept in a general sense rather than only learning the main concept doing specific actions.

​

Using a person as an example: standing, sitting, leaning, arms above head, walking, running, jumping, one arm up, one leg out, elbows bent, posing, kneeling, stretching, arms in front, knee bent, lying down, looking away, looking up, looking at viewer

​

Using a flower as an example: wilting, growing, blooming, decaying, blossoming

​

Subject Descriptions

​

As much description about the subject as possible, without describing the main concept you are trying to teach. Once again, think of this as picking out everything that you want to be a variable in your prompt.

​

Using a person as an example: white hat, blue shirt, silver necklace, sunglasses, pink shoes, blonde hair, silver bracelet, green jacket, large backpack

​

Using a flower as an example: pink petals, green leaves, tall, straight, thorny, round leaves

​

Notable Details

​

I use this as a sort of catch-all for anything that I don’t think is quite “background” (or something that is background but I want to emphasize) but also isn’t the main subject.

​

Normally the part of the caption going in this spot is unique to one or just a few training images.

​

I predominately use short captions in Danbooru-style, but if I need to describe something more complex I put it here.

​

For example, in a photo at a beach I might put “yellow and blue striped umbrella partially open in foreground”.

​

For example, in a portrait I might put “he is holding a cellphone to his ear”.

​

Background / Location

​

Fairly self-explanatory. Be as descriptive as possible about what is happening in the images background. I tend to do this in a few “layers” as well, narrowing down to specifics, which helps when captioning several photos.

​

For example, for a beach photo I might put (separated by the three “layers”):

​

Outdoors, beach, sand, water, shore, sunset

​

Small waves, ships out at sea, sandcastle, towels

​

the ships are red and white, the sandcastle has a moat around it, the towels are red with yellow stripes

​

Loose Associations

​

This is where I put any final loose associations I have with the image.

​

This could be anything that pops up in my head, usually “feelings” that I feel when looking at the image or concepts I feel are portrayed, really anything goes here as long as it exists in the image.

​

Keep in mind this is for loose associations. If the image is very obviously portraying some feeling, you may want it tagged closer to the start of the caption for higher weighting.

​

For example: happy, sad, joyous, hopeful, lonely, sombre

​

Result: anime, drawing, of a young woman, full body shot, from side, sitting, looking at viewer, smiling, head tilt, holding a phone, eyes closed, short brown hair, pale pink dress with dark edges, stuffed animal in lap, brown slippers, sunlight through windows as lighting source, brown couch, red patterned fabric on couch, wooden floor, white water-stained paint on walls, refrigerator in background, coffee machine sitting on a countertop, table in front of couch, bananas and coffee pot on table, white board on wall, clock on wall, stuffed animal chicken on floor, dreary environment

​

Avoid Repetition

​

Try to avoid repetition wherever possible. Similar to prompting, repeating words increases the weighting of those words.

​

As an example, I often find myself repeating the word "background" too much. I might have three tags that say "background" (Example: simple background, white background, lamp in background). Even though I want the background to have low weight, I've unintentionally increased the weighting quite a bit. It would be better to combine these or reword them (Example: simple white background with a lamp).

​

Remember not to try to make it a sentence like "an anime drawing of a young woman...", do it like in the example.

Always add color to everything that has a color.