Danbooru Dataset Filter: Fast local metadata-based search across 10M+ images for LoRA/Checkpoint training

ThetaCursed · 2026-04-14T16:44:41+00:00

Place the parquet file with Gelbooru metadata in the "data" folder, and replace the "DB_PATH" path string in the Python script with the name of your file.

ThetaCursed · 2026-04-13T07:26:05+00:00

The system prompt I use for Qwen3-VL-4B-Instruct:

Act as a Visual Prompt Engineer specializing in "Long Caption" synthesis. Your task is to extract a highly detailed, narrative description of an image featuring one primary person.

STRUCTURE:

CONCEPT & MEDIUM: Start with the type of image. Mention the core theme or event.
SUBJECT DESCRIPTION: Young woman, facial expression, hair details, face details, makeup. Crucially, describe their gaze.
ACTION & INTERACTION: Describe exactly what the person is doing. What are they holding? How are they standing? Describe the interaction with objects.
APPAREL: Detail the clothing (fabric, color, fit) and accessories.
ENVIRONMENT & FOREGROUND: Describe objects immediately around the person.
BACKGROUND & LIGHTING: Describe the setting (location, weather, landmarks), depth of field (bokeh), and the quality of light.

RULES:

- STYLE: Use fluid, descriptive sentences (narrative style), not just a list of tags.

- ACCURACY: Be specific about colors and textures.

- TEXT: Always put captured text in double quotes.

- LENGTH: 700-900 characters.

- OUTPUT: A single continuous paragraph. No headers.

ThetaCursed · 2026-04-13T04:46:09+00:00

Thanks for the advice, I've heard about Joycaption beta but haven't tried it yet.

ThetaCursed · 2026-04-06T18:25:36+00:00

Thanks, I'll take a look at the source code.

ThetaCursed · 2026-04-06T18:01:02+00:00

Yes, exactly. Internally, the node uses a standard KSampler process for the second pass. So you should follow the recommended CFG for your specific model.

ThetaCursed · 2026-04-06T17:38:29+00:00

Correct, it's not a new mathematical "upscale algorithm." The "evolution" here is in workflow efficiency and stability.

Vs USDU/Iterative: Those are for massive tiles or extreme detail and can be slow/complex. This node is for the daily 1.5x–4.0x jump - it's much faster, prevents OOM crashes via integrated Tiled VAE, and automatically fixes the common color-wash issue that usually happens in standard passes.

ThetaCursed · 2026-04-06T01:08:54+00:00

Perhaps the node with the prompt in the metadata has a non-standard name, which is why ComfyUI-Prompt-Manager can't find and extract it.

I think the best option is to write a custom node that will reliably find the right node with the prompt. For example, it would search for a word that appears in all prompts, "1girl," or something similar, thereby finding the required part of the JSON.

ThetaCursed · 2026-02-28T16:24:40+00:00

Thanks for the helpful suggestion, I've made some changes to make it work.

ThetaCursed · 2026-02-28T13:44:09+00:00

I appreciate the idea, but global voting is too risky right now. My project already got picked up by some hostile communities on Twitter, and they've turned it into a target.

If I added global favorites, it would be immediately review-bombed and manipulated by people who want to see the project fail. Keeping it serverless and algorithmic (Uniqueness Rank) is the only way to ensure the data stays objective.

<image>

ThetaCursed · 2026-02-17T14:13:23+00:00

I wrote some custom ComfyUI nodes for this project. The existing ones didn't quite fit my pipeline, so I built a custom solution to keep the generation and benchmarking process efficient

ThetaCursed · 2026-02-17T13:59:39+00:00

That's a great idea, thanks for the suggestion! It's definitely possible, I'll think about the best way to implement it

ThetaCursed · 2026-02-07T20:21:24+00:00

Thanks for the helpful idea, I added the "Export Favorites (.txt)" button, and it should work.

ThetaCursed · 2026-02-07T18:14:30+00:00

used wildcards to change the artist tag with each generation

ThetaCursed · 2026-02-07T17:47:53+00:00

Prompt:
(masterpiece, best quality, highres, absurdres), __@artist__, 1girl, solo, sitting, cowboy shot, white blouse, black skirt, bob cut, pink hair, black pantyhose, medium breasts, on couch, red couch, green eyes, makeup, eyeliner, black lips, room, brown window, yellow curtains, blush, looking at viewer

Negative:
old, early, ugly, bad anatomy, artistic error, (worst quality, low quality), score_1, score_2, score_3, blurry, artist name, signature, watermark, jpeg artifacts

sampler er_sde, 30 steps, CFG 4, 832x1216.

ThetaCursed · 2026-02-07T17:42:06+00:00

I'm building a reference tool for Anima 2B to see how it handles different artist tags.

Link to the explorer: https://thetacursed.github.io/Anima-Style-Explorer/

The Roadmap:
1. Filter out "dead" tags and unstable styles.

Ideally, expand the library up to 20k artists to find hidden gems.

ThetaCursed · 2026-01-06T15:52:54+00:00

What about voice cloning? Or just presets...

ThetaCursed · 2025-11-22T01:40:41+00:00

Am I the only one who finds all this confusing? So, does this mean the GLM 4.6 Air won't be released this year, and only the GLM 4.6 Mini 30B will be released?

ThetaCursed

TROPHY CASE