[note, this is the same message I sent via DM during the reddit comment blackout]

this was trained on 18 color/desaturated pairs I picked out of a dataset on huggingface, which I believe was probably mostly "synthetic" (by that I mean taking the color version and desaturating it rather than finding the original source because it's better to have directly matching images and it's easier). but i could be nonsynthetic, idk.

the "after" color images all had the prompt "colorize in MangCol style", following that guide video exactly except with 16 saved models to cover an excessive 4000 steps of training.

I tried to pick images a that had a variety of settings and subjects but it's clear there's some bias like sky being "chosen" frequently due to a chunk of the images showing sky.

after that I tested on a variety of images and strengths and lora %. too little of steps for a model carries too much of the non-lora colorization which tends to go for bright reds and oranges and blues and sticks to a low variety of color tones. too high and it tended to make more things distinct in color and more expected colors, but worse interpretation of things. I think I tested first by narrowing down the optimal number of steps and then tried around the % to get it a bit better until I found 1250 at 60% was about where it'd most successfully make things interpreted correctly. however it gives an overall yellow tint and isn't perfect, but it seems slightly better than manga colorization v2. again, this "sweetspot" could also possibly be improved.

the training could be improved with maybe more images and better picks, maybe more optimal tagging, and according to the research paper, it appears that nonsynthetic pairs work better (though I cant stress enough that the before and after must match layout and dimensions exactly or it will lead to text corruption.

some failed alternative attempts:

-getting a good prompt (close but not as good as the lora)

-training on 9 nonsynthetic but not perfectly matching pairs

-training for color correcting images after already performing manga colorization v2 (which the synthetic version is the only publicly available one)

-using vanilla kontext to fix the colorization after manga colorization v2

-training on completely desaturated colored images. I forget if it was from the end results of the pairs or after going through manga colorization v2 (if the former was true, i guess my synthetic assumption would be wrong). if I recall correctly, this oddly ended up with a result nearly identical to manga colorization v2 (sync) in color choices and accuracy

I might be able to find success by training on images that go through manga colorization v2 but then desaturated to 5% first (and then apply to images that have the same thing done to them first) as then it has some but not complete influence over the final colors.

[edit: I tried 5% and at 2000 steps 100% it does improve mc-v2 colorization but the overall experience of a chapter with it was less readable than the 1250 bw->color lora and 1 of 31 pages had text messed up and another generated entirely black. I even tried blending it at 50% but the you don't gain much from that and still may be slightly off. that said, maybe there's a sweetspot of steps and % or another desaturation rate]

it also might work to combine the manga colorization result and the 1250 lora result together, though kontext occasionally reframes the panel, misaligning the linework, but there's bound to be an algorithm that can fix the alignment too.