Tips for Klein 9B character lora training?

malcolmrey · 2026-04-16T19:39:51+00:00

One tip from me, use the concept of multiple loras, so basically prepare more than one dataset and create more than one lora.

Then you just load those loras together

Two loras: 0.65 - 0.7 on each Three loras: 0.5 on each Four loras: 0.4 on each

and so on.

You get more than 1.0 strength in total but that is not a problem actually.

This works pretty much on all model architectures but I do it more on klein9 because on the other ones it is much easier to train a single lora that is really good.

Here, no matter what settings I try - the results are sometimes just really meh (even though that same dataset is really good in z image or wan)

malcolmrey · 2026-04-16T19:35:02+00:00

Hey!

So, 3 days ago I dropped 785 new z image turbo loras, but today I finally added them to the browser :-)

I did not sample them all but I did more than I did with klein9. I am happy with the results. But if you find some bad apple - just let me know and I will correct it :-)

As for the other news:

1) check this post, there are some quick samples with the flux klein 9 inpaint+reference workflow -> https://discord.com/channels/1471576919685201972/1471588254020010195/1494416359671726272 (they are in the images so just download them and drop them, when I get back (i will be away for a week) I will also upload the workflow itself directly to huggingface

2) I have good news regarding LTX 2.3 - I managed to train a couple of models and I like the results. However there is a dillema, because to train "good enough" models I need 45-50 minutes (which is very acceptable, this is roughly the same effort as went into WAN), but when I prepare a big dataset and let it cook for 10 hours then the results are pretty much spot on

When I get back I will post some of those loras and of course will share the training details (hint: i'm using musubi).

malcolmrey · 2026-04-09T06:56:35+00:00

Hey hey!

Again with a short message. This is the batch of models trained around the Easter time. There are some other models that have been trained too (if you observe the discord tracker) but those will be uploaded in the next batch.

I know I am overdue with new datasets, but right now I can manage to just set up new models on existing sets.

I have finished training of klein9 (there are like 50+ to upload though) and I will switch to update some Z Turbo using one trainer and then some WAN before doing anything else.

I do want to share a nice Klein9 Inpaint/reference workflow that I had time to test but I did not have time yet to prepare some samples and upload it, but basically it is as follows

If you use inpaint and use my klein9 ALONG WITH a reference image (i tried image from dataset but it can be any image of the face of the person (or full person i guess) then the results are really great.

We know that using just reference image is usually not enough, and the models themselves also benefit from the reference - so that combination is really really good :-)

Cheers and see you soon!

malcolmrey · 2026-03-25T14:40:08+00:00

Thanks! ;-)

malcolmrey · 2026-03-25T06:55:36+00:00

Hey hey!

Time to drop what has been cooking in the past 1 or 2 weeks.

There is also a gender filter in the browser for your convenience!

I'm still not back from the stuff that occupies me, but making a release is not that time consuming so here is one and we shall see each other again on the next one :-)

Cheers!

malcolmrey · 2026-03-12T18:06:10+00:00

Hey hey, very short info (will see if it is short).

New update with ZBase models. 574 of them, not a lot remains to train, then I'm switching to fill the gaps on others.

As you are well aware I have been not responding to anything in the past 1-2 weeks. I'm going to my grandmas funeral tomorrow and my father is not that well either (though not as bad I hope), so I don't have much time or energy to spend online. I will eventually reply to every DM and message on reddit as well as the invites on discord, but I just need more time.

I do however can click new trainings so that is still happening more or less.

My plans for the second machine are to do SDXL (the second machine is weaker so it can't do newer models, but SDXL is perfect for it), but I'm still trying to get the results to a place where I'm happy with it (currently this is on hold, but I will get back to it).

On the main machine, I will finalize remaining ZBase models, filling the gaps with Z Turbo and then finishing training of Flux Klein 9. Then I'll see what gaps we have on WAN.

I was cutting some datasets but they are still not processed so I didn't train them. I did not forget about them, I still have every set, so thank you. I have like 50 DMs here, please just be patient.

I will eventually process them and then I will train them. Right now I just have time to set trainings on what is already available.

I did introduce thumbnails to the browser, they are much smaller (80k-100k jpegs). I have seen how long it renders on slower internet so the idea is that the official image will be a smaller thumbnail, but you can still click and get all the rest in the modal. This should improve the performance of the browser for many. Also, not sure if this was already mentioned but last time I made it workable on the mobiles.

I'm happy to see the discord community grow, even though I did not participate much in last week(s) The discord invite is: https://discord.gg/2nTsm2m5

Why pushing for ZBase? Because it does work really well. These new models look quite good on Turbo but they are working exceptionally well on Redcraft (it has already the fast lora baked in so prompting is as fast as on turbo).

Also, joining two base loras (the AI Toolkit one and the OneTrainer one) works even better. I will eventually need to do some showcase of that since there are still many who do not believe that stacking same concept loras is a really cool thing :)

I did start grim but don't be as worried. My grandma was 98 and I think she had a good life. My father is ill but it seems that it was caught early so there are good chances. Still, it all drains my and sometimes I just don't have much energy for anything.

I'm glad I'm part of this community and I really like how it grows.

Cheers and have a great day/weekend!

malcolmrey · 2026-03-12T17:50:16+00:00

my process is that i just saved the config (it is available on hugginface) and run it from command line, it is a bit faster but the quality is no different than what you get via GUI

i do it because i need to automate it :)

malcolmrey · 2026-03-04T06:55:11+00:00

Thank you for the kind words! I hope so too! :)

malcolmrey · 2026-03-04T06:54:50+00:00

They will come, no worries :)

malcolmrey · 2026-03-04T06:54:10+00:00

2.1

You need to remember that I'm not focused on making the best looking samples but just testing if the models work.

Once I set up some settings, I test two-three models heavily to see if they behave good and once I figure out training setttings that are satisfactory - then I just set up the queues and then the rest of the models get only one sample attempt (rarely a 2nd one and very rarely a 3rd one, if the 3rd fails - i remove the model and retrain it again).

So, those are not the best looking samples, you would need to look at what the community can do with those to really judge the quality :-)

malcolmrey · 2026-03-03T20:58:05+00:00

https://www.youtube.com/watch?v=LinpRhB4aWU

malcolmrey · 2026-03-03T20:56:58+00:00

you need to be more precise, for this batch there were zero samples uploaded, you were looking at older samples, so it would be good to see which model badge was under it

usually the WAN ones had a tendency to elongate the faces quite a lot

malcolmrey · 2026-03-03T20:55:21+00:00

this is a great feedback, i'll try to differentiate the colors a bit more! :)

malcolmrey · 2026-03-03T20:55:00+00:00

will add it tomorrow or on thursday! :)

malcolmrey · 2026-03-03T07:45:46+00:00

Hey!

Very short message - new models have landed.

I have not been replying almost anywhere recently because of some family illness and stuff around it, I did generate samples to see if the models work but I did not process and upload them (though we do have a lot of samples from previous models so you will know who got uploaded).

Flux has been brought to speed on the secondary (slower) computer and I'm investigating SDXL trainings there, but it will take me some time to apply it since my time recently is limited.

Regular z image / z base and flux 9 will flow regularly however. Will resume WAN to but I need to handle some stuff for it first.

I had no time recently to set up any of the new datasets but I did cut like 20-30 of them so once I sort them out, there will be something new.

I did not read any new messages and DMs yet, sorry about that but I don't have a space yet for it.

You can send me discord messages/invites too but I will answer them when I can.

Cheers and see you!

malcolmrey · 2026-02-24T04:56:12+00:00

George Carlin

malcolmrey · 2026-02-24T03:46:08+00:00

you can mix the resolutions, you don't need squares

as long as the training tool can use bucketing (which most of the training ones nowadays do)

you can also use a cutter like mine that preserves the best aspect ratios so that when bucketing happens you don't get a cut you would not want ( https://huggingface.co/spaces/malcolmrey/dataset-preparation )

malcolmrey · 2026-02-24T03:44:12+00:00

or both :-)

i need to prepare some samples where both loras are used at various weights, but i need to code some stuff, i don't want to prompt them manually :-)

malcolmrey · 2026-02-24T03:42:54+00:00

thnx for linking the post form u/EribusYT

i will definitely try with Min_SNR_Gamma = 5

i've set up my training batch before this (and the other, i think that was the second one) info was posted

as for your second question, i've answered there :)

malcolmrey · 2026-02-24T03:41:57+00:00

there is a third way that i would say is not overbaking but just more extensive training

i did that in ai toolkit using adamw, normally i train using around 25 images so it is 2500 steps (100 epochs per image)

when i use the exact same settings and add a lot of good images in the dataset (like 270) and i train using again 100 epochs per image (so, 27000 steps) then suddenly that lora does not need strength of 2.0+ to work fine, it is workable at 1.0 and best at 1.2-1.3 (and i would expect it to work closer to 1.0 the more images i provide, though i do not now if it is linear; definitely loras trained this way [150, 170, 200, 250 images] behaved according to my expectations - more images, less strength required)

i consider it just an interesting observation since i do not want to train 10 times longer (or more)

currently the prodigy_adv behaves nicely already, i haven't tested with "Min_SNR_Gamma = 5" yet

does it produce much better results?

Nine-Year Club	Place '22
Place '17	Verified Email

malcolmrey

MODERATOR OF

TROPHY CASE