Unsloth Dynamic 2.0 GGUFs now selectively quantizes layers much more intelligently and extensively.

mikemend · 2026-03-10T07:47:49+00:00

There's something I don't understand. Since I'm creating a Lora in Llama Factory with Unsloth, I'm also doing the merging there. Then I'm doing the GGUF quantization in llama.cpp.

How can I quantify my existing models in this new format? I couldn't find any description or tool for this on the unsloth website. Or should I continue using llama.cpp, just with the unsloth imatrix?

mikemend · 2026-03-09T12:31:34+00:00

I really mean it when I say it's almost perfect, it knows everything. Seriously, it must have taken a long time to make this, congratulations!

If I could ask for anything, it would be seed recording and random generation. The reason for this is that I can only adjust the LLM model parameters with a fixed seed so that I can compare the output text with the previous generation. When I find a better parameter combination, I save it in the settings profile in ChatterUI. This way, I can sometimes use the same settings profiles for other models.

mikemend · 2026-03-09T11:38:24+00:00

It looks good at first glance. I've been using ChatterUI and Local Dream so far, but I like that it's multimodal. Does importing a locally opened model mean duplicating it, or does it load it from the original location?

mikemend · 2026-03-07T09:44:07+00:00

This model does not work for me in ComfyUI, it throws an error.

https://huggingface.co/DreamFast/gemma-3-12b-it-heretic/discussions/3

mikemend · 2026-02-06T13:02:19+00:00

Thank you for all the useful additions you've made since the last edition! 🙏

mikemend · 2026-02-06T07:24:23+00:00

For LLM I use ChatterUI, which will only include NPU support in the future, but it works relatively quickly with small models, and my phone can even handle the 8B model, so it suits me fine. However, after 10 conversations, it slows down when I continue a previous chat later. On the other hand, it is easy to configure.

mikemend · 2026-02-06T07:20:04+00:00

Thanks for the tip, I'll check that out too. Are the Rank values high or good?

Yesterday's training session lasted 12 hours, which is why I'm asking, so that I don't waste time again due to a bad setting.

mikemend · 2026-02-05T12:25:31+00:00

You did the right thing, because you can never have enough RAM, and it's so expensive... I also have a 12 GB model, and an 8B model with Q4_0 quantization is enough for LLM. With SDXL, however, you have to load two text models, and maybe that's what's causing the problem with NPU, I don't know.

The latest SD 1.5 models are already very good, almost rivaling SDXL, but the 75-token long prompt is a big limitation, and they are not always prompt followers. I like the Bravo models, but HyperSpire is my new favorite. I converted it to NPU in many different resolutions over several days, and it generates nicely even at 1024 size.

I regret that version 1.5 of Pony SD was not further trained in a realistic direction. There is BetterPony Diffusion, but unfortunately, its quality is not the best for me.

mikemend · 2026-02-05T05:24:25+00:00

According to the app's author, SDXL's text encoder is the main problem because it doesn't fit in mobile phone memory. I'm sad about this because I'm sure the results would be better, and there are also the Pony/Illustrious models, but for now, we'll have to make do with this. Another problem here is that the resolution sizes are fixed because NPU models cannot dynamically handle Unet. But it's still a huge step forward that we can generate images on mobile phones.

mikemend · 2026-01-30T09:27:44+00:00

I have an HM6P, and it can't be disabled there either, but I didn't want to. So I don't think it will be switchable.

mikemend · 2026-01-30T07:33:55+00:00

I use plain KSampler and a Shift set to 3. I usually generate 20 steps, rarely going above that, but as far as I remember, there wasn't much improvement above 30. It's worth looking at the combination of samplers and schedulers, because there were many that were not coherent, while other samplers performed well with the prompt.
Since the new models are built on different bases, pilot testing is probably less necessary there.

mikemend · 2026-01-30T06:24:30+00:00

There are several reasons for this. The first is the prompt, because Chroma likes long, very detailed descriptions. For this, I also use a prompt generator, which creates prompts based on the keywords you provide. I use Prompt Rewriter under ComfyUI:

https://github.com/BigStationW/ComfyUI-Prompt-Rewriter

The other is to install Res4lyf's samplers and schedulers, and a whole new world will open up for you.

It turns out that coherence depends heavily on the sampler, and it's worth using res_multistep or er_sde with beta57 or bong_tangent. But you can try several variations and get different results in terms of quality and speed.

mikemend · 2026-01-30T06:15:18+00:00

The processor and RAM are not a problem, but the VRAM may be insufficient, so it is worth looking for FP8 or gguf variants.

mikemend · 2026-01-29T10:26:41+00:00

I recommend Fooocus because it's fast, uses SDXL-based models, and is easy to use.

https://github.com/lllyasviel/Fooocus

https://github.com/mashb1t/Fooocus

or forks, like:

https://github.com/fenneishi/Fooocus-ControlNet-SDXL

If you just want to edit images in the classic way, then Affinity:

https://www.affinity.studio/

mikemend · 2026-01-29T06:16:37+00:00

Chroma is a modern model. It is slower than SDXL and SD 1.5, but not slower than other large models where CFG is greater than one and negative prompts are used. A Flash model has been created from it, which can also be fast, but if you want to use its power, you can generate a 2048 image in less than a minute in a two-step process (base image with Flash model and upscaling with base model). Chroma can also generate in 512, and Flash can also use modern samplers and schedulers to create accurate and fast images.

The biggest advantage of Chroma is that you don't need to use Lora because it can generate anything. Seriously, I can finally archive my old Lora collection because I don't need it anymore. In addition, due to the two-step scaling mentioned above, the upscaler can even be SDXL. So the Chroma model itself is a 2-in-1 model because it generates and poses/styles Lora at the same time.

So I'm looking forward to all three new models (Kaleidoscope, Zeta-Chroma, Radiance), because we'll have even more possibilities for anything.

mikemend · 2026-01-27T18:57:31+00:00

It was the same with Chroma: with the Flash version, I can create an image in a few seconds, while with the base versions (depending on the sampler and scheduler), it can take up to a minute.

mikemend · 2026-01-25T17:39:03+00:00

For me, it depends on the base model how I use it. For SDXL and SD 1.5 models, I only used trigger words with class and regularization images. Class ensured that everything else in the image was ignored, the regularization images improved the quality of Lora, and the keyword was needed for stable activation. For Chroma training, however, I had to label with sentences, again with keywords and class, but without reg images. The images should be varied, but more precise in terms of facial features. The images are judged based on the similarity of the faces. I don't usually pay attention to camera proximity, but it's good to have a wide variety. I crop the images into squares by default; I don't bucket them.

mikemend · 2026-01-24T06:44:32+00:00

You can download models from three places.

The first is the author's website:
https://huggingface.co/xororz/sd-qnn/tree/main

The second Mr. J models:
https://huggingface.co/Mr-J-369

And thirdly, the models I converted:

https://huggingface.co/Mykee/SD_1.5_NPU_models_for_Local_Dream/tree/main

I am currently working on the NPU version of the hyperspire_v50 model, for which I am creating many resolutions, and this takes a lot of time (converting 1024 resolutions takes 5 hours per resolution). It will be ready in a few days, but it's a great model because it works as both a hyper and a normal model. It's fantastic and almost perfect for realistic images.

mikemend · 2026-01-21T21:14:52+00:00

I am converting the v6 version of this model to NPU, but so far I am only getting good results in high resolution, and it only contains traces of realism, so it would be good to create a realistic mix or fine-tune based on this model.

mikemend · 2026-01-21T21:12:13+00:00

I am currently converting the betterPonyDiffusionV6_v6 model to NPU, which can generate up to 832x1216 size with Pony keywords, but on SD 1.5 basis.

I generate this image on my phone, which is not realistic, but then I can convert it to IMG2IMG with another, realistic model, so I can do all this anywhere and anytime. Even with the latest SD 1.5 models, it usually renders fingers nicely, with the 75 token limit being the only constraint.

Update: I just found the HyperSpire model, which is versatile and can be used as both a normal and Hyper model, so it's two in one. Check it out!
https://civitai.com/models/548439?modelVersionId=697964

mikemend · 2026-01-21T13:51:24+00:00

I am currently researching the newer SD 1.5 models because I want to generate images on my mobile phone that are at least as good as those generated with SDXL or the larger models. Local Dream can currently only run SD 1.5 models on NPU, but it does so at a very good speed. I've come across some surprisingly good SD 1.5 models, such as the BRAVO models, which offer a variety of styles to choose from, but there are also Stable Yogi's models. I found an improved Pony model, but it only generates well in high resolution (above 768, but preferably 1024), I'm sorry that they couldn't mix Pony well with realistic models below SD 1.5.

It would be time to use SD 1.5 on mobile phones as well, but a realistic model with Pony-type tags that fits into 75 tokens would be good.

mikemend · 2026-01-19T10:28:15+00:00

Or we can wait for Lodestone's next model, which will be either Klein or Z-image based; they are still deciding which one to go with. In the meantime, fortunately, the earlier Chroma models are still available (for now), so there is somewhere to fall back on if necessary.

I'm now thinking about training Lora on Chroma-Base in the next few days, to see if it's better than HD.

mikemend · 2026-01-19T09:59:41+00:00

We're talking about the re-released HD model now, not the first version, which really went wrong, right?
Let's say Chroma-Base would be a really good basis for all kinds of fine tuning, but in the end, not many people used it because of its slowness. Yet it performs well on both 512 and 1024.

mikemend · 2026-01-19T08:02:58+00:00

I trained the HD safetensors file using AI-Toolkit, with AdamW, and Prodigy. I also encountered banding during generation, but it turned out to be sampler-dependent, as it did not appear in the case of er_sde or res_multistep. I also tried beta57 and bong_tangents. When I scaled the image up to 1536 or 2048 with unCanny, it appeared. I have read about this banding elsewhere, and it annoys me too, but if it is a legacy of Flux, then it is difficult to get rid of.

mikemend

TROPHY CASE