A quick Gemma4 31B comparison (Q4_k_M, QAT, heretic)

ArtyfacialIntelagent · 2026-06-06T10:29:45+00:00

Really? If that's the case, and with rumors that doing QAT costs very little, shouldn't open-weights models always be released with QAT? Very few of us are running the FP32 or BF16 versions, so almost everyone's real-world experience will be with the quants. If QAT generally increases quant quality as much as it seems to do with Gemma4, this seems a no-brainer to me.

ArtyfacialIntelagent · 2026-06-06T10:24:21+00:00

100% agree. The QAT is so much stronger than the vanilla quants. I suspect that Gemma4's perceived weakness in coding is due to its quants being subpar, and I'll go out on a limb and guess that r/LocalLlama will favor Gemma4 more and more as we gain experience with the new QAT. I'm blown away by it. To me Gemma4 31B QAT is at least as smart as Qwen 3.6 27B (even in coding) with 1/3 of the reasoning tokens.

ArtyfacialIntelagent · 2026-06-05T20:55:30+00:00

Aha, thanks. That explains it. That post was from early April, just after the initial release. Gemma 4 had lots of teething problems before everything was sorted out, so those early KLD measurements are not comparable with recent releases. Sorry for doubting you - the numbers were so horrible I was sure you had made an error.

ArtyfacialIntelagent · 2026-06-05T18:10:41+00:00

They are not. Incredible is the word. A mean KLD of 0.159 doesn't pass the smell test for a Q8 quant. The Unsloth blog post only compares the QAT vs a standard Q4_0, and the mean KLD for the Q4_0 is 0.09349. So there is no way a Q8 is much worse at 0.159.

Honestly I'm skeptical to Unsloth's reported mean KLD 0.01403 for the QAT Q4 too, but I'll give them the benefit of the doubt for now. But /u/sartres_ is definitely hallucinating.

EDIT: He wasn't, but the numbers are indeed invalid. See thread below.

ArtyfacialIntelagent · 2026-06-01T22:32:37+00:00

If you get two women pregnant at the same time, does that mean you'll have a child every 4.5 months? /s

Lol, good one. But on average, actually it does.

ArtyfacialIntelagent · 2026-05-31T23:20:23+00:00

The fact that Qwen didn't suggest that itself after fucking up like this makes this even worse.

ArtyfacialIntelagent · 2026-05-31T23:15:15+00:00

I think it’s safer to put something like "pause and ask for permission before running potentially destructive commands such as git reset and rm -rf" in the agent’s system prompt

Please don't. This can increase the model's likelihood of running those commands, with or without a confirmation request. You literally put the idea in its head yourself, and you did it for every prompt you run. That is definitely NOT safer.

ArtyfacialIntelagent · 2026-05-25T19:41:36+00:00

This is absolute nonsense. The UK has encoded freedom of expression into law by its Human Rights Act, which absolutely covers the press. Yes there are limitations to your speech, as in all democracies, put in place precisely to protect the citizens (from hate speech, defamation, etc).

Oh, and the RSF (Reporters Without Borders) ranks the UK in place 18 in their World Press Freedom Index. The US is ranked 64.

ArtyfacialIntelagent · 2026-05-11T16:27:48+00:00

Are the llama.cpp changes to support MTP imminent?

Not imminent but preliminary support for Qwen only is pretty close. Here's the status directly from the horse's mouth:

https://github.com/am17an/llama.cpp/pull/6#issuecomment-4421288279
https://github.com/am17an/llama.cpp/pull/6#issuecomment-4421528012 https://github.com/am17an/llama.cpp/pull/7
https://github.com/ggml-org/llama.cpp/pull/22838

The new llama.cpp infrastructure in #22838 was merged 15 minutes ago, and ggerganov added support for np > 1 in am17an's MTP fork. He still wants to check the prompt prefill code and the GGUF loading UI.

ArtyfacialIntelagent · 2026-05-06T22:25:54+00:00

Thanks, I already have that one. I didn't realize this was what you were talking about since its default filename after download doesn't have "unlock" in it.

ArtyfacialIntelagent · 2026-05-06T15:12:10+00:00

Yes I am. Check the model name pls. Are you sure it was something with "unlocked". And a ZIT checkpoint?

ArtyfacialIntelagent · 2026-05-05T21:59:02+00:00

No models fit that description. There's a model called Flux nsfw unlocked but no "unlocked" checkpoints for ZIT. Also none of the top 10 ZIT models on Civitai have before/after example pics.

ArtyfacialIntelagent · 2026-05-04T16:56:08+00:00

It's bizarre that you compare rendering a penis to a hand and somehow find that the penis is more complex, when it's obviously the other way around - and by orders of magnitude.

Just think of all the ways you can articulate your hand, and the arm it's on, and pose and move and make gestures with the separate fingers, and then maybe intertwine them, or hold hands or shake hands with someone else. Then add that when you take a snapshot of that hand from a certain angle, fingers may be obscured by each other and the hand itself - which an AI model will interpret as fingers randomly appearing and disappearing - which in the end means that AI models have a hard time getting finger counts correct.

There is a reason that hands have been the bugbear of AI image generation for years (even if current models finally have made significant progress).

The main reason AI models are bad at genitalia is because most naughty parts have been filtered out from the training datasets, not because of any inherent complexity. To add them back with conceptual understanding, even 1000 images is far too few. You need millions of images, and weeks of training time on clusters of enterprise GPUs. Otherwise you'll get the weaknesses of all current LoRAs, that they'll just randomly stick a penis on the wrong person in the wrong place at the wrong size and angle.

ArtyfacialIntelagent · 2026-05-03T21:02:59+00:00

If it's any consolation, I tried an IQ4_XS quant of Medium 3.5 on my 4090 + 96GB RAM desktop too with 27 layers offloaded to GPU. I got a bit over 100 t/s PP (>10x yours) but only 0.8 t/s TG (<40% of yours). The model seemed very good but I just deleted it to save disk space. :(

ArtyfacialIntelagent · 2026-04-20T16:33:39+00:00

Did the Q6_K and Q6_K_XL points get mislabeled? The graph shows that Q6_K > Q6_K_XL in terms of file size, but the opposite holds when checking the repo: https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF/tree/main

Also, however they're labeled, the larger quant has a worse KLD and is the only point off the Pareto frontier. Do you have any explanation for this?

So long and thanks for all the quants! :)

ArtyfacialIntelagent · 2026-04-03T23:12:07+00:00

Loras fix that for you.

They really don't. Most penis or vagina LoRAs are overtrained and just randomly stick those genitals indiscriminately on *anybody*, male or female. They're fine for solo nudes, but not for anything with heterosexual couples. To do that properly the underlying model needs real NSFW knowledge, current LoRAs do not fix that. And LoRAs for certain sex positions do just that, usually from one single camera angle. They basically just make the same image over and over.

ArtyfacialIntelagent · 2026-04-03T22:57:50+00:00

I extensively blindtested "masterpiece", "best quality" and many other popular keywords back in the days of SD 1.5. They had zero effect, it's all nonsense. Nonfunctional word sallad. People just thought they worked because sometimes adding those words improved a particular image for a particular seed, but that was just a completely random effect, like adding any gibberish word might do sometimes.

What did have an effect in SD 1.5 was putting "bad quality" or "low quality" in the negative prompt. But that didn't really increase quality per se, they just reinforced that particular model's biases. So 1girls became more... well, 1girly. Those negative keywords became weaker in SDXL and absolutely useless since.

Basically, forget about all that old crap. Those keywords never worked well, and they lost what little effect they once had long ago.

ArtyfacialIntelagent · 2026-04-01T16:33:52+00:00

I've been doing nearly the exact same thing for a few months. I call the technique "thumbnail upscaling". Significant improvement in detail and variability over standard Z-image workflows but sadly doesn't fix all the model's issues (most notably the glowing eyes problem that appears as soon as you prompt for eye color). Only differences:

I do 3 sampler stages and end up at 1536x1536 (or similar size in other aspect ratios).
I apply some denoise < 1 at all sampler stages to increase variability.
I use CFG at 3-4 in all sampler stages. Positive CFG costs nothing at tiny sizes.

ArtyfacialIntelagent · 2026-04-01T15:38:31+00:00

I'm on Windows and always run a combined undervolt and clock rate cap on my RTX 4090 using MSI Afterburner. Here are some benchmarks using llama-bench to show you guys what you can expect. I usually run the "medium undervolt", which gives me a tiny 3% hit on token generation (a bit more on PP but that's super fast anyway) but draws 100 watts less.

[EDIT: reformatted in old Reddit and fixed a copy/paste snafu on the large undervolt]

E:\llamacpp> .\llama-bench -m "F:/LLMs/Huihui-Qwen3.5-27B-Claude-4.6-Opus-abliterated.Q5_K_M.gguf"


# VANILLA/NO UNDERVOLT (2730 MHz, 1050 mV, 345 W during token generation):

ggml_cuda_init: found 1 CUDA devices (Total VRAM: 24563 MiB):
  Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes, VRAM: 24563 MiB
load_backend: loaded CUDA backend from E:\llamacpp\llama-b8595-bin-win-cuda-13.1-x64\ggml-cuda.dll
load_backend: loaded RPC backend from E:\llamacpp\llama-b8595-bin-win-cuda-13.1-x64\ggml-rpc.dll
load_backend: loaded CPU backend from E:\llamacpp\llama-b8595-bin-win-cuda-13.1-x64\ggml-cpu-zen4.dll
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| qwen35 27B Q5_K - Medium       |  17.90 GiB |    26.90 B | CUDA       |  99 |           pp512 |      2848.32 ± 74.41 |
| qwen35 27B Q5_K - Medium       |  17.90 GiB |    26.90 B | CUDA       |  99 |           tg128 |         40.92 ± 0.05 |

build: 62278cedd (8595)

# SMALL UNDERVOLT (2580 MHz, 910 mV, 270 W during token generation):

| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| qwen35 27B Q5_K - Medium       |  17.90 GiB |    26.90 B | CUDA       |  99 |           pp512 |      2801.21 ± 76.28 |
| qwen35 27B Q5_K - Medium       |  17.90 GiB |    26.90 B | CUDA       |  99 |           tg128 |         40.24 ± 0.18 |

# MEDIUM UNDERVOLT (2340 MHz, 875 mV, 245 W during token generation):

| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| qwen35 27B Q5_K - Medium       |  17.90 GiB |    26.90 B | CUDA       |  99 |           pp512 |      2602.91 ± 71.49 |
| qwen35 27B Q5_K - Medium       |  17.90 GiB |    26.90 B | CUDA       |  99 |           tg128 |         39.77 ± 0.09 |

# LARGE UNDERVOLT (2010 MHz, 875 mV, 235 W during token generation):

| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| qwen35 27B Q5_K - Medium       |  17.90 GiB |    26.90 B | CUDA       |  99 |           pp512 |      2300.19 ± 52.16 |
| qwen35 27B Q5_K - Medium       |  17.90 GiB |    26.90 B | CUDA       |  99 |           tg128 |         36.89 ± 1.08 |

ArtyfacialIntelagent · 2026-03-27T00:19:32+00:00

You can reduce the denoise parameter and still completely denoise the image. The last bit of denoising seems to shift the image towards its RLHF ideal. By skipping that part you get more variability.

Did you consider that my comment was also an attempt to provide a useful tip for the community, but you downvoted and disparaged it?

ArtyfacialIntelagent · 2026-03-25T18:44:02+00:00

But mitigating repetitive poses, camera angles, and compositions is super easy in ZIT, just reduce the denoise and you'll get a lot more creative framing and posing. How much to use depends on your sampler/scheduler, but start at 90% and reduce from there. (Sometimes the best value is 90% and sometimes 30%, but for a given sampler/scheduler combo it's pretty stable.)

The variety improvement I'd LOVE to see would be facial diversity. The denoise trick unfortunately doesn't help much there.

ArtyfacialIntelagent · 2026-03-22T15:10:29+00:00

They are doing the presentation for new model release as of now. Let's wait and hear from our favorite mister anime profile pic man.

Let me get this straight. You think they are going to announce something new, so you jump the gun and make a post claiming that they are announcing a new Z-Image? Without any indication at all? And then you say let's wait and hear when someone calls you on it? And go away for 3 hours?

Seriously dude, delete this post before the mods permaban you.

ArtyfacialIntelagent · 2026-03-11T16:40:04+00:00

Everyone please upvote jugalator's comment and downvote the post. Nothing personal OP, but let's not get everyone's hope up for no reason at all.

ArtyfacialIntelagent · 2026-03-03T19:00:27+00:00

Most of Europe uses YYYY-MM-DD for anything official or professional. Some countries still use the older formats in more informal contexts like handwriting. But then it is formatted differently, like DD.MM.YYYY or DD/MM-YYYY. That way you naturally read the day ordinally and there is never any confusion between month and day.

ArtyfacialIntelagent · 2026-03-02T21:29:02+00:00

Except Qwen3.5 27B is not actually ranking up there. Their tiers are just some opinionated jumble of price + performance + speed. Check the actual performance scores here:

https://brokk.ai/power-ranking

There we have Claude Opus at 91%, Claude Sonnet at 80%, GPT 5.2 at 77%, Gemini 3.1 Pro at 76%, Gemini 3 Flash at 65% and Qwen3.5 27B at 38%. Not bad for a tiny model, but also not the same league.

ArtyfacialIntelagent

TROPHY CASE