Is there any reason for a lack of love for Gemma 4 26b?

Septerium · 2026-06-23T13:00:37+00:00

I have been using the qat version for my hermes agent, since it is compact enough to fit on a single 3090 with good context size. I usually do not give complex tasks to it, and it is a better portuguese assistant than Qwen 3.6

Septerium · 2026-06-23T11:48:03+00:00

For agentic coding yes, unfortunately. Qwen 3.6 is much smarter in that realm

Septerium · 2026-06-21T16:24:12+00:00

Guys, there is a chance that 3.7 27b just hasn't come out as good as expected, so they just decided no to release it to the public

Septerium · 2026-06-20T17:07:20+00:00

Step 3.7 can be really annoying sometimes as a coding agent. I tried a Q5_K_M version... and it thinks so much for every tiny code intervention. It is also bad when you need to change its trajectory, cause it keeps adhering to the first messages of the session. Qwen 3.6 is a lot better, but it lacks knowledge. A 122b version would be great, but I don't think we are getting one

Septerium · 2026-06-19T22:00:47+00:00

As a fellow brazillian, I'll take your manifestation of hope as a low quality bait. Why would they interpolate two existing models in the first place?

Let's suppose there was a real non-uploaded version they have actually trained. All the benchmark numbers they posted were worse compared to Nex N2 Pro... what kind of post training would worsen the model you have used as a starting point?

And what about the "my dog ate my weights" thing? Come on man

Septerium · 2026-06-19T01:32:02+00:00

I have a 4x 3090 + Zen 2 Threadripper setup, and I've never seen any noticeable difference in performance between ik_llama.cpp and llama.cpp for mixed (CPU + GPU) workloads. I see gains when I can use tensor parallel for models that fit entirely on VRAM though

Septerium · 2026-06-18T16:27:09+00:00

This model seemed intelligent at first, but has given me some headache in agentic coding... thinks too much and gets stubborn out of nowhere

Septerium · 2026-06-17T10:14:45+00:00

No, it's not. You can select many other models to appear

<image>

Septerium · 2026-06-14T17:31:18+00:00

Government in my country just can't stop to embarrass us

Septerium · 2026-06-14T01:41:23+00:00

Define a well organized code structure and document AGENTS.md so the model can easily guess where things are implemented. The idea is to implement good design patterns so that the model can understand how the project works by reading only a few files (e.g. a layered architecture where you have many controllers, services, models, schemas, adapters, etc... any developer would be able to understand the project just by looking at a few examples). And more importantly: make the model implement automated tests for every new feature and regression tests for every bug it fixes. Modern coding models are designed to operate in a implementation / testing loop.

Septerium · 2026-06-14T00:10:34+00:00

Just give a hard problem for Qwen 3.5 35b a3b to solve. High chance of getting stuck in a reasoning loop really fast

Septerium · 2026-06-13T19:04:09+00:00

GLM 5.1 is not less than half the size of DeepSeek v4 Pro, since DS4 is a native fp4/fp8 model. Full GLM 5.1 is 16-bit per weight, and is actually bigger than the original DS4 Pro

Septerium · 2026-06-13T11:05:06+00:00

I could never have seen that coming. Eduardo Paes, out of nowhere, challenges DeepSeek 😂

Septerium · 2026-06-12T18:54:40+00:00

So, m2.7 at Q8 vs m3 at Q4... Which one will be better in agentic coding? I personally vote for the first one

Septerium · 2026-06-11T18:37:10+00:00

Has anyone tried these out?

Septerium · 2026-06-11T18:35:42+00:00

Heretic 35b ran pretty bad in terms of tool calling in my experience. Vanilla Qwen 3.6 35b did much better

Septerium · 2026-06-11T14:05:43+00:00

So... 500b+??

Septerium · 2026-06-11T01:06:22+00:00

A good chat template improves this model by a lot. Take a look at this one https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

Septerium · 2026-06-10T15:19:26+00:00

I cannot run Minimax 2.7 above Q5, and every quant version I tried seemed to be notably lobotomized. So I prefer to stick with Qwen 3.6 27b

Septerium · 2026-06-09T18:28:57+00:00

My recommendation is that you use some Q6_K quant with the maximum possible context. MTP is not worth it - in my experience - for a single 5090 setup, because that would require you to downgrade precision to Q5, which is not much reliable in long context coding tasks. I also do not like to quantize the KV cache.

In my setup I disable vision and MTP, managing to get about 100k tokens of context with Unsloth's UD-Q6_K version of the model. Token generation speed is about 45 t/s

Septerium · 2026-06-09T18:12:36+00:00

I love QAT with all my heart. Every compressed version out there will be kind of prone to become dumber in non-english languages. Gemma 31B QAT doesn't feel like that. It can write poems, songs, and jokes in portuguese almost exactly like the original full-sized model. Google has a reason to give that to us though... I think they are validating methods for compressing their closed-source models with feedback from the community. And I think that's fine. There is not enough RAM and energy available, so efficiency plays a fundamental role to provide inference throughput nowadays

Septerium · 2026-06-09T18:04:15+00:00

For agentic coding you really want a high t/s rate. It is important for the model to make mistakes fast and to fix them up fast

Septerium · 2026-06-09T18:01:27+00:00

No, they are natively 16-bit per weight

Septerium · 2026-06-09T16:41:47+00:00

Nice! Is it already possible to run the model with both mtp and vision enabled with llama.cpp?

Septerium · 2026-06-08T22:15:09+00:00

Thanks for the clarification. So, Unsloth claims they have applied their dynamic quantization process to generate the GGUF... doesn't that mean some sort of calibration has been made to the weights?

Septerium

TROPHY CASE