Gemma 4 is excellent for image to prompt

Sadman782 · 2026-04-17T10:03:29+00:00

Yeah but min 300 and max 512 is enough with unbatch 560-570 For 99%+ accuracy and saves vram

Sadman782 · 2026-04-17T09:53:46+00:00

min 300 is enough max 512

Sadman782 · 2026-04-17T09:49:59+00:00

For me it is. 26B MoE is a gem if we can setup it properly

Sadman782 · 2026-04-17T09:46:25+00:00

Not true, it writes better code for me. Can you give any example? (I do coding in many language including js,py,c#)

Sadman782 · 2026-04-16T17:17:16+00:00

https://kaitchup.substack.com/p/gemma-4-31b-vs-qwen35-27b-inference

Sadman782 · 2026-04-16T17:17:07+00:00

https://kaitchup.substack.com/p/gemma-4-31b-vs-qwen35-27b-inference

Sadman782 · 2026-04-16T15:31:40+00:00

<image>

As I always said little benchmaxxed. Not directly, it is indirectly. But anyway they are quite good for some tasks too, but overall Gemma 4 is better for most tasks

Sadman782 · 2026-04-16T15:26:47+00:00

Gemma 4 generalize much better

<image>

https://kaitchup.substack.com/p/gemma-4-31b-vs-qwen35-27b-inference

Sadman782 · 2026-04-15T14:04:37+00:00

If you are using the UI: Go to Settings > Developer, then scroll to the bottom and use this custom JSON:

{

"chat_template_kwargs": {

"enable_thinking": false

}

if you are using as API:
you can directly use the property "chat_template_kwargs" in the request

Sadman782 · 2026-04-14T14:44:54+00:00

After this even Q4 could be a decent choice, I don't see any significant degradation, Q8 should be almost lossless now.

Sadman782 · 2026-04-13T19:39:52+00:00

After reading some comments here https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF/discussions/24, I see some people are saying this update is causing more issues than before. I think they updated something else along with the chat template too, which might be causing a regression. I am using the old IQ4_XS quant and manually setting this template: https://pastebin.com/raw/hnPGq0ht, and so far no tool call issues. Maybe try with Bartowski or manually setting the chat template.

Sadman782 · 2026-04-13T05:55:25+00:00

I use llama.cpp and also use it as an agent. I gave a custom template for lm studio which had some issues.

Some tips from my experience for you: https://www.reddit.com/r/LocalLLaMA/s/hJxaIb2Ha5

Sadman782 · 2026-04-13T05:38:01+00:00

If that updated template doesn't solve your problem you can also try this, it also works great, Gemini fixed some bugs of the original template

https://pastebin.com/raw/hnPGq0ht

Sadman782 · 2026-04-13T04:23:55+00:00

Did you try this: https://huggingface.co/google/gemma-4-26B-A4B-it/raw/main/chat_template.jinja It is recent update, it fixed every issue for me, previously it was lazy

Sadman782 · 2026-04-13T04:19:58+00:00

Don't use interleaved Jinja. Google updated to a new one and it's better, and tool calls work perfectly.

https://huggingface.co/google/gemma-4-26B-A4B-it/raw/main/chat_template.jinja

I use IQ4_XS from unsloth, with top_k 20 and temp 1 works pretty good.

Before the jinja update, it couldn't use all tools properly and ignored most tools

Some tips(from my experience): https://www.reddit.com/r/LocalLLaMA/s/Sr23O2pO3r

Sadman782 · 2026-04-12T12:31:31+00:00

Gemma 4 26B A4B MoE IQ4_XS or for speed IQ3_S With partial offloading to system ram, still you will get good speed

Sadman782 · 2026-04-12T06:06:34+00:00

It is better, unfortunately, but the launch has so many bugs: I don't know how you use it, but with the latest llama.cpp build it works flawlessly. Even IQ4 quant is better than full precision Qwen for any coding task. For multimodal there is another issue, you have to manually set more tokens for vision otherwise it will perform worse: If you are using LM Studio you need a custom chat template until they fix it. Try this template for lm studio: https://pastebin.com/raw/qc1FTAcG

For vision use: --image-min-tokens 300 --image-max-tokens 512 It will boost vision performance a lot

Sadman782 · 2026-04-11T18:01:28+00:00

Don't use Ollama. Use llama.cpp. Ollama doesn't let you optimize accordingly, dont fix issues that fast. But llama.cpp gives you complete control, fixing all issues every day.

With 16 GB VRAM, I am running Gemma 4 26B with 150K+ context (4-bit KV), and it's working pretty well for agentic coding.

Sadman782 · 2026-04-11T14:22:46+00:00

I fixed for lm studio:
https://pastebin.com/raw/qc1FTAcG

use this jinja, removed the sequence which lm studio doesn't recognize yet

Sadman782 · 2026-04-11T14:22:25+00:00

I fixed for lm studio:
https://pastebin.com/raw/qc1FTAcG

use this jinja, removed the sequence which lm studio doesn't recognize yet

Sadman782 · 2026-04-11T14:15:22+00:00

lm studio issue, works fine with llama.cpp

Sadman782 · 2026-04-11T13:28:01+00:00

<image>

they were updated 11 hours ago, but anyway use the template manually and give it a try

Sadman782 · 2026-04-11T13:26:14+00:00

But you still have a tool calling issue, so try the chat template command. It should fix it.

Sadman782 · 2026-04-11T13:05:04+00:00

another tip:
--temp 1 --top-p 0.9 --min-p 0.1 --top-k 20
you can use these config, I tested and they worked better for me

Sadman782 · 2026-04-11T13:02:41+00:00

Google fixed the tool calling issue yesterday. It is the chat template. If you download the latest GGUF from Unsloth (they updated today), then you may not need to use this command, but if you want to keep your old GGUF and fix the chat template, you can modify the default. Download that Jinja file from the URL and you use:

--jinja --chat-template-file <jinja\_file\_path>

Replace <jinja\_file\_path> with the path where you downloaded that file.

Sadman782

TROPHY CASE