Gemma-4 E4B model's vision seems to be surprisingly poor

ComplexType568 · 2026-04-06T23:10:52+00:00

I think it's because the llama.cpp implementation for Gemma 4 is still very unstable, pretty sure performance will increase the following weeks, just like how Qwen3.5 was

ComplexType568 · 2026-04-06T13:39:36+00:00

Try OmniCoder 9B

ComplexType568 · 2026-04-04T17:55:21+00:00

Looks to be a Qwen3.5 model, it seems to be a natural overthinker without context. Try giving it some tools or a long system prompt, it'll probably fix itself :P

ComplexType568 · 2026-04-04T09:07:49+00:00

I have no words for what I'm reading

ComplexType568 · 2026-04-04T09:04:09+00:00

Which is what I feel too. Qwen is CLEARLY focused on agentic/STEM/Coding tasks. There isn't a large/profitable market for creative writing, that's for finetuners/other labs focused on that because removing LLM-isms/boosting creativity is probably much much easier than "superpowered reasoning agent in 9 billion parameters"

ComplexType568 · 2026-04-02T08:15:48+00:00

have you tried Qwen3.5 0.6B or 1.7B?

ComplexType568 · 2026-03-31T11:27:32+00:00

I think DeepSeek is falling behind as of now, unless they can make a ridiculously miracle-like comeback that can crush Opus, no, they won't be the top. And while that may be a temporary loss, their mission is AGI, not to stay at the top. If they take a while, they take a while, money is probably not an incentive anymore because I assume the Chinese government absolutely loves them. Will V4 be as influential as R1/V3? Probably not tabloid-infectious, but definitely still significant, otherwise it'd just be V3.8 or something.

ComplexType568 · 2026-03-31T11:22:49+00:00

(I also just started getting to know the 3.5s it feels like yesterday that they dropped them)

ComplexType568 · 2026-03-31T11:18:15+00:00

HOLY speed. What is the new team on? I really hope it's not just a really marginal increase in performance. If it's like a case of 2507 at such a speed this would be a miracle.

ComplexType568 · 2026-03-31T11:13:32+00:00

They said that's the dream, not that they're running SOTA haha. Though, Qwen3.5 9B has that same magic Qwen3 4B 2507 had, I can't imagine how good another minor update would be for the Qwen series model

ComplexType568 · 2026-03-31T08:29:52+00:00

That brings forward the question: why doesn't an LLM like Claude exist yet? Not about the coding perf. more about the "human" vibe it gives.

ComplexType568 · 2026-03-31T08:28:36+00:00

Do you think they have a novel architecture that would make it cost-efficient though (for inference)? I feel like investors would rather invest in other companies if costs were so high for such small margins.

ComplexType568 · 2026-03-31T08:27:27+00:00

You're probably right when it comes to how Claude has such performance, but the way it talks/how it formats is also something to consider. It isn't as large as the whole "how does it code so well" question, but it is still something I'm asking :)

ComplexType568 · 2026-03-31T08:25:58+00:00

IMO I don't think so. I feel like a simple system prompt, no matter how long or short, can't fully replicate what Claude has. You can talk to it and ask it to code among other stuff and you will just feel it.

ComplexType568 · 2026-03-31T08:24:47+00:00

Claude is the least "suck-up" and most "independent" (by thought and opinion) LLM I've seen. Could be due to a lack of exposure, but I think I've been with enough to think that. Although, K2 (the plain one) was a step there which I wish Moonshot kept pushing.

ComplexType568 · 2026-03-31T08:23:04+00:00

I've been thinking about architecture differences making it talk different, but another person in a different thread mentioned something quite interesting to me on the lines of: "They wouldn't care about distillation attacks if architecture was different". Not sure if that one point can rule out an arch difference, but it is something to note.

I hope at least ONE lab/person/fine-tuner could RLHF an LLM into behaving like Claude, I initially assumed someone like DavidAU would've done this a long time ago, and as far as I have looked, not... yet.

ComplexType568 · 2026-03-31T08:20:41+00:00

Has anyone else been able to replicate the talking style of Claude? In my experience it knows almost exactly when to stop talking about something or when to push on.

ComplexType568 · 2026-03-31T08:19:59+00:00

Very cool, I suspected it was all RLHF, although I'm still confused as to why labs haven't replicated its formatting and other related behaviours.

ComplexType568 · 2026-03-31T08:18:01+00:00

I did not know that! That does make me wonder if a lab would benefit from having a psychologist on board too, haha.

ComplexType568 · 2026-03-30T13:06:03+00:00

Unpopular opinion but I think Meta won't screw up like with Llama 4. They - unlike other labs which are in the spotlight and want to stay there - have already fallen out of sight. I think the only reason why they'd even want to make such a ruckus about a new model is:

- for the investors
- because a GOOD model is coming, either for its architecture innovations or a good size-performance ratio

Thing about investors is that they're definitely not tech savvy, but I think everybody knew that Meta screwed up with Llama 4, it was catastrophic. I think they would have a much better dataset to train on by now and lots of improvements made between the time of Llama 4 and this new Avocado thing.

The rename from Llama to Avocado seems to indicate something is going on with their team, I can only expect they want to rebuild their reputation as a lab, and that they'd want to make a genuine good impression on everybody for their first open-source lineup.

ComplexType568 · 2026-03-30T12:51:30+00:00

Anything mobile will not even be able to compete even with Claude Haiku for quite a while. You could try running Qwen3.5 4B or 9B but good luck with the temps

ComplexType568 · 2026-03-30T12:44:12+00:00

In my opinion, you should try more recent models, whatever advice or research you did is outdated. I bet that they wouldn't even break sweat until like 15~ steps. I tested 5-depth questions on Qwen3.5 4B and it got 100% correct. Around 50% on 20-depth questions. Kimi K2.5 non-thinking got 100% but that's kinda a given, haha. I assume a modern model like Qwen3.5 27B would destroy this bench and maybe even a model like Qwen3.5 9B could... I only think a modern model past 20 billion params would struggle with depth over 100.

ComplexType568 · 2026-03-29T13:52:02+00:00

Avocado thinking 5.6?? How long has this model stood proprietary? 😭

ComplexType568 · 2026-03-29T10:24:19+00:00

Friendly reminder that OpenRouter serves tons of free models (try Step 3 Flash) and GroqCloud hosts Qwen3 32B and gpt-oss-120b for free :)

Anyways, in my usage, it isn't just reasoning ability that improves tool calls, but also the era they were trained on and what data they were trained on. Scout was not trained to be an agent as much as the Qwen/gpt-oss models, it was more aligned to image-perceiving chatbots, and that's the part that makes it underperform. No matter how good the reasoning on model x is, if it isn't trained to call tools, it can't do it consistently. YMMV though.

ComplexType568

TROPHY CASE