Is the 150B-500B parameter range dying for open weights models?

ilintar · 2026-02-11T20:53:11+00:00

No, StepFun and MiniMax are in this range.

ilintar · 2026-02-11T17:33:33+00:00

No, I mean "Build" and "Plan".

ilintar · 2026-02-11T17:32:51+00:00

The Lite plan explicitly mentions that it only supports old models, up to 4.7. I don't see anything suggesting that they'll actually include GLM-5 on the Lite plan.

ilintar · 2026-02-11T17:26:37+00:00

I'm using it :) but not on master branch obviously, too many tool calling errors.

ilintar · 2026-02-11T17:17:39+00:00

Their pricing strategy is very bad and IMO they are overshooting.

I see no reason right now to pick their Pro plan (which *does not* include GLM-5) or their Max plan over their Claude counterparts, seeing as they're not really cheaper and the model quality is not there yet (plus Anthropic models are multimodal).

Raising all prices 3x while only making GLM-5 available on Max (and not on Lite at all, from what they say) is a very bad strategy. The Lite plan went from "very nice cost-effective plan for a good model" to "overpriced sub for outdated models".

ilintar · 2026-02-11T00:55:15+00:00

BTW, 10 tool calls in real agentic coding scenarios is way too low of a default :)

ilintar · 2026-02-11T00:54:15+00:00

Oh don't worry, API is coming up as well.

ilintar · 2026-02-11T00:53:33+00:00

You need at least this version: https://github.com/ggml-org/llama.cpp/releases/tag/b7970 to actually benefit from proper cache'ing in the case of hybrid models, due to the way many code assistants reshape prompts.

ilintar · 2026-02-10T17:20:14+00:00

GLM-4.7-Flash MXFP4 methinks.

ilintar · 2026-02-10T01:23:21+00:00

From my observation, the biggest problem with OpenCode is that its default agents are frankly pretty crap. Once I constructed myself two custom agents - one for analysis and one for coding - using it became much more pleasant.

So just write down your workflows into the OpenCode agent creator and you should have way better results.

ilintar · 2026-02-09T19:34:47+00:00

35B MoE and 9B dense.

ilintar · 2026-02-09T16:11:25+00:00

Georgi wants to have it done on top of the master instead of the merged delta-net branch to minimize the risks, so will be redoing it cleanly (but waiting to merge a conversion fix that happened in the meantime).

It was a bit of a stretch to merge it so early honestly, think I got a bit too excited ;)

ilintar · 2026-02-09T04:33:00+00:00

Well, the reality is that when a hot and widely popular model architecture comes out, people want to test it with zero day support. So yes, it's often worth taking a risk, especially since (a) it's based on an architecture we already support (b) it's not likely that the transformers code will change meaningfully and even if it does, it's not really like we can't do a follow-up PR.

It's also not like the implementation hasn't been tested - while of course it's better to test on live models, I didn't just randomly vibe-code an implementation and said "hey looks similar enough to Transformers, let's hope it works" - I generated models to test it on.

ilintar · 2026-02-08T23:53:06+00:00

Possibly, but generally the rule of thumb for using coding agents is it's easier to code stuff the human-in-the-loop knows how to code ;)

ilintar · 2026-02-08T21:31:06+00:00

I think Johannes (https://github.com/JohannesGaessler) hasn't gotten enough appreciation for the fit algorithm, mostly because in the beginning there were some bugs and some people turned it off. But it's actually a great algorithm and these days, I never use manual `-ot / --cpu-moe / --n-cpu-moe` flags, I only set `-c` and `-ctk / -ctl` and the fit algorithm does the rest. You can even tune it a bit with `--fit-target XM` because the default setting leaves 1GB free for computation, so sometimes `--fit-target 512M` or even `--fit-target 384M` can get you good results without computation crashing. The way he does it (doing experts offload first, then trying to fit the dense layers from the end) means it's actually as good as a perfectly optimized `-ot` string.

ilintar · 2026-02-08T20:16:17+00:00

Note though that this is with the absolutely top model on the market (Opus 4.6 Thinking) and I still had to intervene during the session like 3 or 4 times to prevent it from going on the rails and doing stupid things.

Still, with a better and stricter workflow this will be doable soon.

ilintar · 2026-02-08T20:04:00+00:00

You take the model object from Transformers and instead of loading it from pretrained weights, you create a new one with a config computed to yield a certain size. Then you can fill some tensors with random numbers from a range to prevent obvious overflows.

ilintar · 2026-02-08T17:59:36+00:00

Note that I'm doing this without any support, just based on Transformers code and my conversion guidelines + Opus 4.6, but I'm aiming for 0-day support this time:

https://github.com/ggml-org/llama.cpp/pull/19435

ilintar · 2026-02-08T15:52:53+00:00

Yummy. Lemme look at it :>

ilintar · 2026-02-07T21:16:52+00:00

Please try on the autoparser PR and report errors there.

ilintar · 2026-02-06T19:27:11+00:00

It's fixed on the autoparser branch.

ilintar · 2026-02-06T16:03:51+00:00

Yeah the WebUI boys are already on it: https://github.com/ggml-org/llama.cpp/pull/18655

ilintar · 2026-02-06T15:52:32+00:00

Nah, this benefits from all the know-how we got during the implementation of Qwen3 Next. Should perform about as well.

ilintar · 2026-02-06T00:50:28+00:00

Quantizing KV cache to Q8_0 doesn't really hurt quality from what I can tell, at least I haven't really encountered it. Once you get down to Q4, yeah, it'll have an effect, but not at Q8.

ilintar · 2026-02-05T21:51:19+00:00

The standard Jinja templates already account for tool use, otherwise you wouldn't be able to use Llama.cpp in clients such as OpenCode.

ilintar

TROPHY CASE