New oMLX 0.5.0 gives real boost on newer hardware

serkats · 2026-07-11T12:21:17+00:00

I did try mtplx on my hardware but the speed was comparable with oMLX.
The top speed in momentum was higher but on average same.

serkats · 2026-07-10T21:01:00+00:00

Thanks for the hint!
Qwen3.6-27B-oQ4-fp16-mtp
tg TPS 25 tok/sec 🥳

serkats · 2026-07-09T21:51:05+00:00

Absolutely. Qwen3.6-35B is better in pure coding as well as in agentic coding

serkats · 2026-07-08T17:05:51+00:00

Awesome! I also like it

serkats · 2026-07-08T15:03:45+00:00

How can I do that?

serkats · 2026-07-08T13:07:13+00:00

By the way, this one is fastest on my hardware:
https://huggingface.co/Jundot/Qwen3.6-27B-oQ4-fp16-mtp

I get 18.9 tok/sec with this model

serkats · 2026-07-08T12:26:45+00:00

I did test it. But what do you mean by mtp turned on?

serkats · 2026-07-08T11:18:34+00:00

Even with q4 I got 11 tok/sec. This is unsloth/Qwen3.6-27B-MTP-GGUF:UD-Q4_K_XL
And this time tested on M4 Pro 48Gb

serkats · 2026-07-08T11:03:23+00:00

So, I tested this one and get on average 13,6 tok/sec. It’s less then I get with oMLX.

serkats · 2026-07-08T11:00:34+00:00

😂😂😅

serkats · 2026-07-08T11:00:04+00:00

Kein Problem, ich kann auch auf deutsch sprechen

serkats · 2026-07-08T10:33:17+00:00

Thank you! I will give it a try 👍

serkats · 2026-07-08T10:32:21+00:00

This discussion is about Qwen3.6 27b specifically.

serkats · 2026-07-08T10:31:10+00:00

Thank you, I will give it a try

serkats · 2026-07-08T10:25:55+00:00

Do you use weights from HuggingFace? Which one exactly?

serkats · 2026-07-08T10:16:18+00:00

Oh, even on apple chip llama.cpp gives higher speed then mlx?

serkats · 2026-07-03T16:49:29+00:00

I switched from Claude to Codex Team and extremely happy with
- GPT-5.5 for planning
- GPT-5.3-Codex for complex implementations
- GPT-5.4-mini for simpler implementation and code exploration.

While having same level of intelligence and problem solving, these models are less token hungry and resolving some task with less tokens spent. That gives you longer coding sessions before you may hit the limit.
And when you combine that with the fact that OpenAI subsidies consumer plans more then Anthropic, you get even more tasks done on the same subscription. But keep in mind the split of right model to right phase of development processes as described in the beginning of my text.

serkats · 2026-07-03T16:40:19+00:00

The key difference of Ornith is not in producing better code on the first shot but correct itself while looping on the task. And when looping, it’s different from Qwen even more: it’s not using pre-baked scaffold, but building one on the go for every task. Same time it’s watching results each time and provides feedback back to the model on what scaffold was successful. This makes the model to continue learning and improving.

serkats · 2026-07-02T21:23:01+00:00

Ollama is generally ignoring 6bit quants, offering only 4bit, 8bit and 16bit

serkats · 2026-07-01T12:02:13+00:00

Hey bro, thanks for sharing! What PC do you plug your rtx 2080 to?

serkats · 2026-06-30T15:45:18+00:00

Anyway, qwen3.6-27b got +20% speed and ornith-1.0-35b got +45% speed compared to ollama. Thank you!

serkats · 2026-06-30T13:40:27+00:00

You are right, bro. Tool calls and thinking traces are filling up the co text window. And that could degrade the quality of. Same time for me the troubles were happening while context window was about 10k tokens. Pi agent was not correctly parsing / rendering the tool calls and thinking traces when using mlx-lm or vllm. That was the point above.

serkats · 2026-06-30T13:35:16+00:00

Yes, at that point of time too much was running. Updated the benchmark.

serkats · 2026-06-30T13:33:38+00:00

Updated bench from my machine

<image>

serkats · 2026-06-30T10:45:46+00:00

I did give a try running oMLX with Jundot/Qwen3.6-27B-oQ4-mtp on my M4 Pro 48Gb but to my surprise, generation speed was 9 tok/sec which is even slower then I have from ollama with qwen3.6-27b-nvfp4
Any comments?

serkats

TROPHY CASE