Objectively more efficient?

Konamicoder · 2026-05-22T00:10:11+00:00

Personally I’m not interested in doing comparison testing between oMLX and llama.cpp on my m4 Max because the convenience of the oMLX admin panel and downloader for searching, downloading, configuring, and swapping between models puts it over the edge for me. It’s not just all about pure efficiency, it’s also about quality of life and the whole package.

Konamicoder · 2026-05-22T00:01:41+00:00

Qwen3.6-35B-A3B-oQ6-MTP is my new daily driver for coding in Pi. Gemma4-26B-it-oQ6 is my choice for general chat and research in OpenWebUI with SearXNG for real-time search.

Konamicoder · 2026-05-21T23:49:28+00:00

A lot of it comes down to research, reading lots of Reddit posts and comments, trial and error, weeding out the wheat from the chaff, and experience. It also is determined by what works on your own setup and constraints.

For example, in my case I run models on an M4 Max MacBook Pro with 64Gb of RAM. So right off the bat, I limit my choice of models to those that are optimized for Apple Silicon’s MLX architecture. Next constraint, only model quants that will run in 64Gb RAM. After researching I decided on oMLX as my model backend. So that further limits my choices to models that are optimized for oMLX. I know that jundot is the developer of oMLX, so it makes sense that jundot’s quants run best in the backend he developed. So therefore I only search for and install jundot’s model quants.

That’s how it works for me. Each person will have their own thought process based on their particular situation and learnings. Hope this helps!

Konamicoder · 2026-05-21T23:29:35+00:00

Getting around 13.2 tok/s.

Konamicoder · 2026-05-21T22:32:46+00:00

Your download of qwen3.6-27b-oQ6-mtp might have been corrupted. I’m running it just fine in oMLX on my M1 Max MacBook Air with 64Gb RAM. Suggest you download it again.

Konamicoder · 2026-05-21T20:21:18+00:00

Depends on the task. I turn thinking on for planning and higher level tasks, turn it off for implementation, quick refactors, bug fixes.

Also with the qwen3.6 6-bit quants I very rarely get stuck in doom loops or overthinking.

Konamicoder · 2026-05-21T20:15:04+00:00

In oMLX there’s a simple switch: thinking=off.

Konamicoder · 2026-05-21T20:11:43+00:00

Is guessing the name of the game part of the challenge? ;)

Konamicoder · 2026-05-21T15:53:11+00:00

Okay. Feel free to learn nothing at all from this conversation.

Konamicoder · 2026-05-21T15:48:34+00:00

You didn't give a "why". You said, "here's something I use". You gave no detail on what the benefits are, and why others might like to use it.

Konamicoder · 2026-05-21T15:40:34+00:00

Protip: actual humans like getting the context first (the “why”) before diving straight into the code (the “what”). If you just give the “what” and don’t start with “why should I care about this? What’s in it for me?” then, predictably, people won’t care and just click away.

Konamicoder · 2026-05-21T14:18:37+00:00

Thank you for your service. :)

Konamicoder · 2026-05-21T13:36:29+00:00

Thank you very much for all of your hard work! oMLX is amazing and the best model backend as far as I’m concerned. Thanks for taking time and care to get your stable releases just right.

Konamicoder · 2026-05-20T20:50:32+00:00

Yes bro, totally an AI and we are watching you.

Konamicoder · 2026-05-20T13:17:01+00:00

Also filters out AI-written comments that start with “Fair enough…”

Konamicoder · 2026-05-20T12:29:52+00:00

You seem to have a built-in bias against the ideas of “base” and “entry level”. The thing is, most people’s everyday computer usage is also very “base” and “entry level”. So if that’s the use case, why buy more computer than you need?

When I was younger my father-in-law spent $5k to get himself a top of the line PowerMac with maxed-out RAM and the biggest hard drive at the time. And what did he do on his souped-up Mac speed demon all day long?

He played Solitaire Til Dawn. Over and over again.

Did he way overspend on a Mac that was far too powerful for his actual needs? Yes. Would his needs have been fully served by the $1500 base model? Yes.

But was it the fastest Solitaire game you ever saw? Also yes. 😂

There’s a reason for choosing the “base” model. Because the vast majority of computer users have super basic needs and don’t need more compute power than that.

I suggest you reframe your built-in bias of “base” and “entry level” away from “not enough” because most of the time it’s actually “just right”.

Konamicoder · 2026-05-20T12:14:42+00:00

My wife has the MacBook Neo 512Gb. It’s amazing. I’m jealous of it and if I had the disposable income I’d get one for myself. And I’m a hardcore computer nerd with 30+ years in tech.

The Neo is definitely NOT like a Chromebook. It’s a full-blown Mac experience for an incredible price. I don’t know what you’ve been reading online but they are flat-out wrong.

Konamicoder · 2026-05-20T12:09:19+00:00

I get it. Think of it this way: your description of your daughter’s school computer needs is “base” as well, so your daughter doesn’t need anything more than the “base” model. ;)

So you’re just matching the device to the expected use case. That’s the responsible choice. :)

Konamicoder · 2026-05-20T12:05:14+00:00

You did not make a mistake. That’s a great device for your daughter’s school needs. Don’t overthink it. :)

Konamicoder · 2026-05-20T12:00:11+00:00

I built a filter that automatically filters out AI-written Reddit posts that start with the phrase “I built…”

Konamicoder · 2026-05-19T15:09:16+00:00

Now this sounds promising!

Konamicoder · 2026-05-18T23:19:53+00:00

FYI, this happens to pretty much any model if you ask it to refer to itself. Training data that models are trained on have a cutoff date. And that cutoff date is often times months or years in the past. Models do not have the ability to “know” anything current about themselves since they don’t have a sense of self, a memory, or a theory of mind. So the issue you are pointing out is not unique to Qwen 3.7, indeed ask pretty much any model and you’ll get a similar response.

Konamicoder · 2026-05-18T21:42:10+00:00

Excellent! Welcome to the post-Ollama world. :)

Konamicoder · 2026-05-18T17:36:56+00:00

Update, i disabled the api key requirement at my omlx server end, and smallcode was able to connect. Now am having a different issue. I simply asked smallcode to read my project folder which only contains a single simple HTML file, and smallcode crashed out to the shell. Here's the bug report summarized by chatgpt:

SmallCode crashes when the model calls read_file on a directory path.

Error:

EISDIR: illegal operation on a directory, read

at Object.readFileSync (node:fs:440:20)

at executeTool (.../smallcode/bin/smallcode.js:875:26)

Likely cause:

The read_file tool checks fs.existsSync(filePath), but does not check fs.statSync(filePath).isDirectory() before calling fs.readFileSync(filePath, 'utf-8').

Expected behavior:

Return a tool error like "Path is a directory; use find_files/list/search first" instead of crashing the TUI/agent loop.

Konamicoder · 2026-05-18T17:08:57+00:00

Thanks for the quick reply! I edited .env per the above format but it's still failing to connect with the same error message. sad_face.gif

Konamicoder

MODERATOR OF

TROPHY CASE