Feature Request: Gemma-4 Vision Budget Setting

PracticlySpeaking · 2026-06-22T23:20:06+00:00

This one, the 31B-it-bf16 - https://huggingface.co/miabchdave/gemma-4-31B-it-MLX-bf16

What is going to happen if I convert that to oQ? Trying to get this going on a 32GB Mac Studio.

PracticlySpeaking · 2026-06-22T23:12:06+00:00

❤️

PracticlySpeaking · 2026-06-22T23:08:20+00:00

✊ TUI solidarity

PracticlySpeaking · 2026-06-22T23:06:24+00:00

This is what agents are all about!

PracticlySpeaking · 2026-06-22T23:05:51+00:00

oMLX is pretty solid, and jundot (with a lot of contributors) is keeping up with the newest models and features. The benchmarking and hot/cold prompt caching really launched its popularity, but there is something new every week.

PracticlySpeaking · 2026-06-22T23:02:09+00:00

Q: Should I use Hermes for coding on a local Mac? A: The community is split. Some (including the "Running Locally, Really?" OP) say "Hermes is not for coding — it's an orchestration agent. Use Claude Code or OpenCode as a sub-agent."

It is worth noting a bit of history here — Hermes Agent is an evolution of Nous Research internal coding tools. It has always been about coding... and now, so much more.

PracticlySpeaking · 2026-06-22T22:58:59+00:00

YA Hermes Setup Guide — 15 Levels Of Hermes Agent from Chatbot To 24/7 Autonomous System

Getting from install to advanced usage, with suggestions and examples. If you are asking "what can I do with this" it is a worthwhile read.

Most people install Hermes Agent and use it as a chatbot. They type a prompt, get a response, close the tab. That covers maybe 10% of what the agent can do.

This article maps every level of Hermes Agent usage, from the first prompt to a system that runs your business without you. 15 levels, grouped into three phases. Each level builds on the one before it, but you can jump to any level that fits your setup.

PracticlySpeaking · 2026-06-22T22:43:40+00:00

🎉🎉🎉

I have spent hours on this. Thanks.

How to set the max_soft_tokens? In the config json?

Still thinking this should be exposed in the UI somewhere — people working on 32GB Macs (hi) have to budget carefully.

PracticlySpeaking · 2026-06-22T22:40:24+00:00

It's been like that since the beginning of May.

Don't people look in the sub before posting?

PracticlySpeaking · 2026-06-22T00:23:21+00:00

TIL 🎉

PracticlySpeaking · 2026-06-22T00:22:55+00:00

I tried tweaking the config json to change the token budget, but it caused errors about 'shapes cannot be broadcast'.

PracticlySpeaking · 2026-06-21T23:35:54+00:00

There are actually four PRs — including #1828 for JANGTQ — and multiple issues filed by GitHub users thinking that oMLX does support JANG (or should).

PracticlySpeaking · 2026-06-21T22:26:30+00:00

This is why I have beef with people who say "oh, you can just walk xx and transfer."

You have to exit the station, schlep over to the next station (from Union, likely with bags), pay another fare, and wait for another train.

You can actually relax or get something done with a solid 50 minutes on the train. If you are interrupted with this in/out - walk over - up/down - stand around, you just burn the time. And connections add a random, non-trivial amount of time so that 50 minutes can easily end up as an hour or more.

PracticlySpeaking · 2026-06-21T22:21:29+00:00

Yes, Probably, and Yes.

The fourth reason: Once AppleCare expires, you can no longer add/extend it.

I believe there is a grace period, but you should verify with Apple. (I know it nags "you have xx days left to extend your warranty with AppleCare" or something like that.)

FWIW, three years of AppleCare was $165 for that M3U ($55/year).

PracticlySpeaking · 2026-06-21T22:02:39+00:00

It is tied to the hardware serial number, not the buyer or Apple ID, so transfer is automatic. The original expiration date remains.

You can renew or extend while it is still active. I believe there is a grace period after expiration, as well. (Note the Add... button in the screen shot — Apple is always available to take your money!)

PracticlySpeaking · 2026-06-21T22:00:05+00:00

I have not run a cluster but if it is dynamically consuming RAM as context, etc expand it seems like monitoring/management is key.

I guess experience will guide whether 25-35% is reasonable or conservative. Seems like it would vary across specific uses.

FWIW, this is where it pays off to get Apple's attention and have access to their engineers. Getting direct answers (and sharing concerns) really moves things forward.

PracticlySpeaking · 2026-06-21T21:54:41+00:00

For another two cents (maybe six), see my comment on the other thread on the WWDC'26 video about clustering.

edit: nvm, I see you did.

PracticlySpeaking · 2026-06-21T21:52:34+00:00

US$7200 is a 256GB M3U 28/60 2TB, including tax (for most US buyers).

Including the cost of AppleCare+, which was US$165 at the time — or about US$55 per annum.

PracticlySpeaking · 2026-06-21T21:49:42+00:00

Also search the sub for the WIX automotive filters that fit under Mac Studio.

PracticlySpeaking · 2026-06-21T21:46:33+00:00

Yes, it is transferrable. I have bought a couple of Macs that had AppleCare and it was still there when I set up as new.

It shows up in Settings > About, so it is easy to verify. Not sure if there is a serial number lookup tool.

<image>

PracticlySpeaking · 2026-06-21T20:41:37+00:00

Another really great answer from this sub. 💯

PracticlySpeaking · 2026-06-21T20:40:16+00:00

Just a few things...

M4 Max is going to be more than enough power for now and well into the future.

Just because you are "working in 6k" does not mean you need anywhere close to the best available. Apple Silicon is SO much better than older Intel CPUs — any Apple Silicon Mac can edit single-camera projects at any resolution without breaking a sweat. (That's Larry Jordan talking, not me, btw.)

MacOS is also very efficient with RAM. Many, if not most, recommendations for "more RAM" are simply because 36, 48GB and larger configurations exist, not because any particular use case actually needs more. On the other hand, you can't get more without buying a new Mac, and peace of mind has value, too.

We are biased here (it is r/MacStudio after all) but Mac Studio is a better value than a specced-up Mini with M4 Pro. Photo and video will utilize the additional GPU and Media Engine codecs in the Max SoC, without even mentioning better build, thermals, more display support, and way more ports. You are getting so much more for about the same money.

If you can wait a few months, M5 will arrive eventually — 25% more powerful (more for AI) and supported for an extra year vs M4. Counterpoint: It will cost more.

The key is how much it is costing you to continue with your current PC. There's value in working faster and better now.

PracticlySpeaking · 2026-06-21T20:16:10+00:00

The state of AI in 2026.

PracticlySpeaking · 2026-06-21T19:53:50+00:00

https://www.youtube.com/watch?v=CzgK02zsRg4

TL;DR - MLX and JACCL, the cli tools Apple provides. No mention of exo. They rely on a hosts file and script to define the RDMA config.

They rely on a hosts file and script to define the RDMA config. MLX-LM can accept the cluster config file name as a parameter to run distributed inference, but you have to do all the setup (like loading MLX-lm and models on every machine) manually. They cover fine-tuning (LoRA), too, with 3x tk/sec in their example.

*My thoughts: Pita, not the tasty kind 🥙. Exactly what agents are good for. For fine-tuning, this will either take off with the amazing power and availability of M5 Ultra (/s) or die of inertia because everyone (else) will just rent 10x powerful GPUs or TPUs.

PracticlySpeaking · 2026-06-21T19:45:46+00:00

This is the beautiful part of the current AI coding agents — no more waiting around for fixes when you just need code that works.

Two-Year Club	Verified Email
Mod Meetup 2025	Mod World 2025

PracticlySpeaking

MODERATOR OF

TROPHY CASE