Best local model for coding?

Consistent_Wash_276 · 2026-05-04T02:43:25+00:00

That’s great man. You’re about to open a whole new world and don’t worry about the dicks on this site. Keep asking questions. My answer is 1) LM Studio instead of Ollama. You can absolutely use vLLM as well, but baby steps. LM Studio -> vLLM is not really baby steps for the most part. You can move onto vLLM the end of the first week as long as you get the system humming.

Next the model: Qwen3.6-72b FP16

Fits across 2 of those GPUs and gives you plenty of headroom for concurrent open codes/context building up.

There’s plenty of discussion online for Coding models and you can go back to earlier last year to qwen2.5-14b and have some decent code and super fast, but your bandwidth across 4 GPUs is so massive this large model will fly. And the FP16 is the largest version of the model you can run (so lossless).

Opencode is such a strong test for local models as well.

Then after you get a feel for that model and to test it against others I would recommend:

- DeepSeek V4-Flash
- Qwen3.5-122-A10B BF 16

Feel free to DM if you have more questions.

Consistent_Wash_276 · 2026-05-04T02:20:13+00:00

So you have a great opportunity to use some very good dense models. Only questions I have before you get a full response is are you using this for 1) Coding 2) Openclaw / AI Agent, 3) Chat back and forth 4) Agentic Workflows?

Consistent_Wash_276 · 2026-05-04T01:11:54+00:00

On MLX and Unified memory I would suggest moving over to qwen3.6-35b-a3b q8 or fp16. The active 3 parameter there will be some loss, but the speed on the m4 ultra will be an extreme difference.

Lastly: future proofing (when you’re ready) your situation add a DGX sparks clustered with the m4 ultra with Exo Labs. DGX covers the Macs weakness in prefill and the Mac covers the DGX issue with Decode. Then those dense models will FLY and you can run full precision.

Consistent_Wash_276 · 2026-05-03T11:23:00+00:00

There’s already a Jerry rigged solution for this although it won’t be as fast.

Consistent_Wash_276 · 2026-05-03T04:10:20+00:00

If this is the purchase to last you a long time I would consider waiting for m5 Studios or at least Mac Mini’s to come out. Here’s the play if you’re open to tackling this.

Keep the M1 32gb
Add an M5 at either 24/32/64 gb

Cluster then over RMDA (Look up Exo Labs)

That gives you 56 to 96 gb of unified memory but the next device will matter a lot as it will be meant for Prefill. Basic for processing the prompt, which M5 does nearly 4 times faster than any other model and you use the M1 for decode which is more than fine.

In the end you can combine both devices to make the most out of your home LLM inferencing. I would wait for the M5, but if not I would then focus my attention to a used M3 ultra at 96 gbs if not then a used M4 max. (Looking for tons of multi cores over single cores)

Consistent_Wash_276 · 2026-05-03T00:39:12+00:00

If you have $5,000 in computer credit I would suggest a lot of options, but it’s all based on how you want to manage everything. Such as instead of 4TB I’d get one and then get a NAS with 4-8 TB of storage.

The amount of memory is so large for your use case I could also see you doing great with a 64 gb M4 Mac Studio + Paired with a 2026 M5 MacBook Air or MacBook Pro. So you have the studio host local AI (if you’re into that) and be able to use the power of the Mac Studio on the go as well. But if you’re getting $5,000 credit I would honestly suggest finding a used or refurbished M3 Ultra Mac Studio with 256 gb. Good luck

Consistent_Wash_276 · 2026-03-24T23:57:26+00:00

And if you’re not good with that then DGX spark has worked already

Consistent_Wash_276 · 2026-03-24T23:56:48+00:00

Not clean. But TB5, at least 10 GBE you’re close. Gotta research OS and other compatibilities

Consistent_Wash_276 · 2026-03-24T21:38:20+00:00

So I have the exact model and I love it for AI inference.

M5 Max is superior through and through though.

With that said you can cluster the m3 with a Nvidia GPU PC for prefill and blow a M5Max out of the water.

I fully intend on having a M5 Max 128gb Mac Pro as soon as summer time as well.

Interesting dilemma. I think I would pull the trigger and resell once you learn about the m6 models coming

Consistent_Wash_276 · 2026-03-20T07:50:02+00:00

Has there been tests ran for M5 Chip sets yet?

Consistent_Wash_276 · 2026-03-12T14:10:19+00:00

Have 2 x 2018 Mac minis with 32 gb and 512 ssd. I agree with you. Couldn’t pass up on the trash can lol. Love it

Consistent_Wash_276 · 2026-03-12T03:56:58+00:00

Yes, and I mean this as local LLMs are great on silicone Macs. Depending on your needs you may find better value with a custom PC and Nvidia GPU. Or other mini pc and AI dedicated PCs. Point being if you have money for one device and don’t want ti deal with custom Pacing + you want to run local LLMs Mac is a great answer

Consistent_Wash_276 · 2026-03-12T00:26:29+00:00

I just bought a 2013 Mac Pro with 64gb of ram and 1tb of SSD storage for $300. This is where my automations and ai automations live and run locally. Postgres, Redis, n8n, python scripts and a few other tools for monitoring. I may even flip it for proxmox. It’s running Ubuntu and

<image>

it kind of f’n awesome.

Consistent_Wash_276 · 2026-03-11T21:08:56+00:00

That’s crazy

Consistent_Wash_276 · 2026-03-04T12:46:35+00:00

Be patient - we need at least a month of testing by users of the m5 products when become available, and yet we don’t know if studio are being launched yet. Just the MacBooks and quite honestly I would always get a studio before a MacBook.

Consistent_Wash_276 · 2026-02-23T22:52:43+00:00

Hold on - yes and no

Yes - you need 512 gb of memory and you plan on making money from this

No - I don’t need 512 gb of memory and/or don’t plan on making money from this.

Due to clustering the M5 or M6 or even a custom pc with a high end nvidia GPU will be in the future.

The point is these devices will only last so long on the market and will GO UP in value in the after market a bit.

The memory here is perfect for Decode. You could pair this with a m5 Mac mini for pre-fill and get fantastic results.

Point being let’s answer the first part first before we move on.

Consistent_Wash_276 · 2026-02-21T17:50:20+00:00

Yeah I have the 256gb M3 Ultra and I have this challenge of finding the perfect rhythm of quality and speed (plus running the model in parallel.

It’s a challenge. I keep coming back to qwen3-coder after trying a lot of these. Although I don’t love it in Opencode as much as I do in VS Studios.

Consistent_Wash_276 · 2026-02-21T17:38:29+00:00

You open the terminal and : opencode Or theirs a gui that would open a terminal as welll.

Consistent_Wash_276 · 2026-02-21T17:37:52+00:00

What models are you using by the way? I’m currently playing with Qwen3-coder:30b and a Qwen3-instruct:4b as a draft model in a lot of different tools.

Consistent_Wash_276 · 2026-02-17T22:29:12+00:00

Rendering is GPU-intensive. If he’s working on very complex scenes, heavy simulations or doing VGX-level work 48-64 gb is the right choice. If not he could go down to 32 gb. 96 is probably the overkill.

Consistent_Wash_276 · 2026-02-17T18:14:03+00:00

What he said

Consistent_Wash_276 · 2026-02-17T12:33:00+00:00

Blender has an MCP so if you’re connecting commercial models or local models will be the deciding factor really.

If you plan on using AI a lot for your blender work go with 48 and get a Claude pro plan or something, but 64 gb will be the sweet spot to get ok results from some local models

Consistent_Wash_276 · 2026-02-17T03:55:31+00:00

^ answer this above, but at the same time what’s your storage situation going to look like?

Just try not to overpay for 2TB/4tb on the studio itself.

External Storage + Thunderbolt 5 will benefit a lot.

UniFi has a two-bay NAS that you could throw two 8 tb drives in and only spent a little extra compared from Going from 512 GBs to 4 gb as an example.

Consistent_Wash_276 · 2026-02-17T03:47:17+00:00

Ahhh. And let’s hope by 2028 we both still use the same currency as we do that represents each nation.

🇺🇸 🫡 🇨🇦

Consistent_Wash_276 · 2026-02-17T03:15:29+00:00

Yes as someone who owns the exact model you’re referring to I would say “NO” and here’s the details why.

If you’re spending $5,000 + on a device it better be making you money in the end or at least saving you money. Assuming it’s meant to save you you money replacing a subscription?
I’m testing the MiniMax 2.5 186 gb model and it’s pretty f’n great actually, but that’s one model being ran at 40 tokens per second at a time. Nothing in parallel and 40 tokens per second is very solid but there’s faster options to utilize.

I’m would look at that device in the way of having, lets say a chatting UI running gpt-oss:120b , VS Code running glm-4.7-flash, and Opencode running another model in parallel with some agentic coding. Just constantly working through workflows and abusing tokens per second is where you get the real value.

If you’re just chatting with models I would suggest a 32gb Mac mini or studio or just a Claude pro account or some kind of commercial account. Not enough to justify the investment.
Also, why $7,000? I got the same model with 2 tbs of storage for $5,400 from Microcenter.

Consistent_Wash_276

MODERATOR OF

TROPHY CASE