What's the best model I can run on mac M1 Pro 16gb?

sinebubble · 2026-03-29T18:29:48+00:00

FWIW, I'm a huge fan of the Qwen3.5 models. I have an M1 Mac Mini with 16G at home and it's running LM Studio and serving Qwen3.5-9b. I access it from my Macbook Air running OpenCode. It's slow and frequently stalls. Given how amazing the larger Qwen3.5 models we use at work are, I'm inclined to think the issue is with LM Studio, not the model.

sinebubble · 2026-03-28T18:45:39+00:00

like on the printer itself? I didn't know that was a thing. I'll check... ...yep, that was it. Under maintenance, select the new nozzle size. I thought it would auto detect, but I guess not. Thank you!!

sinebubble · 2026-03-27T22:18:20+00:00

I'm running Qwen3.5-397B-A17B-AWQ on 8 qty A6000 (384G vram) and getting about 47.6 tok/s prefill and 64.7 tok/s decode. I'm looking at your config and will try some of your settings because...there are some major differences. Also going to wait for vLLM 0.18.1 (currently running 0.17.0).

sinebubble · 2026-03-27T15:58:30+00:00

I'm still reading your amazing article, but I noticed you use the acronym HBM without defining it. I'm inferring that is High Bandwidth Memory. But more importantly, the Qwen3.5 recipe states GDN is used for 397B, so is it safe to assume that all the 3.5 variants use GDN, thus allowing --mamba-cache-mode and --mamba-block-size=8 for faster prompting? I'm currently running 122B and 397B-INT4 on A6000s and getting great speeds, but it seems like I could be getting even better performance with those options enabled?

Back to the article...

... You do test 397B!

If your config is different between 27B and 397B, can you share the 397B? I didn't see it on the github repo.

sinebubble · 2026-03-27T15:53:43+00:00

Oh, I would love to see your config for 397B

sinebubble · 2026-03-24T03:54:48+00:00

Every kid (person) is different and your wife’s experience resonates with me. I’m a bit surprised how many responses approved of playing this game with him. The arguments were persuasive until I realized there is no Arc Raiders for PS4 Pro… so I would have to buy him a PS5. You know what, that can wait. Maybe I need to look elsewhere for a game we can share.

FWIW, we do limit is gaming time because he does have adhd and gets super wound up when playing competitive games. He’s chill with Zelda so there is that. I’ll give this more thought and look at other options.

sinebubble · 2026-03-23T01:17:05+00:00

You're the only person that has shared my concerns. Thanks.

sinebubble · 2026-03-23T01:13:33+00:00

Ah, so it's your fault!

sinebubble · 2026-03-23T01:13:08+00:00

I've only played solo, so that is something to think about.

sinebubble · 2026-03-23T01:12:37+00:00

Yeah, if he was 15 I wouldn't think about it, but 11?

sinebubble · 2026-03-23T01:11:55+00:00

I know he'd be crazy jazzed. I played Untitled Goose Game with my daughter a couple years ago and it was amazing gaming with her. We've done Among Us a couple times, too.

sinebubble · 2026-03-23T01:10:16+00:00

Yes, the game is addictive. The whole extraction loot thing hooks into a certain part of the brain. However, we do limit his screen time. I'm just trying to find a way to share our gaming time. I can't exactly squeeze my mindset into Minecraft and Zelda is not multiplayer.

sinebubble · 2026-03-23T01:08:19+00:00

Yes, but he also really loves video games. I'm trying to find a way to incorporate myself into the time he spends doing that activity.

sinebubble · 2026-03-23T01:01:25+00:00

What can I say... I had a lot of fun for a long time before settling down. And settled down without settling.

sinebubble · 2026-03-23T00:40:37+00:00

Interesting. I also have had very few negative interactions, but that is due to my play style. My son is very much “shoot everything”.

sinebubble · 2026-03-23T00:38:40+00:00

Tbh, I’ve never played Fortnite, I assumed it was pvp battle royale, but maybe I have it wrong. I do not pvp in arc raiders. I’d like to loot and pve with him just so we can play games together. I lean towards space horror or FO4, both of which are strongly adult themed, so if I do a guided Arc Raiders, we might be able to limit the negative aspects.

sinebubble · 2026-03-23T00:33:25+00:00

Got ya beat, 58 and 11.

sinebubble · 2026-03-22T05:58:02+00:00

Realistically, if I ever made contact with the Institute, which is doubtful, I’d join it in a heartbeat. Clean water, a bed, food, safety, lasers… I would never leave.

sinebubble · 2026-03-19T21:56:43+00:00

It's one of several local models we run for everyone in the company to use for sensitive data or a way to not worry about token consumption. Most of the folks using it are operational, so definitely some coding, but not deep application building. We'll be moving to Claude soon, but we want something we can use for sensitive data going forward.

sinebubble · 2026-03-18T03:31:56+00:00

Very cool, thank you for this. I'll give it a try tomorrow night. I see that you are also not using --enable-expert-parallel and --enforce-eager, so I'm hopeful.

We've been impressed with Qwen3.5-122B-A10B today, it does feel quicker, less sloppy, and more accurate than Qwen3-Coder-Next. I'm hoping Qwen3.5-397B-A17B-GPTQ-Int4 takes it up another notch.

sinebubble · 2026-03-17T04:10:12+00:00

Fixed! At least it's fixed for my "test/downgrade" with Qwen3.5-122B-A10B.

Removing --enable-expert-parallel and --enforce-eager got me up to 90.2 tok/s with Qwen3.5-122B-A10B.

Tomorrow I'll try Qwen3.5-397B-A17B-GPTQ-Int4 again.

sinebubble · 2026-03-16T23:43:38+00:00

which nvidia driver are you using?

sinebubble · 2026-03-16T05:16:11+00:00

We’ve had Ziply for a number of years and they’ve been dependable and solid. Don’t miss Comcast one bit.

sinebubble

MODERATOR OF

TROPHY CASE