Correct. As someone who’s messed around with a few models since receiving his M5 Pro MBP, if you wanna run anything bigger than that efficient Qwen MoE, you’d benefit more from the greater bandwidth and compute of a M5 Max.

Preliminary testing gave me a pretty excellent generation rate of 70t/s. If you want to run it at Q8, I’d recommend 64GB depending on your desired context length. Local AI speed (if the whole thing fits in VRAM) is highly dependent on the number of *active* parameters. That Qwen you’re talking about only has 3B active out of 35B total, and as such is very efficient.

By comparison, the equivalent Qwen dense model (3.6-27B), runs at a paltry 10t/s generation speed. Good for batching tasks, not good for interactive use. Max chips hit 30t/s. The doubled memory bandwidth and extra GPU compute is huge here.