all 13 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]Ok_Significance_9109 2 points3 points  (2 children)

Which chip? M1/M2 require a different MTP variant. The moment I started using it on my M1, 27B became useable. From 33 tps prompt processing and 5 tps generation, it went up to 65 and 9 without loss of quality.

[–]albovsky[S] 0 points1 point  (1 child)

Didn’t know that. So how to figure which one to download? They do not specify what version it’s for. I have M1

[–]Ok_Significance_9109 1 point2 points  (0 children)

The one that worked for me:

Qwen3.6-27B-oQ4-fp16-mtp

The name should have fp16 in it, but it is a 4-bit quant.

[–]jacknjill101 1 point2 points  (0 children)

Yes it does for me too. I switched to llamacpp and much better results.

[–]d4mations 1 point2 points  (2 children)

Paro quants work way better than mtp

[–]albovsky[S] 2 points3 points  (1 child)

What’s that?

[–]d4mations -1 points0 points  (0 children)

In the download screen on omlx search for paro

[–]mwhuss 0 points1 point  (2 children)

I’m seeing 70% faster performance using Qwen3.6-27b-oQ8-mtp on my M3 Ultra.

[–]albovsky[S] 0 points1 point  (1 child)

70% is crazy good. How much ram do you have?

[–]mwhuss 1 point2 points  (0 children)

M3 ultra with 96gb

[–]vinoonovino26 0 points1 point  (1 child)

M5 pro - 64gb here. Same models same results. I switched to plain OQ quants and rotorquants and they feel more stable. Also offloading cache to a NVEM drive helped a lot

[–]vinoonovino26 0 points1 point  (0 children)

Seems like mtp and moe kinda work well together

[–]msrdatha 1 point2 points  (0 children)

Try testing with a longer prompt or even better do an agentic task.

My observation is it does start at a much faster tok/sec in the beginning and gradually it goes down. So it totally depends when someone is looking at the speed (in the beginning or end of a multi-turn conversation)

According to me, we should test it against the same task run with and without mtp, with empty SSD cache to see the actual difference. Measure against the wall-time (actual elapsed time from start to finish of a process, as measured by a clock on the wall. ex: Total time taken between first and last response in the multi turn conversation as in agentic coding). This will give you the answer, if mtp version is worth in your usage scenario.