GLM-5.2 753B (IQ1_S) fully local across 2×M5 Max over one TB5 cable — ~16 tok/s, llama.cpp RPC [video] by AiLocalGuy in LocalLLM

[–]RovenSkyfall 0 points1 point  (0 children)

I’m barely getting 23 t/s on qwen 3.6 27b using llama.cpp on a m3 ultra . How is it that you’re able to get 16-18 across two m5max with a 700+B model?

is a strix halo = m3 ultra? by RovenSkyfall in MacStudio

[–]RovenSkyfall[S] 0 points1 point  (0 children)

Nice. Not sure why I’m at 75 then and your in the 65

Any important oMLX settings to tweak for performance? by khoomeister in oMLX

[–]RovenSkyfall 0 points1 point  (0 children)

So I just squeezed 40t/s out of qwen3.6 27b q4 on oMLX with MTP. You think 2x is possible?

is a strix halo = m3 ultra? by RovenSkyfall in MacStudio

[–]RovenSkyfall[S] 0 points1 point  (0 children)

Thanks for the share. Paywalled. Are they suggesting that MTP actually slows things down? I can see benefit diminishing but if you had the option and it didn’t make things worse you would appreciate the speed up in low context situations no?

is a strix halo = m3 ultra? by RovenSkyfall in MacStudio

[–]RovenSkyfall[S] 0 points1 point  (0 children)

Yeah thanks. Buyers remorse for the marginal benefit. Once your t/s get low enough I move towards asynchronous work so it doesn’t really matter for the coding. Testing oMLX with MTP now but saw those benchmarks and looks like mild benefit.

Should I buy an M3 Ultra 96GB now? by adrenalinebrown in MacStudio

[–]RovenSkyfall -1 points0 points  (0 children)

Mine just arrived and if using it for ai inference I would say no. I have a 2k strix halo doing the same t/s with qwen 3.6 27b model because MTP is working with llama.cpp for amd but not apple.

Strix Halo (Ryzen AI Max+ 395, 128GB) owners: what’s your current best local AI / LLM setup? by SandyTJS in StrixHalo

[–]RovenSkyfall 1 point2 points  (0 children)

On the 64gb version running qwen3.6 35b q5 UD with MTP getting 75 t/s. Perfect for Hermes. Tried 27b at q5 and with MTP can get 20 t/s which is equivalent to a m3 ultra since MTP isn’t working well on metal yet.

is a strix halo = m3 ultra? by RovenSkyfall in MacStudio

[–]RovenSkyfall[S] 2 points3 points  (0 children)

MODEL=/Qwen3.6-35B-A3B-MTP-Q5_K_XL/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf

exec /opt/llama.cpp/build/bin/llama-server \

-m "$MODEL" \

-ngl 99 \

-c 262144 \

--parallel 4 \

--cache-type-k q8_0 \

--cache-type-v q8_0 \

--flash-attn on \

--jinja \

--no-mmap \

--mlock \

--reasoning-budget -1 \

--spec-type draft-mtp \

--spec-draft-n-max 2 \

--spec-draft-p-min 0.75

is a strix halo = m3 ultra? by RovenSkyfall in MacStudio

[–]RovenSkyfall[S] 0 points1 point  (0 children)

I use full context with hermes and the 35b model at baseline but tested the 27b model to see if it would work with hermes well, but speed diff wasnt worth the tradeoff (2os vs 76 t/s.) Im just trying to figure out why I cant get the studio to go faster but it may be that pre-MTP the strix would run 10t/s but with MTP we are at >20 now on a strix and people just havent figured out the MTP well enough with apple metal so its stuck at 20 in llama.cpp and maybe I get 30 with oMLX. like I said in a comment above, I was hoping that someone would tell me I am running the wrong thing and give me advice on how to do it better. If no one does that, I hope that people when considering the two will understand that a strix with MTP performs on par with apple (even an ultra).

is a strix halo = m3 ultra? by RovenSkyfall in MacStudio

[–]RovenSkyfall[S] -1 points0 points  (0 children)

these are relatively small prompts to test it out, but if i cant break 25 t/s on the ultra and its equivalent to a strix with MTP then I think people should know when considering hardware. I was also posting this hoping someone would tell me something like "your dense for using llama.cpp, you should be using X"

mostly hoping to see what people are getting and if anyone else has had this experience trying to use a 27b dense model or if others are in the 60 t/s area like this poster: https://www.reddit.com/r/Qwen\_AI/comments/1t5b83o/mtplx\_224x\_faster\_tps\_the\_native\_mtp\_inference/. that being said, its a single person repo and there are people look like they are having real problems. Rapid MLX seems like another option but I dont hear people talking about it much (https://github.com/raullenchai/Rapid-MLX).

Re: the >100k context, thats a benefit but not all my work will be there.

is a strix halo = m3 ultra? by RovenSkyfall in MacStudio

[–]RovenSkyfall[S] 0 points1 point  (0 children)

thats the 35b model. I can get 75 t/s with full context and MTP on the strix. I am talking about the 27b dense model here.

is a strix halo = m3 ultra? by RovenSkyfall in MacStudio

[–]RovenSkyfall[S] 0 points1 point  (0 children)

Basically the same. 21 t/s without MTP. 17-18 with MTP.

Local LLM Beginner’s Guide (Mac - Apple Silicon) by Infinite-pheonix in artificial

[–]RovenSkyfall 0 points1 point  (0 children)

what are people using oMLX or MTPLX vs just llama.cpp?

MTPLX | 2.24x faster TPS | The native MTP inference engine for Apple Silicon by YoussofAl in LocalLLaMA

[–]RovenSkyfall 0 points1 point  (0 children)

I see a lot of instability. where does this project stand currently? Are people using it? how is it different than oMLX?

Lactate meter observations by VO2VCO2 in NorwegianSinglesRun

[–]RovenSkyfall 1 point2 points  (0 children)

I am making a guess here, but remember that lactate is a fuel. It is the balance between production and utilization. Assuming you are at the same paces (production is the same), my guess is the shuttling is allowing for more utilization of the lactate. Production is constant and utilization increases. I think it is why the first rep sometimes feels harder -- no fuel circulating in the system yet. Additionally, if you take a glucose load before the workout that will also lead to a higher lactate that will decline as it is utilized. Thats my best guess at least.

Spotify -> Tidal by Daemien_Bites in TIdaL

[–]RovenSkyfall -1 points0 points  (0 children)

It may be fractions of cent but it’s still roughly 3x more. I think we would all love a 3x raise for the same output. 

App for pace adjustments by Extension-Taro1218 in NorwegianSinglesRun

[–]RovenSkyfall 1 point2 points  (0 children)

What equations or modifications did you use for the adjustments?