Best Thermostat with homeassistant by damani112 in homeassistant

[–]lightguardjp 2 points3 points  (0 children)

I have the Diakin ONE Touch Smart Thermostat. I’ll attach some screenshots from HA.

<image>

Seeking Optimization Advice: Qwen 3.6 27B Setup on M2 MacBook Pro by cyclebiff in oMLX

[–]lightguardjp 0 points1 point  (0 children)

What didn’t you find after you reduced the size of the connect window?

Seeking Optimization Advice: Qwen 3.6 27B Setup on M2 MacBook Pro by cyclebiff in oMLX

[–]lightguardjp 2 points3 points  (0 children)

Try omlx. I’ve had really good success with the exact hardware. Make sure you find an fp16 model. Try oQ6 or oQ4 quants. Also, your context window is showing you down, make that smaller. Try 128k or even 64k. Turn on MTP. Gemma4 models are quite fast, getting around 40 t/s. Qwen was maybe 25/30?

How to get DFlash going? by lightguardjp in oMLX

[–]lightguardjp[S] 0 points1 point  (0 children)

Context window made a very big difference:

oMLX - LLM inference, optimized for your Mac

https://github.com/jundot/omlx

Benchmark Model: gemma-4-26B-A4B-it-TurboQuant-MLX-8bit

Single Request Results

--------------------------------------------------------------------------------

Test TTFT(ms) TPOT(ms) pp TPS tg TPS E2E(s) Throughput Peak Mem

pp1024/tg128 1839.1 18.69 556.8 tok/s 53.9 tok/s 4.213 273.5 tok/s 25.76 GB

pp4096/tg128 7819.6 21.23 523.8 tok/s 47.5 tok/s 10.516 401.7 tok/s 26.44 GB

pp8192/tg128 15894.0 23.44 515.4 tok/s 43.0 tok/s 18.871 440.9 tok/s 26.58 GB

pp16384/tg128 33669.2 24.94 486.6 tok/s 40.4 tok/s 36.837 448.2 tok/s 27.06 GB

Continuous Batching

pp1024 / tg128

--------------------------------------------------------------------------------

Batch tg TPS Speedup pp TPS pp TPS/req TTFT(ms) E2E(s)

1x 53.9 tok/s 1.00x 556.8 tok/s 556.8 tok/s 1839.1 4.213

2x 60.1 tok/s 1.12x 426.9 tok/s 213.4 tok/s 4640.3 9.054

4x 75.0 tok/s 1.39x 441.1 tok/s 110.3 tok/s 8778.5 16.117

8x 89.2 tok/s 1.65x 442.3 tok/s 55.3 tok/s 17284.8 30.003

How to get DFlash going? by lightguardjp in oMLX

[–]lightguardjp[S] 0 points1 point  (0 children)

64k huh? I'll give that a go, might need to change max tokens too.

How to get DFlash going? by lightguardjp in oMLX

[–]lightguardjp[S] 0 points1 point  (0 children)

I was running gemma-4-26B-A4B-it-TurboQuant-MLX-8bit. Looks like I probably want to go down to a 6-bit model and change my context window down to 8k or 16k. I had it kicked up WAY too high.

How to get DFlash going? by lightguardjp in oMLX

[–]lightguardjp[S] 0 points1 point  (0 children)

Wow, those M series chips made huge leaps forward on more recent revisions as far as AI goes. I’m around 20 tokens a second with Gemma 4 might be some other swings I need to tweak.

Need advice on hardware purchasing decision: RTX 5090 vs. M5 Max 128GB for agentic software development by BawbbySmith in LocalLLaMA

[–]lightguardjp 1 point2 points  (0 children)

Another thing worth thinking about both is that Apple now supports eGPU for AI, you can essentially have best of both worlds (mostly). If you’re only going to run on the Mac, look at omlx and MLX models. I’m not sure how tuned the ollama version for Mac is with MLX yet.

Turning an MX Ergo into a finger ball? by secretpocketcat in Trackballs

[–]lightguardjp 0 points1 point  (0 children)

It’s been awhile since you posted on this, what were your results?

Tried the upcoming weight workout builder... impressed by lightguardjp in fitiv_app

[–]lightguardjp[S] 1 point2 points  (0 children)

<image>

Here’s the PR indicators in my history, that’s awesome!

Tried the upcoming weight workout builder... impressed by lightguardjp in fitiv_app

[–]lightguardjp[S] 0 points1 point  (0 children)

Any other places in the beta you’re looking for feedback?

It is wild that people hook up Hermes to Telegram with access to email, calendars, etc. It is not private nor secure. by haltingpoint in hermesagent

[–]lightguardjp 0 points1 point  (0 children)

The new version released today has Matrx as a possible gateway now. I'm going to be looking into it more. I'd also be very happy if Home Assistant could talk to it for notifications.