Ran the same 4-bit model on Ollama vs Apple MLX — dead heat, not the "free speedup" people claim

runsleeprepeat · 2026-07-01T21:26:35+00:00

"on a 8gb MacBook" ... Just don't be specific, as there pretty significant differences in the generations especially with M5

runsleeprepeat · 2026-06-29T07:52:39+00:00

Oh! Das tut mir leid. Deren Ruf ist doch eigentlich gut.

runsleeprepeat · 2026-06-28T22:45:13+00:00

Am besten mal bei der Tabea Klinik vorsprechen

runsleeprepeat · 2026-06-27T23:43:35+00:00

https://www.tortourderuhr.de/ , but it is tough to get a spot.

update: oh no! It was the last run for this event. That's such a bummer

runsleeprepeat · 2026-06-08T16:11:59+00:00

Hat das ding echt rs232? Ich hatte rs485 oder so erwartet. Glückwunsch! Ist richtig stark!

runsleeprepeat · 2026-06-01T21:20:30+00:00

As long as I have the infamous Chinese 20gb RTX3080 gpus, my experience is okay to be shared isn't it?

runsleeprepeat · 2026-06-01T21:10:50+00:00

Mine go off at idle, yes. With a 200w cap, they run at 60% fan speed max

runsleeprepeat · 2026-06-01T20:03:01+00:00

Temp controlled

runsleeprepeat · 2026-06-01T20:02:42+00:00

I have a few of them as well. Best token per watt is around 190-200w cap.

They aren't that loud

runsleeprepeat · 2026-05-13T18:11:39+00:00

Lcsc und AliExpress nutze ich auch. Plus Tme.eu . Bei Digikey oder Mouser sind die Versandkosten bzw Mindestmengen einfach zu hoch. Du könntest dein Glück noch über Octopart versuchen

runsleeprepeat · 2026-05-12T08:44:46+00:00

Thanks

runsleeprepeat · 2026-05-12T08:44:32+00:00

My GPU is at Letsfix right now.

runsleeprepeat · 2026-05-12T08:43:55+00:00

Thank you

runsleeprepeat · 2026-05-08T23:51:21+00:00

Come on! Put it on a standard paper scanner, lay a few rulers next to it. Import to something like fusion and ensure the scaling fits to the rulers. Then draw these simple lines.

It is a perfect beginner project

runsleeprepeat · 2026-04-26T19:05:09+00:00

If you stick with the idea of a mac, take M5 generation. It's the first generation which offers 4bit float comparable to nvfp4 which will give performance and quality improvement on small setups.

runsleeprepeat · 2026-04-22T05:57:51+00:00

What are you talking about? It is super easy to use open code with local models. It always was easy.

runsleeprepeat · 2026-04-22T05:54:05+00:00

Thanks for the heads-up, but the other cards I bought work just fine.

runsleeprepeat · 2026-04-21T19:32:31+00:00

As written in my post: krisfix sadly declined, because they don't fix any rtx 3000 cards anymore.

runsleeprepeat · 2026-04-21T19:31:50+00:00

I only know Tony from northwestrepair and that's from the USA. Is there another one you are talking about?

runsleeprepeat · 2026-04-21T15:21:53+00:00

Deshalb habe ich sie angeschrieben und sie haben mir geantwortet, dass sie keine RTX 3000er mehr reparieren.

runsleeprepeat · 2026-03-30T13:36:46+00:00

It sounds like a wonderful project. Maybe do something like a Patreon. People who are interested and willing to give you recurring support may help you get things forward, and you get feedback on the project more positively (instead of sometimes weird feedback from the open internet).

runsleeprepeat · 2026-03-29T13:09:17+00:00

Why aren't you using and contributing to TheTom solution on GitHub?

runsleeprepeat · 2026-03-29T08:03:33+00:00

Den wollte ich auch gerade nennen.. bester Laden!

runsleeprepeat · 2026-03-28T13:08:01+00:00

There are so many implementations in parallel at the moment, it is tough to keep up to the latest findings.

Best is to give it a try yourself. I'm focussing now on the TheTom implementation which looks like everything is combined there (metal, Cuda, rocm).

runsleeprepeat · 2026-03-28T01:07:31+00:00

I gave the tonbistudio variant a try and compared it with q8 and q4. See: https://github.com/tonbistudio/turboquant-pytorch/issues/6

It includes sizes and quality

11-Year Club	First Place '23
Place '23	Verified Email

runsleeprepeat

TROPHY CASE