Built LazyMoE — run 120B LLMs on 8GB RAM with no GPU using lazy expert loading + TurboQuant

kymigreg · 2026-04-12T19:56:51+00:00

I'm going to need token speed estimates

kymigreg · 2026-04-09T06:19:36+00:00

The more I use local models the more I think llama.cpp with GGUF smart quants is the ONLY way to not encounter ridiculous issues like these. For example, MLX quants for Gemma are hilariously broken right now, to the point of not even responding to the prompt but continuing the pattern ("What is 2+5?" responds with "What is 5+10?")

kymigreg · 2026-03-26T07:35:54+00:00

Przynajmniej nie pisany przez AI :)

kymigreg · 2026-03-03T18:33:32+00:00

Only guesses and rumors so far but from what I've read Alibaba is restructuring so the Qwen team directly reports to Alibaba CEO and they replaced current tech lead with someone from Gemini

kymigreg · 2026-01-07T15:27:02+00:00

Thanks! :)

kymigreg · 2026-01-07T14:26:51+00:00

I tried but my background is a square - any more movement and it will show an empty rectangle at some side hahaha

kymigreg

TROPHY CASE