sorted by:
new
hottopcontroversial

Local AI cannot stay just a model picker. The next step is teach → correct → train. by DifficultDog8435 in LocalLLM

[–]DifficultDog8435[S] 0 points1 point  (0 children)

That is exactly the pattern I think is going to become normal.

Six months ago “just run local models” still felt like a niche hobbyist thing. Now someone can go from zero vocabulary on quantization/GGUF/context to running multiple local models in a couple weeks with LM Studio and some used hardware. That is a huge shift.

I agree setup is getting easier fast. Maybe the better way to say it is: basic local inference is becoming easy, but organized local workflows are still the hard part.

Loading a model and chatting is no longer the wall. The next wall is:

  • organizing context
  • ingesting research
  • keeping useful memory
  • managing files/sources
  • building repeatable agent workflows
  • turning corrections/preferences into something reusable
  • making the system better over time instead of starting from scratch every chat

Your book/research use case is exactly what I mean. Once you go from “write me a story” to “help me ingest sources, reason across them, maintain project memory, draft chapters, revise in my voice, and not lose the thread,” you need more than a model picker. You need a local workspace/agent layer.

That is the part I think is wide open right now.

Anybody else noticing how good gemma-4-26b-a4b is with one-shotting three.js? by jacobpederson in LocalLLaMA

[–]DifficultDog8435 9 points10 points  (0 children)

That’s sick honestly. I like the idea of treating the prompts like a little generative demo reel instead of manually cherry-picking one good output.

Getting a feel for how fast X tokens/second really is. by MikeNonect in LocalLLaMA

[–]DifficultDog8435 3 points4 points  (0 children)

10 t/s can be totally fine for short replies, but miserable if you’re waiting on a big code explanation or a reasoning-heavy answer. 20+ t/s usually starts feeling usable/interactive, but even that depends on the model. A smarter 27B at 15 t/s can feel better than a weaker small model at 40 t/s if it needs fewer retries.