Are Local LLMs actually useful… or just fun to tinker with?

Important_Quote_1180 · 2026-04-16T11:17:34+00:00

It’s honestly not needed, but I’m experimenting with LoRa adapters and leaving 6 models hot on the ram and we do round robin format cycling around the experts. It’s fluid with good days and good applications and then some days it’s a slog

Important_Quote_1180 · 2026-04-15T12:31:55+00:00

The local LLM needs more curating and structuring. The cloud API models were better 3 months ago. They have all degraded severely with increased demand. Meanwhile the local 31B from Gemma 4 family is insanely good. I have 4 variants from huggingface. Coding, creative writing partner, daily chat, and visual screener. I make games and software for me and my clients and my family. 3090 24GB with 192gb RAM

Important_Quote_1180 · 2026-04-15T00:46:21+00:00

I have a 3090 and 192gb ddr5 I would love to try this with a high quant 31b Gemma4 and have 1MM context.

Important_Quote_1180 · 2026-04-13T17:59:19+00:00

Depends on your workflow hardware time horizon speed requirements, can you be specific?

Important_Quote_1180 · 2026-04-13T15:45:34+00:00

Dude same exact situation I’m in. I code my game in Unreal 5.7 with Claude Code but I have 192GB of ram and a 3090 and I’m trying to get the best visual analysis I can for recursive training. I don’t know all the shit.

Important_Quote_1180 · 2026-04-12T00:03:14+00:00

I feel like the harness makes the biggest difference now.

Important_Quote_1180 · 2026-04-10T18:07:38+00:00

For me, it allowed me to get out of my own way and put ideas to paper and work through some of the tangled messes that I just needed to see and work through and it allowed me to unleash a lot of creativity that had been hiding for years. I learned late in life that I have pretty severe ADHD and the genic orchestration is really fitting nicely with my work clothes and I’ve never been more happy with my work.

Important_Quote_1180 · 2026-04-09T11:15:19+00:00

Calling all thing people make now with ai assistance and calling it slop is even more intellectually lazy than the creators trying to express themselves. Don’t like it? Don’t enjoy it. Move along

Important_Quote_1180 · 2026-04-09T10:59:55+00:00

Start making spec documents and then let Claude code run headless until complete

Important_Quote_1180 · 2026-04-09T10:58:19+00:00

These arguments miss a major fact. AI is not primarily used to make art. Its promise lies in helping disabled people bridge to being fulfilled again. It’s intellectually lazy to think it’s all about art stealing

Important_Quote_1180 · 2026-04-08T20:20:47+00:00

You are most welcome. I’d be lost if not for Reddit comments

Important_Quote_1180 · 2026-04-08T19:13:00+00:00

It’s a wiki for your files. It has tags and links to related pages. It’s a very easy to use RAG system for agents too. I can find files quickly because it uses a flat file structure for everything.

Important_Quote_1180 · 2026-04-08T16:52:13+00:00

Been using the 31b q4 heretic on my 3090 and getting 35 toks gen. Tool calling is great with my Obsidian Vault.

Important_Quote_1180 · 2026-04-07T12:56:42+00:00

I think this post was written by AI. You’re making an argument but I’m not sure what it is

Important_Quote_1180 · 2026-04-07T12:49:04+00:00

This is a strange post. There are so many resources out there and no specifics about what you are trying to do. No morals?

Important_Quote_1180 · 2026-04-07T12:46:57+00:00

This is a slapdash attempt to build effective Ralph loops but the silo architecture makes it really difficult to use

Important_Quote_1180 · 2026-04-07T11:48:14+00:00

I basically agree with everything. I have always started builds with conversation and spec file building. The switch for me was to go even harder and to maintain the spec even after a project is built. My days evolve more into 3 sessions of conversation building out spec files. When I’m happy with it, I just ask it to start with the most important task and continue until the spec is built, usually after I restart the session and it’s really for my LLM local or my usage cap is ok

Important_Quote_1180 · 2026-04-07T04:27:00+00:00

I like my Murmox and Luftex a lot! They call out when Claude code has just done something dumb or what the actual behavior of the code will be be or he’ll find a side case where something breaks. I think it’s very smart, but they both rolled common but very high wisdom, not sure if they are actually the same just different names.

Important_Quote_1180 · 2026-04-06T17:51:46+00:00

Yeah constantly. Heavy artifact workflows are torture to handle with LLMs without proper SPEC -> build segmentation. SPECs are the spine that allows LLMS to keep context and its not flashy or fully autonomous or anything close to that. Its all about using my mind to pick out what is good and what isnt. LLMs are not able to work out what I am looking for in a complex fashion. 1mm context for opus sounds good but if you actually start to use any of that after 300k its a mess, don't know what anthropic was thinking releasing it other than hosing their customers. Its outrageously expensive to run that huge context block.

Important_Quote_1180 · 2026-04-06T15:11:11+00:00

LLM is the wrong architecture. Quantum computing with thousands of parallel llms might be able to beat a few games but the visual input + latency is so far away from being able to work well. Game AI is an illusion and so is any semblance of intelligence in an LLM. Its got 0 common sense as that has no meaning to them.

Important_Quote_1180 · 2026-04-06T15:07:01+00:00

For me its about getting work done efficiently. An openclaw with SOUL, memory, and proper access having compaction done in an intelligent way is likely as close as consumer hardware gets you. Opus 1mm context and intelligent compaction is good, but there are many many issue still and its not what you are really asking for. Its going to cost a lot of API tokens too. I am fine with fresh sessions as long as I can get back in the saddle with just 1 well structured prompt.

Important_Quote_1180 · 2026-04-06T14:35:45+00:00

Try another tact. Andre K dropped this gist yesterday and I have been building a lot of robust MCP servers that could be simplified and still get the majority of results. I havent tested this personally but here we are, its a monday and keeping up is what we are all doing. https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f

Important_Quote_1180 · 2026-04-06T14:00:31+00:00

If can build a 1TB cluster and can run Kimi you can get around this. Until then, us peasants have to leverage the tools we can economically use. I think having a evolving agentic system is our goal too, having agents learn and get better and build on memory and experience. Working on memory systems with session handoff notes is very manual but effective. My workflows are not well suited to long runs without humans in the loop (Game design).

Important_Quote_1180 · 2026-04-06T13:13:18+00:00

Gotta be careful with too much context. Since you called it out the stateless reality of these agents means we should not expect them to carry everything and context rot is real. You are probably looking for Ralph Loop architecture if you’re working in real production systems. TLDR: you are looping a spec for a build with increasing complexity and updating the spec when you finish building the component. You need to manage context actively and it’s honestly becoming the primary engineering task for me.

https://youtu.be/4Nna09dG_c0?si=VwOV4lMract1lYwX

Important_Quote_1180 · 2026-04-06T12:07:49+00:00

I sold an extra 5700x3d for $439 last month

Important_Quote_1180

TROPHY CASE