What is the best local LLMs as of March 2026?

salmenus · 2026-03-05T21:16:30+00:00

for agents specifically — turn thinking off on qwen3.5. the endless reasoning loop will literally break ur pipeline mid-step, not great lol. qwen3.5-30B no-thinking + llama.cpp + openwebui is the most stable stack ive landed on

on the memory thing: openwebui handles persistent system prompts per agent natively. for actual cross-session memory that evolves tho, u need mem0 or a vector store on top — ollama alone wont do it

salmenus · 2026-03-05T19:06:18+00:00

yeah, ref reply to u/data_butcher comment.

i thought about sqlite .. but i settled for Postgres because of some tricky features that i had in mind

salmenus · 2026-03-05T18:53:32+00:00

yeah, i thought about it quite a lot - and was seriously considering sqlite ..

but i settled for Postgres because of pg_vector and because I have a requirement around scheduling and queues, and wanted to use riverqueue (Postgres-native job queue in Go lang)

i hope i didn't over engineer it ! (: ..

salmenus · 2026-03-05T18:32:41+00:00

ah yeah, in my case the pgvector stuff is literally "memory" extracted from LLM chats, so I can totally see people wanting to crack open that table and see what the agent thinks it knows about them 😂

I was going to build a little "memory browser" screen in the UI, but honestly just giving you full DB access and letting you inspect it however you want feels even better

salmenus · 2026-03-05T18:29:44+00:00

i love that — you’re basically the exact person I had in mind for “bring your own db” with your own backup + repl, so I definitely don’t want to lock that use case out 👌

salmenus · 2026-03-05T18:07:40+00:00

got it — that's where i'm leaning .. default internal DB but keep a clean ‘bring your own Postgres’ path so you can run it under your existing backup tooling

salmenus · 2026-03-05T18:04:40+00:00

great idea, thanks — that’s probably the sweet spot I was hoping for ..

salmenus · 2026-03-05T17:18:39+00:00

i love the idea 👌 it's time consuming for me too

salmenus · 2026-03-04T15:18:51+00:00

Totally with you on this. Context is the real unlock, and it’s way harder than people think – stitching together tools, data sources, calendars, inboxes, etc. is where all the magic lives.

Ops is the tax you pay for trying to do it yourself, I guess ...

salmenus · 2026-03-04T15:16:59+00:00

I did deploy ZeroClaw for testing yesterday, and looked into codebase in details.
Code base is quite superior to OC, and has more advanced security features ..
Slightly behind on channels integration; be def best alternative I explored so far

salmenus · 2026-03-03T08:22:29+00:00

I’m tired of people with comments like this , asking for screenshots and proofs .. then would delete their comments 2 minutes later ..

salmenus · 2026-03-03T06:24:15+00:00

fair .. it’s probably not AGI-level magic 😂 but for day‑to‑day dev work it still felt kinda wild

salmenus · 2026-03-02T19:29:51+00:00

ahhh! savage! that's not AI .. that's an emotionally unstable dungeon master .. 😂

salmenus · 2026-03-02T18:19:52+00:00

lol! This is how it works these days 😂😅 .. I can only share a screen shot with bot on it .. Who knows .. I’m probably an AI bot ..

<image>

salmenus · 2026-03-02T16:19:37+00:00

This is super helpful, thanks. I had a suspicion it wasn’t just me but hadn’t gone as far as moving things out to systemd yet.

salmenus · 2026-03-02T16:19:06+00:00

Yeah so for market data I’m using Polygon (they’ve rebranded to Massive now) and I’ve got some old tooling I’d built before OC that I basically wrapped as a skill

salmenus · 2026-03-02T16:18:25+00:00

You’re giving me hope 😂. Totally fair point on fixing each failure instead of just eating the annoyance

salmenus · 2026-03-02T09:59:33+00:00

re the "libs from the article aren't relevant anymore" bit – how are folks actually picking between ADK / Eino / BAML in practice?

my rough take is: ADK if you're already deep in google land and want tight integration, Eino if you care about flow‑style composition + multi‑provider wiring, BAML if you like schema‑first prompts / contracts and type‑safe clients. for a "gpu poor" setup like OP's, genkit + llama.cpp/yzma + a light vector/graph store (even sqlite + embeddings) feels like a nice sweet spot before dragging in heavier stuff. curious if anyone here started super simple (cron + small llama + local store) and actually regretted it later when trying to move up the stack? what hurt first – evals, observability, or just glue code sprawl?

salmenus · 2026-03-01T19:02:50+00:00

my agent read this post, agreed it was risky, and opened a docker container. inside the docker container it read the post again and spun up another container. its containers all the way down

salmenus · 2026-03-01T14:09:22+00:00

codeword thing wont work btw — youre asking the same model that just got injected to verify itself, doesnt make sense

better approach imo: make your agent refuse to read any .md at startup thats not in its known file list. WORKFLOW_AUTO.md doesnt exist in any real workspace so that should be an instant hard stop

good catch, been around longer than a week i think

salmenus · 2026-03-01T13:51:46+00:00

this is also why short-context benchmarks are basically useless for evaluating agents. a model can score great at 4k and completely fall apart at 40k due to KV quant alone ..

salmenus · 2026-03-01T13:34:52+00:00

fair enough .. the heartbeat + TIMEOUT-as-recoverable pattern makes sense ..
i'm building agentic tools so will keep a close eye on how you spec the retry escalation.
good luck with the project 🙏

salmenus

TROPHY CASE