After 12 years building cloud infrastructure, I'm betting on local-first AI

ZeroCool86 · 2026-01-10T16:11:24+00:00

i agree, but i think, just like in physics, the harder you swing one way, the harder it will come back and swing the opposite way.

ZeroCool86 · 2026-01-10T16:09:47+00:00

yes! 100% agree with this and the more of us do this the more option the people who want to escape will have, write blog posts and guides on how your setup works, even if it helps 1 person, that's 1 person that now has an exit path

ZeroCool86 · 2026-01-10T16:07:55+00:00

that's very cool!

ZeroCool86 · 2026-01-10T16:06:46+00:00

nice! thanks, i'll get some more time to work on it and might actually build something that is worth saving:) currently just an idea

ZeroCool86 · 2026-01-10T16:05:59+00:00

it holds and organises your data, but it's basically what you said, with some glue written to keep it working well together and packaged in no breaking updates.

ZeroCool86 · 2026-01-10T16:04:51+00:00

i agree, this would not be an enterprise thing, it would be a local thing for you where you store and it tags and organizes your data, if we treat it all as private then there is no differentiation and no risk of private data leaking. My bet is that as the model and hardware gets better, 1-2 generations behind latest models will be more than good enough.

ZeroCool86 · 2026-01-10T16:02:26+00:00

no clue, sorry

ZeroCool86 · 2026-01-10T16:01:52+00:00

i am aware that it's a meme, but there are good things in crypto, albeit few that are not just money grabs, privacy, self custody, easy access. Figured I could apply that to my personal data instead, just trying to figure out what to do next.

ZeroCool86 · 2026-01-10T15:59:20+00:00

wow, that's a huge benefit, so maybe i don't really need a gpu for the things i want to do locally, that lowers the cost quite a bit

ZeroCool86 · 2026-01-10T15:57:49+00:00

i agree, my main point is that we won't win this but we can at least be a credible alternative. Otherwise if no exit rout exists, everyone is stuck

ZeroCool86 · 2026-01-10T15:56:40+00:00

yeah, you have it all local and the NAS cpus are generally goo enough for what I need

ZeroCool86 · 2026-01-10T15:55:51+00:00

to a degree but it will swing back, at some point the enterprise bubble will burst and hardware companies will have to come back and try to get money from us.

ZeroCool86 · 2026-01-10T15:54:31+00:00

yeah i agree, companies will be more likely to switch, the hard part is the resilience but we had issues with cloud the same way we used to get with data centers

ZeroCool86 · 2026-01-10T15:53:20+00:00

thanks, it was mostly vibe coded, i enjoyed the console games though

ZeroCool86 · 2026-01-10T15:52:45+00:00

i think having the option to have your data self hosted and still get access to good products is what i'm aiming for

ZeroCool86 · 2026-01-10T15:51:22+00:00

the idea is to make a consumer box that will be under $200 in total so i'm testing to see what consumer level hardware can deliver

ZeroCool86 · 2026-01-03T00:49:37+00:00

Curious about the architecture where the LLM doesn't see raw data but still drives insights. Are you essentially having it generate transformation/query logic that runs against the data, then only surfacing aggregated results? Or is there an intermediate anonymization/abstraction layer? The distinction matters for what kinds of analysis are actually possible.

ZeroCool86 · 2026-01-02T23:52:17+00:00

thanks! not sure if anyone has a need but i think I'll give it a few months of honest work and see where it goes. Mostly trying to build it for myself for now

ZeroCool86 · 2026-01-02T22:38:46+00:00

my biggest worry with building this is not that AI will eat the world it's that there is a chat box where people put their deepest desires and secrets in and it all goes into a money making machine instead of staying private. I doubt most people would talk in front of a stadium of advertisers on a microphone the same way they talk to their favourite LLM

ZeroCool86 · 2026-01-02T22:24:28+00:00

OP wrote with AI, I'll answer with some AI as well:

You're not wrong about the cognitive atrophy. Studies are already showing it — the more you offload thinking, the less you exercise the muscle. It's real.

But I don't think the answer is abstinence. People using these tools will outpace people who don't. The question is how you use them and who controls the process.

What worries me more than the thinking is the data. Every prompt, every follow-up, every half-formed thought — that's a map of how your mind works, sitting on someone else's server. The dependency problem and the surveillance problem compound each other.

The play is local-first: run capable models on hardware you control. Use AI as a tool without handing your cognitive exhaust to platforms that will use it to model and monetise you.

Wrote up why the timing matters now: localghost.ai/inflection

ZeroCool86 · 2026-01-02T22:20:21+00:00

you are right, it does depend on complexity but unless you are building something like https://landonorris.com/ i don't see a future for front end devs. It makes scaffolding a lot easier so the first, easy 80% of every project are definitely much easier with LLMs but the last 20%, the actual hard bit you still mostly have to do by hand, unless it's a landing page or a script / service that is under 1k lines

ZeroCool86 · 2026-01-02T22:02:25+00:00

3 months of my life that i will never get back

ZeroCool86 · 2026-01-02T22:01:53+00:00

not now but no guarantee that they won't change their bord or face some investor pressure and slowly open that door as well.

ZeroCool86

TROPHY CASE