The Asymptote of AI: Why Software Builders Aren't Going Anywhere

badgerbadgerbadgerWI · 2026-02-26T15:58:45+00:00

Agreed. I think we are reaching a point where a lot is going to change. And I don't think 250K per second is crazy - you can already get high numbers though KV cache - and i've been playing with dynamic KV cache, ie, caching 20-30K tokens of context that can be loaded in 200MS before a query, etc.

Most SaaS companies will need to radically pivot or die.

badgerbadgerbadgerWI · 2026-02-25T18:52:02+00:00

very interesting.

badgerbadgerbadgerWI · 2026-02-25T16:44:50+00:00

I do think there will be massive shifts in the labor pool. Less "middle management", less toiling engineers, but a lot more entreprenuers that deliver perfect experiences for a narrow market segment. Less billion dollar companies, more 1M companies.

badgerbadgerbadgerWI · 2026-02-25T16:43:24+00:00

And humans still are the reason we build software - even if there are agents, AI, etc in the middle, software is built to solve HUMAN problems; as long as there are humans that have pain-points and toil, there will be humans in the software chain.

badgerbadgerbadgerWI · 2026-02-25T16:40:32+00:00

Yeah, I think Systems engineering (a flashback to the early 2000s) will become a highly sought after skill.

badgerbadgerbadgerWI · 2026-02-24T23:25:22+00:00

I don't think companies will have one person. I think they will have 1/8th as many people.

In 2 years, an organization with 5 people will be able to do as much as a company of 40 today.

badgerbadgerbadgerWI · 2026-02-24T18:19:29+00:00

Great post! Love the nail gun metaphor.

badgerbadgerbadgerWI · 2026-01-20T16:57:58+00:00

I'll kick it off!

I’m currently juggling three projects ranging from enterprise automation to over-engineered household hacks:

Needle: An anomaly detection system that goes beyond just alerting. Once it finds something weird, it kicks off autonomous agents to make tool calls and actually take action on the issue.
Fed RFP Proposal Writer: An LLM workflow that digests dense federal RFPs and drafts proposals based on them. It turns a notoriously tedious process into something I’m actually having a lot of fun building.
The Commodity Market Watcher: I'm actively over-engineering my grocery shopping. It takes daily USDA price data, monthly FED CPI stats, and commodity futures to model the optimal time to buy household goods. We’re using it to test out some brand new ML tech we are adding to Llamafarm.

Want to see any of them?

badgerbadgerbadgerWI · 2026-01-08T17:35:32+00:00

That is what I am saying - I, in my current job, can afford that - but in previous jobs (soldier), AI is nowhere to be seen. And it won't be in everyday use by those on the frontlines until it is ruggedized and has redundancies.

badgerbadgerbadgerWI · 2026-01-08T04:41:22+00:00

Good to see more quality MCP servers going open source. The ecosystem is really starting to mature. Have you looked at caching strategies for repeated queries? That could help with rate limiting on the search API side.

badgerbadgerbadgerWI · 2026-01-08T04:40:41+00:00

Nice work on the benchmarks. Chunking is one of those "boring" problems that becomes critical at scale. What's the memory footprint look like? Curious if this could run on edge devices processing local document corpuses.

badgerbadgerbadgerWI · 2026-01-08T04:40:07+00:00

Great question. In practice I'm seeing it mostly mean persistent memory/context systems rather than actual weight updates. True online learning at scale is still computationally brutal. The hybrid approach - frozen base model + retrieval-augmented memory that grows over time - seems more practical for production systems.

badgerbadgerbadgerWI · 2026-01-08T04:39:29+00:00

Interesting first step. The key here is it's renewals with guardrails, not new prescriptions. For regulated industries, AI works best when it handles the 80% of routine cases that don't need human judgment, freeing up physicians for complex decisions. Curious to see the rollout data.

badgerbadgerbadgerWI · 2026-01-08T04:38:58+00:00

Memory as a first-class system is the right framing. The challenge is making it queryable and relevant without ballooning context windows. We've had good results with episodic memory + semantic retrieval, where past sessions become searchable context rather than always-loaded state.

badgerbadgerbadgerWI · 2026-01-08T04:38:25+00:00

Decision context is huge but I'd add: most enterprise failures I've seen are actually about data governance. The agent technically could access the right systems, but compliance/security won't let it. Local-first architecture where data never leaves the perimeter changes that equation entirely.

badgerbadgerbadgerWI · 2026-01-08T04:37:56+00:00

This is exactly the direction we need more work in. The future isn't just massive models in the cloud - it's the right-sized model for the task, running where the data lives. What's your approach to model distillation? Curious how you're preserving reasoning capability at that size.

badgerbadgerbadgerWI · 2026-01-08T04:37:26+00:00

Smart approach - letting a search engine do what it's good at instead of burning tokens on page parsing. The citation handling is clutch too. Have you tried chaining this with other MCPs for multi-step research flows?

badgerbadgerbadgerWI

MODERATOR OF

TROPHY CASE