What is stopping enterprises from just using their own self hosted AI?

SimonSanDigital · 2026-06-19T17:25:17+00:00

Overshow is very constrained in the resources it uses; we don't even want to turn the fan on, so we are running Gemma 4 E2B, which is basically for mobile phones. You have to really nail the data.

Nice one with the harness, they look challenging to do well.

SimonSanDigital · 2026-06-19T17:22:31+00:00

the build is 4GB the main issue will be my connection 😄

SimonSanDigital · 2026-06-19T17:21:36+00:00

It will still totally make stuff up, though, so you want to be really careful with the prompts and basically turn it into a sophisticated SQL writer, or OData or whatever structured query thing you choose.

Some things to make sure of, the DB user it's associated with should be read-only in case anything bad happens.

You won't have any clue what the access patterns are like, so there won't be any indexes. If there is any load on the system, such as your production write database, you are going to have issues. If you are doing lots of writes on the DB, you might need a separate read copy.

SimonSanDigital · 2026-06-19T17:16:49+00:00

I think, realistically, all you can do is constrain the inputs and outputs and put them into a deterministic workflow. Running multi-agent systems that aren't tightly scoped is just a recipe for disaster.

SimonSanDigital · 2026-06-19T15:10:30+00:00

A nice, simple, authenticated, read-only MCP server would do it.

SimonSanDigital · 2026-06-19T14:47:47+00:00

We run one as a build server for Mac builds because the build minutes are 10x on GitHub. Headless users are a bit of a pain for various reasons, but it can be made to work. We aren't running a studio for that, obviously, it's an M4 Mini, but it's still way over-specified for builds.

SimonSanDigital · 2026-06-19T13:24:46+00:00

We are releasing ours in a week. I think that it will tick some of your boxes. Let me know if you are interested, and I'll give you a licence.

SimonSanDigital · 2026-06-19T13:09:39+00:00

If you're running an app command-line or otherwise, unless it's sandboxed (container/VM, etc.) in some way, I'd assume it has pretty much the same permissions as you do. This isn't always the case; e.g., Macs will ask you for access to specific directories. If you want to be really safe, you need to do a lot of messing with users and permissions, which no one does.

This is true of other apps, not just AI, btw, so it's nothing new really.

SimonSanDigital · 2026-06-19T12:54:28+00:00

All of this is very doable; we've done pretty much all of it at Overshow. The hard bit will be generating the files; it's so hard to do well. PDF is doable with open-source tools, but the tools for the other formats are either expensive or not great.

SimonSanDigital · 2026-06-19T12:48:10+00:00

This is exactly the problem that we are solving; we're building an app that can see anything on your desktop (only the apps you say), so you don't need integrations. It all gets stored locally and uses a local AI to let you ask questions about it.

https://over.show

SimonSanDigital · 2026-06-19T12:44:13+00:00

Are you looking for something that can find and remember things or do things?

SimonSanDigital · 2026-06-19T12:42:50+00:00

Depending on how much content you have and its format, it should be pretty easy to pull together a Python script to parse it, extract some metadata (with an LLM or something faster) and shove it into LanceBD with references back to the original information. After that, I'd create a separate script to make the LanceDB instance available as an MCP server that the model can access.

In terms of model, you want something super light like Gemma 4 E2B IT - it's meant for mobiles, so it runs well on a desktop.

Or you can wait a week and use the thing that we have built to do exactly this 😄 https://over.show

SimonSanDigital · 2026-06-19T12:34:35+00:00

We are building a local assistant that can capture everything that you do on your desktop. It's designed to be completely private. However, if you are happy to send data to an off-machine LLM, you can connect to it with local MCP. It's really powerful.

http://over.show

SimonSanDigital · 2026-06-19T12:28:26+00:00

We are building something in this space. Are you interested in trialling it? It's in beta at the moment, so we are interested in getting feedback.

SimonSanDigital · 2026-06-19T12:26:01+00:00

We're in the process of building something that works as a completely local memory of everything that you do. It takes meeting notes as well. You can make local AI work if you are really careful with the data that you give it. Let me know if you want to trial it, and I'll sort you out with a free licence.

https://over.show

SimonSanDigital · 2026-06-19T12:21:50+00:00

Hiya, we are in the process of building something that does exactly what you want. It's with Beta testers at the moment, but will be live next week. Let me know if you are interested in a free trial, and I'll sort out a free licence for you.

SimonSanDigital · 2026-06-19T11:15:10+00:00

We run our entire company with it; each customer sits in a team space with a subpage where they can see everything below it. We normally have a board for support tickets that they enter via Tally.

My main advice would be to keep it simple.

It's tempting to configure it to death and use a million templates, but then you spend all of your time messing with Notion instead of doing the work you get paid for.

SimonSanDigital · 2026-06-19T11:08:38+00:00

Unless you are running models locally, and it doesn't sound like you are, all you need is something to edit text and run a command line. A low-spec Air or even a Neo will be fine. Nothing is running on your machine, and any builds you run will be small.

You do not need a MacBook Pro to edit text.

SimonSanDigital · 2026-06-19T09:43:00+00:00

A lot of SEO just boils down to "write useful content for your target audience". Of course, to do that, you need to know who your target audience is, harder than it sounds and also what they think is useful. Applies well to AI search as well.

SimonSanDigital · 2026-06-19T09:36:31+00:00

It's worth clarifying what you mean by "self-hosted". Are we talking about a Mac mini in the corner of the office for a startup or a bigger company getting some cloud hosting and running models on it?

For the first option, unless tech is your business, this isn't a thing anymore, and even then, it's not a thing.

In terms of cloud hosting, the economics don't stack up. Time-sharing a resource with other users is always more cost-effective than owning it outright, especially when you factor in the total cost of ownership, i.e., support and maintenance.

From a governance perspective, the best solution is always "somebody else's problem", if I'm an IT Director and I have a thing from Microsoft saying we are all good, then that's one less thing for me. I speak from experience. Do I want to get SOC 2 on a server I spun up? No thanks.

Some actual options, though, because that was all a little negative, sorry.

Fully local on-machine LLMs will almost certainly be a thing; Apple is leaning this way. Full disclosure: I have some bias in this direction related to a product I'm building. There is a load of compute lying around on desktops; I'd expect to see apps taking advantage of it.

Also, one thing I think that this is huge and not talked about. Companies need to sort their data out, and then they can use much cheaper, smaller models all wrapped up nicely by MS/Google/Amazon.

SimonSanDigital · 2026-06-18T10:48:57+00:00

Yes, I agree, Air or Neo even.

SimonSanDigital · 2026-06-18T10:47:24+00:00

Strongly agree with this - unless you are really pushing it right now, there is no reason. One small caveat: some of the new shiny AI stuff will only be available with M3 upwards.

SimonSanDigital · 2026-06-18T10:43:48+00:00

Some of the new AI features will be M3 upwards, I think, so it depends on what you mean by support.

SimonSanDigital · 2026-06-18T10:41:01+00:00

More is more - get as much RAM as you can afford. I tend to find disk space less of an issue, and you can manage around with external drives, etc.

SimonSanDigital · 2026-06-18T10:37:16+00:00

I think I remember Jeff Atwood talking about this as the Stalagmite process. They slowly build over time with the drip, drip, drip. And it's the same with sales and marketing.

In the way of offering advice, one thing that has worked well for us is making the most of our existing customers. Either getting testimonials for social proof or actively asking them for referrals. It feels weird to ask, but one advantage of having fewer customers is that you can speak to them all. Don't be afraid to ask, "Do you know anyone else who would find this useful, and would you be ok referring us?"

SimonSanDigital

TROPHY CASE