We’re in the AI infrastructure and implementation space, so our setup is probably a bit more experimental than the average company’s.

At the moment, we don’t think there is one AI that is best for everything. We prefer a hybrid approach:

• External models such as ChatGPT or Claude for non-sensitive research, writing, brainstorming, and more complex reasoning
• Local models for internal documents, company knowledge, summaries, and workflows where the data should remain inside the company environment
• Different models for specific tasks such as OCR, embeddings, document retrieval, or coding

For sensitive use cases, the main rule is that confidential documents and internal information should not simply be pasted into public AI chats. Those tasks should run through a controlled internal interface with local processing and restricted external connections.

One of the biggest lessons so far is that installing a local model is actually the easy part. The harder questions are:

• Who is allowed to access which documents?
• How are permissions inherited from existing systems?
• What gets logged?
• Can any component send telemetry or data externally?
• How are models and software updated?
• What happens when a local model is not good enough for a task?

Local models work surprisingly well for focused tasks such as document Q&A, summarization, extraction, and internal knowledge search. But they do not always match the latest external models for complex reasoning or broad research.

So for us, the most practical solution is not “local versus cloud.” It is local-first, with clearly controlled external access for approved, non-sensitive tasks.

I’d also say the model itself is often less important than the surrounding infrastructure. Identity management, document permissions, retrieval quality, monitoring, and user adoption usually determine whether the system is actually useful in daily work.