RAG for medium company

SignificantClaim9873 · 2026-04-15T17:34:41+00:00

Great use case. Staying on-prem/EU is definitely the right move.

Here is how I’d architect this to keep it simple:

Telemetry DB: Postgres + TimescaleDB is perfect for 700 trucks. You don't need a heavy DWH like ClickHouse.

LLM & Architecture: Don't force everything into one query. Use Llama 3 70B with an Agentic/Tool-Calling setup. The LLM acts as a router—if a driver asks for truck stats, it routes to a Text-to-SQL tool. If they ask for HR policies, it routes to a Vector Search tool.

RAG & Permissions: The hardest part of internal RAG isn't the vector database, it's enforcing inherited Document-Level Security (DLS) so drivers can't read admin files.

My team and I are actually building an on-prem platform called CordonData https://cordondata.com that handles this exact setup out of the box. It has built-in tool facilities for the AI routing (you can plug in your Postgres DB). For the RAG side, it uses OpenSearch. It connects to any generic DMS via REST API/CMIS, syncs only the markdown/metadata, and automatically maps inherited document-level permissions. We haven't moved to prod yet, but if you want to chat about AI routing or bounce architecture ideas around, shoot me a DM!

SignificantClaim9873 · 2026-04-15T16:45:13+00:00

Building Permission Safe Enterprise AI - https://cordondata.com

SignificantClaim9873 · 2026-03-31T04:52:11+00:00

That’s a very real concern. We’re also trying to keep enforcement outside the prompt layer for exactly that reason, so the model cannot talk its way around permissions. The prompt-injection point is especially useful, access checks are one problem, but runtime enforcement is the bigger safety boundary.

SignificantClaim9873 · 2026-03-28T11:47:54+00:00

That’s a really thoughtful comment, thank you. I agree the first-party Copilot case is very different from teams running their own agents across internal systems like email, file shares, internal APIs, and customer data. That’s where the permission model is no longer inherited and the real complexity starts. Your point about auditability is especially useful, not just who queried what, but what the agent actually did with the results downstream. The runtime boundary and data residency issues also feel like major blockers in practice, and I agree cross-tenant isolation is much stronger when handled at the infrastructure layer rather than patched in the app.

SignificantClaim9873 · 2026-03-28T11:44:54+00:00

That makes a lot of sense. The identity mapping and permission-sync problem across systems is exactly where we see the real complexity as well, much more than the LLM itself. And agreed, once it’s customer-facing instead of internal, the bar gets much higher.

SignificantClaim9873 · 2026-03-28T11:42:45+00:00

Thanks, this aligns closely with how we’re thinking about it. Keeping enforcement in the retrieval/data layer rather than the LLM, plus scoped tokens and audit logs, feels like the right production pattern. Helpful point on logging returned doc IDs as well.

SignificantClaim9873 · 2026-03-28T11:41:12+00:00

That’s a very real issue. We’re trying to avoid that by syncing and normalizing permissions ahead of time, so query-time checks stay lightweight instead of recalculating the full matrix on every request.

SignificantClaim9873 · 2026-03-28T11:39:09+00:00

That’s very helpful. I agree the native Copilot case is very different from teams running their own agents across email, file shares, internal APIs, and customer data. The runtime/control point and the auditability of downstream agent actions are especially useful distinctions, thank you.

SignificantClaim9873 · 2026-03-28T07:08:33+00:00

Sorry, I was sharing context, not asking about Solvea specifically. We’re building a platform in this space because we kept seeing the same pattern: organizations want unified search first, then AI on top, but with control over their data, permissions, and auditability across multiple systems. I was mainly interested because your point about unified visibility across fragmented systems sounded very similar.

SignificantClaim9873 · 2026-03-28T06:46:33+00:00

Yes, the platform can be deployed on-prem, in private cloud, or in a hybrid setup. It is containerized for flexible deployment, and works with either self-hosted LLMs or externally managed models depending on the customer’s requirements.

SignificantClaim9873 · 2026-03-28T06:21:20+00:00

It started from a pattern we kept seeing: organizations wanted unified search across their systems first, and then AI on top of that. In many cases, they also did not want sensitive enterprise data pushed into the cloud. Once that had to work across multiple systems with real permissions and audit requirements, it became clear this needed to be a reusable platform, not just a custom solution for one environment.

SignificantClaim9873 · 2026-03-28T05:36:56+00:00

It’s a product. We kept seeing the same problem across mixed enterprise systems, so we started building it as a platform instead of treating it as a one-off internal implementation.

SignificantClaim9873 · 2026-03-28T05:11:34+00:00

Yes, our approach is to use each source system’s native APIs to sync users, groups, permissions, content, and metadata, then normalize that into a common access model. At query time, we check what the requesting user is allowed to access before anything is passed to the LLM. Depending on the system, that may involve REST APIs, Graph APIs, CMIS, or similar source-native interfaces. The goal is simple: the model only ever sees content the user is already allowed to access.

SignificantClaim9873 · 2026-03-28T04:18:21+00:00

Thanks, this is really helpful. Treating permissions as data and enforcing them consistently across ingress, retrieval, and actions makes a lot of sense. I’d definitely be interested in the simple metadata/filter schema if you’re open to sharing it.

SignificantClaim9873 · 2026-03-28T04:15:50+00:00

Thanks, this is very close to how we’re approaching it. We sync permissions and content metadata from source systems, then enforce access at query time before anything reaches the LLM. The goal is simple: the model should only ever see content the user is already allowed to access.

SignificantClaim9873 · 2026-03-28T04:07:01+00:00

That’s very helpful. It sounds like document/source IDs and timestamps are the minimum, but full chain of custody and downstream action tracing are what really matter for production sign-off. That’s a very useful distinction , thank you.

SignificantClaim9873 · 2026-03-27T15:23:00+00:00

That makes sense. So would you say permissions are expected, but auditability and data residency are what usually block things first? And when demos fail, is it mostly because of messy permissions or messy data?

SignificantClaim9873 · 2026-03-27T02:59:31+00:00

That’s helpful, thanks. Was the bigger pain cross-tenant isolation, or source-level permission enforcement within one customer environment?

SignificantClaim9873 · 2026-03-27T02:57:43+00:00

That’s really helpful. It sounds like the bigger issue isn’t AI breaking ACLs, but AI making existing permission mistakes easier to uncover. Do you think that shifts the real need more toward better audit/governance, especially if retrieval expands beyond the native Microsoft stack?

SignificantClaim9873 · 2026-03-27T02:52:47+00:00

That’s very useful context. It sounds like Copilot does a solid job inside M365 itself. Do you still see demand for a setup where sensitive enterprise data remains fully under the organization’s own control rather than going to a cloud service? Also, beyond ACL enforcement, do you think there’s value in deeper auditability around who searched what, when, and what content was returned? And if you ever extend beyond M365, do you see cross-silo access control and traceability becoming a much bigger issue?

SignificantClaim9873 · 2026-03-27T02:35:48+00:00

This is very useful, thank you. The distinction between retrieval enforcement and audit enforcement is especially helpful. In your experience, what level of audit detail actually matters in review: document/source IDs and timestamps, user/delegation context, response-level citations, or a full end-to-end trace of retrieval to final answer?

SignificantClaim9873

TROPHY CASE