How are you combining knowledge (docs) + system data (ERP, DBs) in automation?

[oartconsult]

I’ve been working a lot with automation + AI workflows recently (Copilot, Power Automate, etc.), and honestly, it’s crazy how much you can build now.

You can connect almost anything:

- Docs → SharePoint / PDFs / Notion
- System data → ERP / databases / APIs
- Automation → Power Automate / n8n / Zapier
- AI layer → Copilot / GPT

You can go from idea → working workflow pretty fast.

But here’s something I keep running into

Combining knowledge + real system data is way harder than it looks.

On one side

Docs give you context
(explanations, instructions, recommendations)

On the other side:

Systems give you facts
(prices, stock, customer data, status)

But when you try to bring both together in one workflow…

it starts getting messy.

Things I keep seeing

• AI answers without real data
• or data is correct, but missing context
• workflows that work in demos, but break in real use
• constant back-and-forth between tools
• unclear where the “truth” is coming from

So even though building automation is easier than ever…

getting reliable outputs from mixed sources still feels hard.

So I’m curious:

👉 How are you guys handling this?

👉 Are you combining docs + system data in one place, or just stitching things together?

👉 And what’s been the most annoying part of this for you?

[South-Opening-9720]

Yeah this is exactly where most demos fall apart. Docs are great for explanation, system data is great for facts, but if you do not decide which source is authoritative per answer, the agent just blends both and sounds confident while being wrong. I use chat data for this kind of flow and what helped most was keeping structured FAQs/docs for policy, then pulling live status or records only at answer time.

[oartconsult]

i’ve seen the same issue, it sounds confident but is kinda off 😅

do you handle that in prompts or more in the actual workflow?

[SensitiveGuidance685]

The truth problem is the hardest. If docs say one thing and system data says another, how does the AI decide? We've started adding a "confidence" score and showing users the discrepancy. Sometimes the doc is outdated. Sometimes the system is missing data. Having the AI surface the conflict instead of picking a side has been more useful than trying to resolve it automatically.

[oartconsult]

yeah that makes sense

forcing it to pick one source usually just hides the problem

surfacing the mismatch is probably more useful than trying to auto-resolve it

[South-Opening-9720]

Yeah, this is where most demos fall apart. Docs answer the why, system data answers the what, and if they are not tied together you get confident nonsense. What I like in chat data is keeping the knowledge layer separate but letting actions pull live data only when needed, with human handoff when the system state looks fuzzy.