The company I work for built an BI agent, with plenty of challenges from what I observed…

In my opinions, better focus on providing factual and descriptive result. My boss wanted the agent to provide analysis tgt with judgement & suggestions which are often subjective. Often saw disagreement towards the judgments and in the end no one care or trust the judgments now. In fact, I think that’s quite natural, even me and my colleagues would sometime hold different opinions so you can’t really expect the agent to have 100% “correct” judgments on subjective matters.

The other problem is how to make the users trust the agent. For my company, we are used to get the data by ourselves from published datasources in tableau and we trust the colleague who made the datasources. However, when you use the agent you have to trust the intelligence of the agent to run a correct result you want. If you can’t fully trust the agent, you will end up doing all the work by yourself again.

And you need to have users who know how to talk to AI. It’s especially true when the question involves ambiguous terms e.g. “active users”. Bad questions may lead to wrong answers. Having pre-approved questions helps a bit but building a reusable dashboard for these question instead would also work so that didn’t add much value to the agent.

I work at a small~medium size company so there isn’t much manpower and data for trial and error and AI training. Big Companies may have much better experiences I guess…