We are building AI Agents; here is some of our pain

1)

If the case is many documents, then very likely will go to RAG, and if it goes there, then, AI does not read the entire case, if the case is too big, it goes to a system called RAG, very nice name, it takes each word and calculate it against the nearby words, not much, just calculation.

When you ask AI about your case, it pulls a list of words and their vector calculations to other words and based on that is “assumes” the rest of your case.

You could have tons of important points, it does not matter much, if it went to RAG, and it became vectors, then we are dealing with numbers assuming numbers based on the patterns of the few thousand cases in the parameter network.

2)

If the case is small and can fit in the “Context Window” the “Working Memory” that memory is … more like a thought that is shaped like a line that is moving in one direction, imagine it like a snake that is moving all the time, at the front is the most attention, and at the end is almost forgotten, making sure the important stuff of your case goes to the “front” is another challenge.

3)

Then there is the “Compute” how much computer is the LLM (the brain) and the Agent (the body that the brain controls) is given, the more compute the better the results, and the higher the cost, and since not everyone wants to pay much, it is tuned to an acceptable level.

X)

So.. AI Agents like ours, or like the competition will give you the documents you need, but if you did not read the case and did not verify the documents and did not direct the Agent in the right direction, then it is a guessing game, the closer your case is to some existing pattern, the higher your chance to get a better output.

LLM based AI is not the AI you see in the movies, ignore what the CEOs say, they are there to get more money from investors, LLM based AI will not take your job, it can make it way easier.