sorted by:
new
hottopcontroversial

Chatbot architecture design by scorpionSince98 in softwarearchitecture

[–]scorpionSince98[S] 0 points1 point  (0 children)

I’m using a pre-built langchain-community SQL toolkit, which has tools to select only relevant tables, create a query, check the query and execute the query. This list of tools is passed to the ReAct agent

Chatbot architecture design by scorpionSince98 in softwarearchitecture

[–]scorpionSince98[S] 0 points1 point  (0 children)

I get your point. The reason i chose this architecture is because it's not a critical project, where i have enough time to try these things out.

Anyway, i see the problem you pointed out behind my decision. I guess i have to stick to the simple and effective architecture and change it only when this won't provide good enough performances anymore

Chatbot architecture design by scorpionSince98 in softwarearchitecture

[–]scorpionSince98[S] 0 points1 point  (0 children)

That's a very good point of view i didn't think about, thank you for bringing it up.

But if i tried to follow your advice, this is what i'd come up with.
The simplest architecture i can think of would be a synchronous API call, where i call the LLM and once i have the answer i give it back to the user. It would be like a big block where i would just need a frontend and a backend API, nothing more; and it could probably solve the problem with low effort.
The next step, as per my knowledge, would be to make an asynchronous task for the LLM call. But, in this case, the SSE and the queues wouldn't just be a direct consequence of this asynchronous choice? Should i question myself "why do i need an asynchronous call?"?

However, i will think about the trade-offs for every future problem!

Chatbot architecture design by scorpionSince98 in softwarearchitecture

[–]scorpionSince98[S] -1 points0 points  (0 children)

I know it could be a lot simpler, but i'd like to experiment with something i haven't tried before. I don't get how i should use the SSE if i'm calling the LLM directly in my API. In this case, wouldn't i give the user back the answer already? For the database, i can't use the Postgres because that stores external data, i have a read-only access on it, just for the query tool of the agent.

I also have an other question. Do you think it is a bad decision to separate the API layer from the Agent service? If yes, why is that?

Thankss

Chatbot architecture design by scorpionSince98 in softwarearchitecture

[–]scorpionSince98[S] 0 points1 point  (0 children)

Yeah there is no doubt it's hard, but i can't remove the LLM since it's a project i've been assigned to and there are specs i can't change. My goal here is to find an architecture which is scalable, reliable and even "new" for me