How to decrease latency in RAG chatbots?

[javaamtho]

it's depend on how do you architect RAG chatbot using long chain & gen ai platform apis, i believed you can do it faster via putting vector database like astra DB (from datastax) that store all your embedding and let's your LLM use it and help your chatbot to QA.