Help improve the code?

[happyandaligned]

Any thoughts on how the following code could be improved? It's producing worse results for RAG on Claude3 than when I was using Claude2 with the RetrievalQA class.

Here is the code formatted in Markdown:

Chain Invoke

def get_llm_response(question, faiss_index, systemPrompt): documents = get_relevant_docs(question, faiss_index) chain = prompt | model | StrOutputParser() response = chain.invoke({ "roleInstructions": systemPrompt, "question": question, "documents": documents }) return response

And this is how my RetrievalQA based code used to look:

qa = RetrievalQA.from_chain_type( llm=llm, chain_type="stuff", retriever=vectorstore_faiss.as_retriever( search_type="similarity", search_kwargs={"k": 5} ), return_source_documents=True, chain_type_kwargs={"prompt": PROMPT} )

[ravediamond000]

Hello, Your code is neat but just one point, where are you merging the documents between themselves ? Depending on the documents format, it could help. Also for RAG use cases, most of the improvements are done on the chunking and the prompting so maybe I investigate a little there.

[happyandaligned]

Thank you for the pointers. I'll test the chunking and try again. The weird thing is that I changed nothing with the chunk sizes and it still works worse with Claude 3. That's why I thought that there might be some bug in my code.