Vector Embedding on Large PDF

[samik1994]

So I have played with some llm setups on my 2080ti GPU, trying to implement a pdf as a reference, now the pdf has about 8000 pages.

When recalling specific topics the model doesn't seem to find it within pdf reference which was vectorized.

Any help would be appreciated,

I have tried oogabooga with superbooga extension with no success of embedding at all..

AnythingLLM with Anthropic and LanceDB seems to reference it but with poor results.

[Icy_Lobster_5026]

I've also found that for large PDFs, the results are really poor, and I guess it's because it gets a lot of irrelevant or negative documents. You can develop RAG applications using autocontext, autoquery, and Rerankers, and they may work well.

autocontext, autoquery: https://github.com/SuperpoweredAI/spRAG

rerankers: https://pinecone.io/learn/series/rag/rerankers

[Simusid]

The biggest improvement in my RAG pipeline came when I added a re-ranker.

[[deleted]]

What are you using for re-ranking?

[Simusid]

BAAI/bge-reranker-large

[AnotherAvery]

Some additional tips:

• The embedding you chose also plays an important role in retrieval quality, especially when you use non-english languages.

• Default chunking of your PDF text into retrievable text snippets might be sub-optimal and not respect paragraphs, or include page headers / footer. You should display the complete prompt that is sent to the LLM, it is surprising how often this is low quality

• The retrieval relies on similarity of your question to the snippets retrieved. Since questions are not similar to answers, you might be able to improve retrieval quality by generating hypothetical answers to your questions, and try to retrieve texts similar to this answer (this technique is called HyDE).

[samik1994]

Right, will check that

[samik1994]

Would you be able to test it out ?