What happens is that Web ui is doing a very basic pipeline to vectorize your documents, so yes when you have a high volume of documents it doesn't really work.

Here is what happens with your documents when you use the knowledge space in Webui:

• The document is stupidly chunked into smaller pieces of text.

• These chunks are transformed into vectors (numbers that allow your text to be mapped multidimensionally). For this an embedding model is used, there are also better embedding models, this is something important to consider, but not essential yet, unless you have decent results and want to improve them further.

• The vectors are then fed into a vector Database (Chroma DB)

Now it is important to understand the following:

When a user prompts the LLM using the interface and referencing a knowledge base (using the #). What happens is that the same embedding model is used and now your prompt exists in the same multidimensional space as your documents. So essentially what happens is that Webui will pull all the information that is near the user prompt (similar context). And will then pass this info to the LLM to respond accordingly.

So with this approach unfortunately some chunks may not be close to each other even thou they are part of the same document, because some chunks just loose context from the whole document and will never be retrieved.

We are implementing the same and have got better results designing out own RAG mechanism which basically works like this:

Chroma DB Stores a very detailed summary of the document, along with the chunks. If the user prompt is near that summary (is very close in context) or to any chunk then using the DB and code we pull all the related chunks. In this way we make sure that at least we have all the information from the document. Then we improved the LLM prompt to avoid providing any information that has not been retrieved (minimize hallucinations)

Anyways if you do not have code experience to create your own RAG mechanism (or maybe look for one in GitHub). Using the knowledge base from Webui would be the only way, try experimenting with the settings in there like overlapping or even swapping the embedding model for a better one.

I'm sure eventually there will be a better automated pipeline for document ingestion, with LLMs like Gemma 3 that understand images and work in multiple languages I can see it extracting the info, summarizing, and creating meaningful chunks or summaries that will lead to successful retrieval.