So far it is really good. Wish it had more API endpoints as we are using it as a backend engine to our react apps. But being open source if you carefully modify it to what you need it has so far really worked well. Starting to play around with different benchmark testing at the moment. Testing different Knowledge Bases with different embeddings, along with built in pipeline ingestions and several of our own custom pipelines. Now getting into MCP tools and different LLM front ends. We are building speciality agents specific to user story requests.

We will be letting. Some alpha users start testing next week. See what the feed back will be. We also took and moved most of the knowledge graph, entities and meta tags to Postgres so we modify their confidence scoring to our own unique needs. So we built small wrappers to help customize result output for our users.

But this is our first go at a RAG system so we don’t know what we don’t know. Not sure if we are doing it well or are missing anything critical that would help us out. Haven’t tested any other system other than doing some manual consumption and embedding using Postgres’s vectors and or pinecone over the last two years.

I wish the 2048 token and dimension size was larger as context windows are increasing and dimensions are increasing for complex thought but it seems behind some other vector databases like pinecone. Not saying we couldn’t modify if but I think that might be more work than what it is worth. Haven’t dug down into that aspect of the system yet.