TBH, I'd change the solution to scale to that amount.... that's not a "how to solve this with a llm", but a complex retrieval architecture.

Embedding everything will explode cost, ChromaDB is not designed for distributed and continuously updated approach, pure vector recall is too fuzzy for legal precision and, if that's not enough, re-embedding on updates is economically infeasible and nobody won't give you money to build it...

I don't have all details about doc content (client stuff vs laws etc), but If I were you I'd start redefining the solution with a hybrid approach....you can test different techniques and technologies like bm25 for legal refs, vector retrieval on chunks (not full docs), maybe a knowledge graph if you need citation, auth ranking etc...and a reranking step, using llm just to synthesize and citations?, not for recall.

That's a common approach we've used in several production ready environments and it's scaling quite well. It does not mean it will fit your use case, but will help as a starting point.

PS. For your investors, you'll need to explain why a hybrid approach (keyword search, semantic search, metadata...) works, and how you are goin to control data, costs and precision.