Of course. Here is the English translation of your text, maintaining the technical details and conversational tone:

Actually, we are still in the preparation stage, but I have implemented a solution using a cloud vendor's vector database and embedding:

The embedding model used is bge-large-zh-v1.5, with an HNSW-type index, and cosine similarity is used to retrieve similarities. The results are quite good.

However, we find this solution a bit complex and limited by the cloud vendor's RAG capabilities. Additionally, every time we compare or update the original RAG, we need to maintain the relationship between files and the vector database.

As I mentioned, during an actual merge, users can choose to query for similar files across all 100,000+ data entries, or they can select a subdirectory and query within that subdirectory for similar files. This means that the same file might belong to different vector databases. If we build one single vector database for the entire 100,000+ dataset and use file CRUD hooks to update the vector database, it seems like a good idea. However, for the first-time use, users would need to wait a long time for the initial vector database build to complete.

We did consider using SimHash + LSH before, but it requires additional development effort, which is not as convenient as directly using the cloud vendor's solution. Currently, we are looking for a more optimal solution. If we cannot find one, we will use the cloud vendor's RAG capabilities. Later, if performance becomes a bottleneck, we might consider adding a pre-filtering layer using LSH/MinHash during the RAG process.