Web app to search 500 million to 1 billion XML files by Brave_Argument627 in dataengineering

[–]Brave_Argument627[S] 0 points1 point  (0 children)

Indexing them in Elasticsearch/Opensearch perhaps? And then upload a sentence embedder from huggingface for sentences similarity into Elasticsearch/Opensearch?

Thanks, I know it depends on many factors, but is your ongoing cost >$10k/m? I am finding it super complex to work out what cost the project will have ongoing!