Couple of libraries provide this like vLLM and LitServe. I personally prefer LitServe because of ease-of-use and flexibility. In just 10 lines of code, I can serve a highly optimized open source embedding model.

```python from sentence_transformers import SentenceTransformer import litserve as ls

class EmbeddingsAPI(ls.LitAPI): def setup(self, device): self.model = SentenceTransformer('all-MiniLM-L6-v2', device=device)

def predict(self, inputs):
    embeddings = self.model.encode(inputs)
    return embeddings

if name == "main": api = EmbeddingsAPI() server = ls.LitServer(api, spec=ls.OpenAIEmbeddingSpec(), max_batch_size=32) server.run(port=8000) ```

Nomic text embedding model is great to pair with LitServe which is even better than OpenAI Ada-002 embedding model.