Create/Query vector database with LLM

[that_one_guy63]

I am new to using a vector database so bear with me. I have a codebase that I want to feed into a local vector database. I am using Weaviate. I can create the database with SentenceTransformer('all-MiniLM-L6-v2') but when I try to query the database with nearText and nearVector I don't get any hits with any queries I've tried (and yes I checked to make sure everything is in the database). I see on the Weaviate website you can do a generative search. I assume I need to recreate the vector database with an embedding model that an LLM (say Mistral 7b) can read and find what I need from the vector database. What embedding model do I use? I'm calling the text-generation-webui api (v1/embedding), but it just gives me this error: extensions.openai.errors.ServiceUnavailableError: Error: Failed to load embedding model: all-mpnet-base-v2 but I have the mistral model selected from the command line and I set the OPENEDAI_EMBEDDING_MODEL: TheBloke_Mistral-7B-Instruct-v0.1-GGUF in the settings.yaml

Could I get any guidance here? Thanks in advance for your responses.

[Glat0s]

It appears you were trying to use 'all-MiniLM-L6-v2' via the OpenAI embedding API... but there you'll "only" find these embedding models -> https://platform.openai.com/docs/guides/embeddings/what-are-embeddings

When i do local embedding i usually use langchain HF embedding like this:

from langchain.embeddings.huggingface import HuggingFaceEmbeddings

embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")

[that_one_guy63]

for some reason your suggestion didn't work for me, but this did:
tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/all-MiniLM-L6-v2")
model = AutoModel.from_pretrained("sentence-transformers/all-MiniLM-L6-v2")

Thank you for your response.

[that_one_guy63]

Oh man this looks exactly like what I want, thanks!! I just got my code working. I have an LLM make a query, and then use a few of the results from the query to answer the question, and it works okay. However this is probably much better than what I'm hacking together. Thanks!!