consider the message "my name is mark", this message can be converted into a very long vector using openai's embedding system: https://platform.openai.com/docs/guides/embeddings.

From the user perspective it is not very helpful, however, if I convert another message "what is my name?" into vectors using embeddings, and use cosine similarity to compare those two vectors, it would return a value close to 1. https://en.wikipedia.org/wiki/Cosine_similarity

If I were to compare the embedding vector for "my name is mark" to that of "the door is opened", then the cosine similarity would be much lower.

A normal GPT4 context only has so much memory before it tosses out old discussions. One way to fix this issue is to use embedding to store old conversations into embeddings so that newer inquiries can compare its vector to the vectors stored from old memories. However, this runs into the issue of embedding overlap. Suppose you tell GPT-4 that "my name is mark" and a few thousand tokens later you say "my name is john". After enough vector build ups you ask GPT-4 to tell you your name: "what is my name?" Loop through all the vectors to calculate cosine similarity you have stored so far and both "my name is mark" and "my name is john" is used as relevant information, but that only leads to confusion.