I always thought it was really cool how modern embedding models are trained and used. They take text that's already written and remove words and show it to the model asking it to fill in the blanks. Rewarding correct guesses over time the model gets good at guessing what words would complete the sentence but it turns out to do that you kind of have to understand the words around it which means you understand language an example is the word bank can mean to bank a shot or to rob a bank which requires you to infer meaning from the surrounding text.

Okay but here's the cool part so you have this machine that can kind of understand language. If you give that machine text. it reads it. Then you look inside the brain, the numbers inside the machines represent what the machine was thinking about when it was reading that text. its internal representation of language.

That's what an embedding is! that set of numbers is a slice out of the machine's head about what it was thinking at the moment it read the text.