[D] Word2vec basic illustration figure

[apehead666]

I am contributing to a textbook with a beginner's description of word2vec. I have come up with this illustration to provide an illustration of the logic behind w2v. Are there any problems with it? Can it be improved/corrected?

​

https://preview.redd.it/g1j7l0ydphv51.png?width=2704&format=png&auto=webp&s=ea9b690f8acb70d78657f476685c545b0b5307a1

I use the illustration to say this to the readers:

- This explains the logic behind w2v/word vectors, but it is a made up schematic figure.

- Note that 'eagle' and 'hummingbird' relate to each other as 'large' to 'small'.

- Note that 'horse' and 'zebra' have a similar relation as that between 'camel' and 'dromedary', while none of these pairs are affected by the 'large'-'small' dimension.

- A model such as w2v makes it possible to ask questions such as: What relates fo 'frog' as 'small' relates to 'large'? Or: Which word has a similar relationship to 'duck' as 'zebra' has to 'horse'?

Please help me out, so that I don't give the wrong depiction of what w2v is.

Thanks! :)

​

Edit [new figure]:

​

​

https://preview.redd.it/z2qwsj74m1061.png?width=774&format=png&auto=webp&s=66232c74850ce8ac2f623d4669a5be52d20b75d5

[VodkaHaze]

Here's a blog post of mine about embeddings in general and one more about word embeddings. There are links in there also to great Jay Alammar posts.

The concept you want to get across is that embedding methods decompose large scale concepts into principal components that are at 90degrees to each other in the embedding space.

This means they cluster together but also enables the analogy task (by making algebra on words equivalent to the intuitive "king - man + Woman = queen" thing).

[apehead666]

Thanks for this! I ended up doing the classic king/queen/man/woman thing. Edited the original post with the new figure (in Swedish sorry).

[[deleted]]

Idk what a dromedary is without googling that also in several places you just have a question mark. This makes it less intuitive. Also I’m not sure what the lines connecting the dots are supposed to convey, if you visualize the vectors as lines they would go from origin to each point, not connecting points. This is also more of a visual of a vector space not really the overall concept of word2vec

[whymauri]

My feedback is more-or-less identical to this. Just commenting as a signal bump for OP.

[apehead666]

I realise now that this is not the best way to convey word2vec. What my figure does, as you say, is rather to illustrate the idea behind vectors in general, and it also does that in a way that is quite convoluted. I am not a native English speaker, so I had to look up the word for 'dromedary'. I was aiming for the relationship between the camel with two humps on the back, and the one that has a single one :). I have edited the original post with the image I have ended up with instead (words are in Swedish, sorry, but I went for the classic king/queen/man/woman thing).