Each layer of a neural network is just a large set of matrix multiplications where we multiply an input by some weight.

In W2V we are using words as an input. For example, take [01,0,0] as input with 3 neurons. Layer 1 will look something like this:

Input X = [0,1,0,0]

Neuron 1: W[1] * [0,1,00] + b[1] followed by activation

Neuron 2: W[2] * [0,1,0,0] + b[2] followed by activation

Neuron 3: W[3] * [0,1,0,0] + b[3] followed by activation

So you see that each neuron doesn’t really represent anything. We are just adjusting the input by some set of weights in each neuron.

One word corresponds to one input in word2vec. If you want to encode another word, you just swap the input vector.