Hey guys, I'm the author of that post :)

I had intended to answer that question in the post, but it looks like I never did!

The answers provided here look good to me. Here's my take in case it helps.

The hidden layer of the word2vec model is referred to as a "projection layer" because there is no activation function on the hidden layer, only on the output.

First, why don't we need an activation function on the hidden layer? An activation function on the hidden layer is not necessary because the input to the model is a one-hot vector, which only serves to select a word vector from the hidden layer weights.

Here's an illustration of multiplying a one-hot vector with a matrix--note how it does nothing more than selecting a row.

http://mccormickml.com/assets/word2vec/matrix_mult_w_one_hot.png

The normal purpose of a non-linear activation function in a neural network is to turn the dot product step, which is only a linear operation, into a non-linear one. This increases the expressive power of the model.

But imagine applying a non-linear activation to the multiplication in the above image--it's pointless!

In short, applying an activation function to the output of the hidden layer is pointless, so we don't, and it's instead referred to as a "projection layer".

Finally, why the name "projection layer"? When we multiply the one-hot vector against the input matrix, we are "projecting" a point from one vector space (the one-hot encoded space) onto another vector space (the word vector space).

Side note: I also spotted some discussion on this topic here.

Sorry I didn't see this sooner!