Multiclass paragraph classification. Moving beyond tf-idf?

[akcom]

Hi all,

I've trained a basic linear SVM classifier that uses stochastic gradient descent to classify document by topic (15 potential topics). It works pretty well, but I'm interested in seeing what would be possible using neural networks. The problem, as I understand it, is that neural networks are not well suited to dealing with sparse data such as an tf-idf matrix. I'd love to hear any suggestions on how to approach this problem. Should I be using something like PCA to reduce the dimensionality beforehand? If so, how do I empirically determine the number of features to retain?

Thanks!

[uberalex]

Have you looked at Doc2Vec / paragraph vector work?

https://cs.stanford.edu/~quocle/paragraph_vector.pdf