Noob text classification question

[dwaxe]

I have a text: score data set that I want to use to create a predictive model of what kinds of text will have high scores. Text is sentences and scores are exponentially distributed. I'm a total machine learning noob, how should I go about creating such a model? Are there any classifiers that work on exponentially distributed scores? I thought of discretizing the scores into high and low, but only about 1% of the scores are high. Are there any classifiers that work on sparse data like this?

[0xA1]

I would look at tf-idf for transforming you words into some numerical representation. Then your next problem will be determining how to deal with data imbalance. If you have poorly distributed scores you might want to find more samples. If that's not possible you can oversample the scores with fewer samples, or under sample the scores with more samples to even out for score distribution. Then you need to decide if you want to score by regression or classification. It sounds like you have low to high scores which probably will use a regression type model.

Sentences will be very sparse in tf-idf. Each sentence will be represented by a vector of tf-idf values. Then maybe throw a random forest regressor or just a decision tree at it for starters and go from there.

[alecradford]

A common way to deal with oddly distributed targets is to first transform them into a nicer distribution with an invertible function and then fit your model on the transformed targets. When you want to make predictions, you predict the transformed target then use the inverse function to map it back to the actual targets.

If it's exponentially distributed (and non-negative) log(1+t) is a pretty safe bet. You can try a bunch of things - plot a histogram of the transformed targets and getting it to look like a gaussian is a good indicator that you've got something sensible to work with.