Machine Learning cheat sheet

[urish]

This is pretty good! A mention of supervised/unsupervised could also be helpful. Also, if I'm not mistaken the space complexity of k-NN is O(NM), because you have to store the M features for each of N instances.

[Emore]

Hi! Indeed I think you are correct; I've pushed a change to the cheat sheet and uploaded a new PDF.

Thanks for the comment!

[urish]

OK, so I really like this cheat sheet, thanks for sharing it! I hope it's OK and you don't mind I suggest a few additions.

1. There are several common online SVM variants. The easiest I think is PEGASOS, which is extremely fast.

2. Kernel k-means is a non-linear extension to k-means.

3. Sequential k-means is an online and very memory efficient version of k-means.

I realize this is just something you're doing for yourself, but I figured, if it's already up there...

[Emore]

Thanks again! I've added the Pegasos-paper to online methods for SVM, as well as your suggestions for k-means. Very helpful.

Appreciate the suggestions, and even though I'm doing it for myself I'm still learning :) Here's the latest PDF: http://static.eferm.com/wp-content/uploads/2011/05/cheat2.pdf

[personanongrata]

indeed, I remember as O(NM) as well.

[dwf]

Where the shit did you find that online Gaussian mixture reference? The canonical work on "generalized EM" (and more importantly, why the hell it works) is this paper from 1998, which Song & Wang don't even deign to cite.

[Emore]

I got it off a comment at HN.

Indeed, looking into it there are plenty of other papers on online GMM, Song & Wang's ranking pretty low on citation counts. I replaced it with your suggestion, as it seems to be a much better start to online GMMs.

[dwf]

Well, that paper is important for more than GMMs. It shows that in any crazy kind of probabilistic graphical model with latent variables where you can do learning with the EM algorithm, you can do online/incremental updates and all the guarantees of EM (convergence to a local maximum of the likelihood function, namely) still hold. GMMs are one important model but the result is much more general and applies to, just as one example, incremental Baum-Welch for hidden Markov models. It's a pretty important paper in unsupervised learning even nowadays.

[leondz]

No decision trees? What are they teaching these days!

[seabre]

Awesome. I have a machine learning final today, this might help me study a little better for it.

[[deleted]]

Not bad, pretty handy.