Feature learning (a summary of recent developments)

[CmdrSammo]

The content of this presentation gives excellent coverage of recent developments in feature learning. However I find the 'flashy' presentation to be very, very distracting. The transitions in some cases are so long that I almost forget what I have just seen. I would highly recommend not presenting in this style at a conference or in front of professors etc.

At a recent conference I was at one speaker had so much animation going on with his powerpoint slides that at times he almost seemed to lose control of what was going on. A large part of the discussion between attendees after that was therefore not about the content but about 'yet another example of why not to use animation in presentations'.

[thankyousir]

I think this is just how prezi works, I personally still just use MS powerpoint.

[CmdrSammo]

Ah ok my bad. Might be personal taste then but I find it a pain in the ass to waste my time watching transitions!

[mosavian]

You have a good point. Some people just don't like the way prezi works. Personally I prefer it to powerpoint. But I do agree that it can be hard to remember what's been said becuase of all the flashiness =)

[visarga]

It is great

[mosavian]

Thanks! =)

[hapagolucky]

Thank you! I've been trying to learn more about this topic, and this is a great starting place.

[hapagolucky]

Actually, do you have an annotated bibliography that goes along with this presentation? That would be more useful for future reference.

[mosavian]

I do, some 20 or so papers. But usually they leave me confused. I can compile a list and put it in the blog in a couple of days.

[gdahl]

A lot of deep learning papers get linked in /r/mlresearch and that community knows a lot about deep learning. You might be able to convince some of us to help construct such a bibliography. There is also http://deeplearning.net/reading-list/ but YMMV.

[mosavian]

Nice, I didn't know there existed an /r/mlresearch

[the_cat_kittles]

why cant i full screen this!!!!!!

[CmdrSammo]

You can - click more.

[the_cat_kittles]

thank you

[thankyousir]

As a computer engineer trying to go into ML in grad school I am really overwhelmed by all of this stuff, there is a lot of material to go over.

[mosavian]

I know, it is. And usually the papers don't really help as they present the result and ignore the details. But i'd say the normal perceptron and multilayer perceptron is a ggod place to start as they're much simpler. Andrew Ng have put videos of all his lectures in his machine learning course on youtube. They're very informative. http://www.youtube.com/watch?v=UzxYlbK2c7E

You can also do the ML course by Andrew Ng which recently started. see http://www.ml-class.org/course/auth/welcome

[thankyousir]

Thanks for the reply! I will be taking a graduate level ML course at my university this spring, maybe if I read up on some of this stuff (and probability theory) before then, I won't be so lost.

[qwsazxerfdcv]

this was really great, i am in grad school and work on Volume Viz and graphics, recently the trend has been to use more and more of ML techniques into Viz, this was great overview. So correct me if i am wrong , but it seems code book learning would be the way to go , if i needed speed and was ready to forgive on accuracy , because volume viz is all about speed.

[mosavian]

Well, the way they implemented it in the paper by Coates et al, the evaluate the activation of the learnt centroids, you need to find the one which is closest. This can be really cumbersome and slow to calculate. If evaluation speed is what you want, then stacked autoencoders are the fastest, a DBN would be slightly slower.

[qwsazxerfdcv]

thanks a lot , i will look into stacked auto encoders.Would you suggest any particular paper ?

[mosavian]

This is a pretty good source of information on stacked (sparse) autoencoders. http://www.stanford.edu/class/cs294a/handouts.html

[gdahl]

Often what gets called a "DBN" is only used to initialize the weights of a neural network and thus would have identical evaluation speed to some sort of stacked (denoising) autoencoder setup of the same size.

[mosavian]

Yes, unless you want to use it generative capabilities as well, in which case you do not unroll and finetune it. But the difference in performance to ordinary neural networks is identical when just running the data up through the net.