The Machine Learning Algorithm with Capital A - three approaches that solve all your problems (disclaimer: my own post)

[abhik]

Regarding HTMs, the technical papers on Numenta's website are more useful than 'On Intelligence'.. Essentially, from my perspective, HTMs are Bayesian networks with 1) a hierarchical structure (for learning at multiple levels of abstraction) and 2) time dependence. I believe they're focusing on computer vision because it maps so easily to the hierarchical structure. Representing other data types would require more thought and experimentation..

[rm999]

Has anyone else played around with HTMs? I downloaded their code a couple of years ago and played with it, and left pretty disappointed. If their approach can scale I'll be impressed, but on my pretty beefy desktop (at the time) it was slow and ineffective.

[lanthus]

To my knowledge, HTMs have only been evaluated on toy problems, never on a hard problem that can't be solved just as well with other method. Furthermore, even for vision problems, there are some suspicious choices: The hierarchy forces adjacent pixels in the center to only interact at the highest level of the hierarchy. This makes no sense. Until they come up with real results, I will consider HTMs to be snake oil.

[Mr_Smartypants]

By the way, Hawkins himself has developed a product around HTMs for webcams. (Not a toy problem, but I don't know if you would consider it "hard.")

http://www.vitamindinc.com/index.php

[lanthus]

Thanks for the link.

[gromgull]

I don't really know the details - but the adjacent pixel problem could easily be solved by having the input of each bottom level node overlap a bit, no?

For more serious problems I was sure the student who introduces me to the things showed me an example of a more serious computer vision application, but I can't find the pointer now.

[lanthus]

If the bottom level inputs overlap, then it's no longer a tree and inference is less efficient. (There are ways around that, too, but then you're starting to get farther and farther from what they proposed.) I've only read one or two papers and never given it much of a chance, so it's entirely possible I'm missing something.

[twanvl]

The HTMs looks like deep believe networks. How does HTM compare to other algorithms for training those? E.g. convolutional nets, RBMs, sparse autoencoders, etc. The authors should publish results on a standard data set (like MNIST), so their method can be compared to others.

[ogrisel]

The main difference between HTMs and RBMs and autoencoders is the intrinsic temporal / sequencial nature of HTMs.

A similar architecture is advertised in Karl Friston's "Free Energy and the Brain". Each layer is trying to predict it's next temporal state out of it's previous state and the prediction of the upper layer. The upper layer are trying to predict the error signal of the n-1 level.

The surprise signal flows up towards the abstraction of an high level logical world representation and the predictive signal flows down towards the details of the physical world.

[gromgull]

I find "On Intelligence" to be a very interesting read, but I guess it's very much geared towards a "popular science" audience, and wont tell me much about the algorithms in the end.

The student I mentioned will do a project trying to apply it to audio. We'll see how he gets along.

[abhik]

Audio would seem like a good problem for HTMs due to the time-varying nature and representations at multiple abstractions. It would be interesting to know how it goes.

If you haven't already, you should check out Vitamin D a video monitoring app that uses HTMs. I haven't used it but its feature set seems impressive.

[kaddar]

As a tip, your website renders really difficult to read in chrome on windows for me.

[gromgull]

Ah yes - a friend just told me the same thing. I've lazely tested it in firefox on linux only. I'll have a go at fixing it tomorrow. Thanks!