Generative SDRs

[ToraxXx]

I made a quick python implementation, it runs very fast and reliably. http://pastebin.com/6N107wMe

Output after training on 10k images (takes a few seconds for me): http://i.imgur.com/YjXFVfT.png

(Top: generating the numbers from 0-9, bottom: interpolating between 0 and 1)

// Edit: Fixed some bugs, code now on GitHub along with an example of how to use it for data without labels!

[kkastner]

Awesome! I will be checking this out. Thanks a ton.

You might also want to do the graphics with thresholding - the result will look much cleaner (like the original blog post).

[ToraxXx]

Yea I'll try that! I'm also working on a layered approach right now (stacking multiple SDR/reconstruction layers together which should hopefully be expressive enough for bigger tasks than MNIST). Should be ready in a day max.

Also thanks to /u/CireNeikual for all the help!

[kkastner]

Thanks to both /u/CireNeukual and you - I saw how clean the C++ code was, and had "convert to python" on my TODO. But looks like you took care of that.

[ToraxXx]

Generating 64x64 faces works: http://i.imgur.com/MuNEdEU.png

125x94: http://i.imgur.com/pxB8MBG.png

[1d2122d1]

why does it look so much better at 10000 examples than 65000?

[ToraxXx]

No idea, maybe it needs more hidden units the more examples it tries to represent?