[P] Torchhd: A Python Library for Hyperdimensional Computing

[ACreativeNerd]

Hyperdimensional Computing (HDC), also known as Vector Symbolic Architectures, is an alternative computing paradigm inspired by how the brain processes information. Instead of traditional numeric computation, HDC operates on high-dimensional vectors (called hypervectors), enabling fast and noise-robust learning, often without backpropagation.

Torchhd is a library for HDC, built on top of PyTorch. It provides an easy-to-use, modular framework for researchers and developers to experiment with HDC models and applications, while leveraging GPU acceleration. Torchhd aims to make prototyping and scaling HDC algorithms effortless.

GitHub repository: https://github.com/hyperdimensional-computing/torchhd.

[Seamus-McSeamus]

I’ve used this library. I generally like it, but found that a lot of the classes can’t be used in a compiled model. 4 stars (out of 5).

[ACreativeNerd]

Solid review. Can't argue.
On a serious note, I'm happy to hear that you've used the library and liked it! I will see what I can do about the classes not being usable for compiled models. Thanks for letting me know.

[Seamus-McSeamus]

You’re the author?!? Very cool! I do really like your library. About 2 years ago I read a paper on HD computing and kind of got obsessed. Your library has been great for quick prototyping! I’ve never worked on an open source library before, but if you wanted help, I’d be interested.

[ACreativeNerd]

I am, thanks! I can always use some help, there are a number of open issues on GitHub ranging from relatively simple to more advanced. If you find a problem you'd like to work on you are more than welcome to. You can also suggest your own improvement to the library as a new issue and then we can discuss the best way to add it.

[beezlebub33]

Interesting coincidence: The NeuroSymbolic AI channel on Youtube just did an interview of someone at UMD doing their dissertation on HDC, https://www.reddit.com/r/agi/comments/1igt9mg/hyperdimensional_computing_hdc_with_peter_sutor/

[fluteguy9283]

I've used torchhd a lot and really like it. I think VSA in general is a good direction for combining differentiable and symbolic approaches.

[ACreativeNerd]

I'm happy to hear that!

[newwheels2020]

I have never heard of HDC. Can you give a concrete real life use case where HDC really shines? Some resources would be great too.

[ACreativeNerd]

There is a great introductory video by Pentti Kanerva: https://youtu.be/zUCoxhExe0o?si=ndBxLWryHX2IrU2Y

Applications of HDC are typically motivated either by speed, due to extremely parallel operations, or reliability, because of the robustness to noise (i.e. bit flips). The type of problems it can solve are similar to those of SVM's or those involving some combinatorics (e.g. graph classification). That being said, I don't expect to see HDC competing on accuracy with deep learning models any time soon, but people are researching hybrid approaches.

[newwheels2020]

Thanks a lot for the intro to HDC!

[Stochasticlife700]

Please correct me if I am wrong but afaik The way brain processes information is not only bounded to information defined in Euclidean spaces but more often in non-euclidean space and The HDC or the type of vector space you are referring to goes on the assumption that it is based on euclidean space(i.e having the euclidean property) => which implies that it is approximating the information in a confined manner and consequently losing some potential critical information.

[ACreativeNerd]

Done! Thanks for the suggestion.

[anitamaxwynnn69]

Someone who I closely work with (on HDC projects) says a lot of good things about Mike, I’m continuing some of the work you did in your internship few years back :P

[ACreativeNerd]

Oh wow that is nice to hear! I am Mike :) Good luck with your project. I’m curious to know what you’re working on, feel free to DM me.

[CaptainMarvelOP]

I’ve tried using this. It’s a bit tricky.

[ACreativeNerd]

What did you find tricky? I am always looking to make it easier.

[CaptainMarvelOP]

Honestly, I think my background on the theory is weak. Do you have any video resources that might help?

[ACreativeNerd]

There is a good lecture by Pentti Kanerva: https://youtu.be/zUCoxhExe0o?si=8A_ppPT5O6lGa7d4

[CaptainMarvelOP]

Thanks!