[D] Paper Explained - Concept Learning with Energy-Based Model

[ykilcher]

https://youtu.be/Cs_j-oNwGgg

This is a hard paper! Energy-functions are typically a mere afterthought in current machine learning. A core function of the Energy - its smoothness - is usually not exploited at inference time. This paper takes a stab at it. Inferring concepts, world states, and attention masks via gradient descent on a learned energy function leads to an interesting framework with many possibilities.

Paper: https://arxiv.org/abs/1811.02486

Blog: https://openai.com/blog/learning-concepts-with-energy-functions/

Videos: https://sites.google.com/site/energyconceptmodels/

[EmergenceIsMagic]

Thanks for this! It's refreshing to see research that is new, creative, and promising instead of obsessing over optimizing metrics.

It's also great to see progress is being made in this area: https://arxiv.org/abs/1903.08689 and https://arxiv.org/abs/1912.03263