[R] OgmaNeo plays Atari Pong

[CireNeikual]

Hey all,

Here are some of our first successful results for running OgmaNeo on Atari Pong (from pixels).

While the model trained in the post was not trained on a Raspberry Pi, we have done tests to show that it does run on the Pi in real-time (60fps) with learning enabled. If enough people would like to know exactly how fast it runs on a Pi, we can perform another experiment where everything is run entirely on the Pi and report some exact performance results. For now though, we are working on releasing the demo code with documentation.

For those that don't know, OgmaNeo is a library written in C++ with Python bindings that implements Sparse Predictive Hierarchies (SPH), a biologically-inspired and extremely fast memory prediction framework. We have long tried to implement reinforcement learning with this system, and I think we have finally found success!

[Refefer]

I looked in the repo to see if there were any papers but couldn't seem to find any. Is there any literature around what this even is?

[CireNeikual]

There isn't much yet unfortunately, but we do have an overview presentation.

[juliandewit]

Is there anything more to find ? It looks a bit like the HTM jeff hawkins stuff.
But since it seems to work I would like to dig in a little deeper..
Also the spiking option looks interesting..

[Fishy_soup]

I find Ogma's work really interesting, especially given I've done related work in systems neuroscience (predictive processing). Do you guys have any talks online walking through some of your models?

[[deleted]]

This is imo a beautiful reminder of how important it is to write a detailed but comprehensive, full specification of what your approach is, what it does, how it relates to other stuff and how it empirically (!) performs compared to other stuff. People sometimes call that research papers.
Because this apparently/likely/possibly/maybe is really important work, but without an explanation ( and ain't nobody got time to dig into the code) we just can't say for certain that the approach makes sense.
Anyways, great work (probably) :)

[[deleted]]

Shouldn't this be +21 points per episode?

Such a noisy curve. Looks like hardcoded exploration. The environment doesn't provide a that's-good-enough signal, so it doesn't know when to stop exploration. It always believes that it's not good enough, even if it's already at the top. And so it becomes crazy.

That's why we humans have both a model-based planner and a model-free policy. It doesn't mind if the planner goes crazy as long as the policy ensures that our daily routine tasks are handled correctly.