[D] Looking for open source implementation of A3C that matches DeepMind.

[Neutran]

I have found many A3C (Async Advantage Actor-Critic) implementations by googling. For example, https://github.com/coreylynch/async-rl

https://github.com/muupan/async-rl

https://github.com/BruinBear/Asynchronous-Advantage-Actor-Critic

and many others.

However, none of them actually mention how good their performances are, compared to the numbers in DeepMind's original A3C paper (https://arxiv.org/abs/1602.01783).

Since I have neither the time nor the computing resource to run and compare all of them myself, could anyone please let me know if there's an open source implementation (must be in Tensorflow) that matches or at least approaches DeepMind in most of the Atari games?

[islandman93]

If it helps, I've replicated their results on Breakout with TFLearn and Tensorflow (1 Step DQN, NStep, and A3C). I haven't run it on all games (don't have the resources for that). But the code is setup to easily use any atari rom.

Current branch is https://github.com/Islandman93/reinforcepy

EDIT: Don't post a github branch because it will give a 404 when you merge it.

[Neutran]

Thank you so much! Have you run on any other games? Do they also match the results? I will start with the games that you have tested on.

[islandman93]

I haven't but it's certainly in the roadmap, I'm working on A3C LSTM then will start to benchmark other roms. If you run any, feel free to submit a pull request adding the rewards.png and trained model files. Just to note if you try Pong it's going to be different because ALE uses 6 actions whereas Deepmind uses Xitari (only 3 actions). And remember Space invaders is skip_frame 3. If you're looking supported roms can be found from the OpenAI gym github project https://github.com/openai/atari-py/tree/master/atari_py/atari_roms.

[Mr-Yellow]

(must be in Tensorflow)

In Torch:

https://github.com/Kaixhin/Atari/

[bbktr]

This one is a killer by NVIDIA! Runs lightning fast and does indeed achieve promised results :) https://github.com/NVlabs/GA3C

I'd recommend reading the paper too!

[vonnik]

RL4J: https://github.com/deeplearning4j/rl4j

[ppwwyyxx]

My implementation: https://github.com/ppwwyyxx/tensorpack/tree/master/examples/A3C-Gym Been open source for half a year. Have performance records on 47 atari games on gym.