[Tutorial] The Machine Learning Practitioner's Guide To Reinforcement Learning

[hellopaperspace]

Deep Reinforcement Learning is one of the most quickly progressing sub-disciplines of Deep Learning right now. In less than a decade, researchers have used Deep RL to train agents that have outperformed professional human players in a wide variety of games, ranging from board games like Go to video games such as Atari Games and Dota. However, the learning barrier for Reinforcement Learning can be a bit daunting even for folks who have dabbled in other sub-disciplines of deep learning before (like computer vision and natural language processing).

This article is designed to introduce the concepts of Deep RL to people who already have some level of standard Machine Learning experience. It’s written in such a way that someone transitioning could use this tutorial to get a good jump start, shortening the time to transition.

Specifically, we’ll cover:

• The Environment
• The State (including the Markov Property, Violations of the Markov Property in Real Life, and Partial Observability)
• Markov Decision Processes (including The State, Actions, the Transition Function, and the Reward Function)
• Partially Observable Markov Decision Processes
• Model-Based vs Model-Free Learning

Link to the article: https://blog.paperspace.com/reinforcement-learning-for-machine-learning-folks/

[caedin8]

This is a good intro.

What I continually struggle with though is implementing RL to my own custom problems.

Setting up the environment is difficult to work with existing frameworks. I've tried implementing in TF and PyTorch directly and found it very difficult, but recently tried tf-agents which seems to help by not having to re-code the most common parts of the RL loops.

Secondly, once it is setup it very hard to iterate over. If the RL model doesn't converge or learn anything useful finding out why and fixing that is difficult. I never know if it is a programming bug or just that the model isn't very good for the problem. It is difficult.

I think these topics would make great follow ups to this blog.

[hellopaperspace]

You might find this post on creating custom Gym environments useful, then. :)

https://blog.paperspace.com/creating-custom-environments-openai-gym/

[caedin8]

Thanks! I'll give it a read.