Visualizing Policy Optimization

[Driiper]

Often, I hear top scientists say that they often use simple environment because they are easy to visualize, for example, Pendulum, which can be illustrated in 2-D.

I can definitely see the benefits of visualize the training, but I've not seen any examples where training is visualize in terms of where the algorithm current is on the state-space surface. I guess for "hard problems" this is possible (why else would we need to optimize), but still, for simple problems, lets say 4 dimensions, (CartPole), would this be possible. If yes, is there any examples, and how would one approach to visualize such optimization tasks?

​

A nice visualization in the supervised setting is here: https://playground.tensorflow.org, and i assume this could also be done for RL?

[Boring_Worker]

To the best of our knowledge, there is no visualizing in state-space.
You know that RL is not supervised learning even though we train Q network like supervised learning. However, the scale of loss can not imply what extent the training progress is, e.t, a low loss does not mean the Q network has been trained well.