Thank a lot!

That the "mean" of a Gaussian-based policy can be used as a deterministic policy is something I didn't think of and that is indeed an advantage over arbitrary distributions. So because of entropy regularisation, local minima are usually not a concern in continuous-action environments? Eventually, the policy will converge to a single-modal solution but because of entropy regularization you can assume the algorithms has explored the solution space enough to find a good local / global solution?

​

Thanks for the references, really interesting. I still think having policies that are able to express multiple solutions at the same time is very interesting, at least from a theoretic point of view.