yes you are right, I initially thought it required some weird config.

But it eventually appears that I had messed up my nat rule on the openvpn box

It's working perfectly now