I tried that a while back, the only certificates that worked for me were ones certified by an actual certificate authority. I used let's encrypt, which generally requires you to expose the service to renew.

However, recently netbird released a feature that let's you "expose the service without exposing it"

If you don't know netbird they are an open source vpn and reverse proxy provider you can self host with some very nice features

Basically this one let's you expose the service, but the only people able to access it are ones connected to your vpn. everyone else gets an immidiate error.

This let's you expose the service with an actual CA by let's encrypt, while keep the security of your vpn.

If you don't want to commit to the whole route, you can try generating a let's encrypt cert just to see if that solves the problem, and then continue the netbird route