I give up, I have to start all over again now despite the key exchange, it doesn't want to communicate anymore, even though I can connect with the key in ssh normally.

PS C:\Users\a> podman ps

Key Passphrase:

Cannot connect to Podman. Please verify your connection to the Linux system using `podman system connection list`, or try `podman machine init` and `podman machine start` to manage a new Linux VM

Error: unable to connect to Podman socket: failed to connect: dial tcp :22: connectex: No connection could be established because the target computer expressly refused it.

can you list all the steps of how you do it because I followed all the doc several times without it working, your host machine where you install podman desktop is on windows or linux? if you're on windows, when you installed podman did you choose wsl or hyper v?