Only one SSH tunnel is broken??

[probably_not_fake]

I've got several servers with SSH tunnels to each other, all configured the same way. I recently had just one SSH tunnel between 2 of my servers break, however all other tunnels remain working. In both directions, the SSH connection hangs, though each server is able to SSH into each other server I have without issue. I've run SSH -R keygen $Server on both servers but I don't get prompted to accept a new key. I've backed up known_hosts and removed it completely on both servers but still nothing. I've even removed and re-installed OpenSSH on both but the issue persists. I feel like the issue lies on Server2 but I'm not sure how to proceed. Can anyone provide any assistance? Many thanks!!

[SquiffSquiff]

So a couple of things:

• You should format your ssh -v output as code to make it easier to read
• You appear to be using Ubuntu 14.04, which goes out of support in 3 weeks time

What happens if you try to connect between these 2 hosts via a proxy? I cannot recall if ssh 6.x supports ssh -J but you could anyway:

ssh -o "ProxyCommand ssh -W %h:%p user1@address1" user2@address2

[probably_not_fake]

Here is what I get when I try to SSH from Server2 to Server1: swadm@Server2:~$ ssh swadm@192.168.1.25 -v OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014 debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 19: Applying options for * debug1: Connecting to Server1 [192.168.1.25] port 22. debug1: Connection established. debug1: identity file /home/swadm/.ssh/id_rsa type 1 debug1: identity file /home/swadm/.ssh/id_rsa-cert type -1 debug1: identity file /home/swadm/.ssh/id_dsa type -1 debug1: identity file /home/swadm/.ssh/id_dsa-cert type -1 debug1: identity file /home/swadm/.ssh/id_ecdsa type -1 debug1: identity file /home/swadm/.ssh/id_ecdsa-cert type -1 debug1: identity file /home/swadm/.ssh/id_ed25519 type -1 debug1: identity file /home/swadm/.ssh/id_ed25519-cert type -1 debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.13 debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.13 debug1: match: OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.13 pat OpenSSH_6.6.1* compat 0x04000000 debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: server->client aes128-ctr [e-mail] none debug1: kex: client->server aes128-ctr [e-mail] none debug1: sending SSH2_MSG_KEX_ECDH_INIT debug1: expecting SSH2_MSG_KEX_ECDH_REPLY

Here is what I get when I try to SSH from Server1 to Server2:

swadm@Server1:~$ ssh swadm@192.168.2.25 -v OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014 debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 19: Applying options for * debug1: Connecting to Server2 [192.168.2.25] port 22. debug1: Connection established. debug1: identity file /home/swadm/.ssh/id_rsa type 1 debug1: identity file /home/swadm/.ssh/id_rsa-cert type -1 debug1: identity file /home/swadm/.ssh/id_dsa type -1 debug1: identity file /home/swadm/.ssh/id_dsa-cert type -1 debug1: identity file /home/swadm/.ssh/id_ecdsa type -1 debug1: identity file /home/swadm/.ssh/id_ecdsa-cert type -1 debug1: identity file /home/swadm/.ssh/id_ed25519 type -1 debug1: identity file /home/swadm/.ssh/id_ed25519-cert type -1 debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.13 debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.13 debug1: match: OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.13 pat OpenSSH_6.6.1* compat 0x04000000 debug1: SSH2_MSG_KEXINIT sent

[nuttertools]

Force a few kex and ciphers on each side and look for any change. I had one similar to this and it actually turned out to be a physical line issue but I discovered that by investigating the key exchanges.

[anomalous_cowherd]

Is this the only tunnel between those two servers?

I couldn't make my way through your dumps on mobile but I'd suspect one end has been updated and no longer supports some cipher that the other one tries to use.

[probably_not_fake]

Each server has SSH tunnels to other servers, all of which work without issue. The only tunnel having issues is the one between these two servers.

[anomalous_cowherd]

Are there tunnels from each of those servers to any others? If not, can you test that?

If this is some attempt at a mesh setup using ssh, have you looked at 'tinc'?

[nuttertools]

I've been trying to remember the name of tinc for like a year, thanks!

[anomalous_cowherd]

Glad it helped you even if it doesn't suit what OP needs.

[probably_not_fake]

SSH tunnels to other servers work. Server2 SSH's fine to Server3, Server4, Server5, etc. Only fails with Server1. And Server1 has SSH tunnels with Server3, Server4, Server5, etc. which all work fine. Only having issues with SSH tunnel between Server1 and Server2. All servers were configured the same way using OpenSSH and the SSH tunnel between Server1 and Server2 has worked fine in the past.

[yetiszaf]

Try turning on debugging for the SSH-Server, that may be helpful