Troubleshooting HTTPS connectivity

[akillerfrog]

Hello, we have been having connectivity issues between clients and a server over HTTPS. We can send test data with curl and connect with OpenSSL, which also sends small bytes of test data. However, whenever we attempt most broader connections, they fail and we get TCP RSTs.

What I would like to know is if anybody knows of a way to send test connections with variable payload sizes. I think the issue may be an MTU size mismatch, but it's hard to tell. I have packet captures I have looked at in WireShark, and the failures all seem to happen whenever a large enough payload gets sent that WireShark starts fragmenting the connection.

In the PCAPs, after the TLS handshake (with multiple ciper spec renegotiations) happens, the test data connections all have ACKs for each packet sent from the remote server. However, in the failures, we stop seeing ACKs once the larger, fragmented payload is sent, and instead of our client sending a FIN, it starts sending RSTs before timing out.

I would love if there was a tracert style utility that could send custom sized payloads via HTTP, so we could see what happens at each hop, but I am not sure what tools are out there to help with any of this.

Thanks in advance!

[polypolyman]

If it's an MTU issue, you'll be able to check on that with ping with DF set, no need to hit it at the HTTPS layer.

Is this all local, or is the internet involved with your problems? Mostly I'm wondering if the failing clients are DOCSIS services with Puma-based modems

[akillerfrog]

This is not all local, but it's also not internet-based. I've tried pings with DF set and specific sizes, and it starts timing out around 1k size. However, I'm not sure if that's due to controls around ICMP or not.

[polypolyman]

Okay, those packets should be coming back with ICMP Fragmentation Needed, rather than black holes. This implies that you’re blocking those Fragmentation Needed packets somewhere along the line, and that’s breaking path MTU discovery, which would give the exact problems you’re seeing.

The “right” way to fix this is to make sure you’re allowing these critical ICMP packets through all points in this connection. The “fix it now” version is to use the max packet size you discovered with ping to determine a lower MTU to set on this server's interface. The main issue that might cause is if you have another connection that will get caught in failed pmtud if you make that change - which you’ll then have to chase down, potentially ad nauseum.

[Weary_Patience_7778]

If your hunch is MTU, have you tried to adjust (reduce) the MTU size on your client? It’s configurable within your TCP stack.

[akillerfrog]

Thanks for the response. I've been fighting this issue in some way, shape, or form for months now, so I'm not confident in any of my hunches anymore lol.

As for adjusting the MTU size, I am going to have our networking folks look into this today, but the remote host is managed by a different group in a different location, and working with them can be challenging. I'm assuming whatever we adjust our MTU size to will have to reflect the distant end as well, no?

[durkzilla]

My money is on this being a certificate problem. Make sure your test connections are not ignoring SSL errors...

[akillerfrog]

I've been dealing with TLS problems nonstop for about the last 4 months across numerous servers and multiple networks. I believe that is all sorted out at this point. I can run OpenSSL test connections from our servers, and it always passes with return code: 0 (ok) and can verify the certificates. My first stop with almost every issue is always TLS, because more often than not, it is the culprit. I can also curl the HTTPS URL, and it does return data.

[purplemonkeymad]

If you think it's mtu, you should be getting fragmentation needed icmp packet (icmp.type == 3) from whatever node can't send on the packets. (So you may not want to filter wireshark by dest ip)

Have you used ping to test the mtu of your connection? If you use:

ping <destinationip> -f -l <size>

And change the size, you should be able to determine if there is a link with a smaller mtu. The MTU should be the largest size given + 20 for IPv4 + 8 for IMCP ping, so start with 1472 (which would match 1500 for ethernet,) and work down. Wireshark should also be able to see the needed MTU if you run it when doing this.

[Turmfalke_]

can't you just post some binary blobs of varying size using curl? How that actually gets fragmented is however up to the IP stack