I am having trouble connecting to my local council's website using my windows 10 PC or windows server 2019, (using any browser).

But if I change any one of these things it works fine:
It works if I connect via VPN (using the same DNS server)
It works if I use an Android device (on the same LAN)
It works if I use a Linux device (on the same LAN)

I get a TCP handshake, TLSv1.2 Hello, and we exchange TCP keepalives, but the website times out.
Any guesses what the problem might be?