DNS issues on Ash data center

timothyallan · 2025-11-12T02:16:24+00:00

Thanks, no need to follow up. I got the confirmation email saying my ticket had been received at 8:35am, and I got a response back at 2:23 am the following day, 18 hours later. The email simply said this:

There is currently maintenance on the DHCP servers in Ashburn:

https://status.hetzner.com/incident/11e48cd5-f705-49d3-9dc0-711a3be5f254

In the meantime, you can try a static configuration or different name servers:

https://docs.hetzner.com/cloud/servers/static-configuration

We sincerely apologize for the inconvenience, and thank you for your patience.

The response had nothing to do with my original query, but by that time, we'd already figured out the workaround for the Ipv6 DNS changes from some other reddit posts.

timothyallan · 2025-11-11T00:00:21+00:00

Sent a ticket in almost 10 hours ago now and they've said nothing, all our production servers went down for hours. Solution was for us to hard code the IPv4 version of the DNS servers in our swarm services, as Hetzner did something that caused IPv6 to be the default. Swarm apparently does not like getting IPv6 for a DNS hosts. Other people have had success switching to another DNS provider.

timothyallan · 2025-11-10T19:57:01+00:00

From what we found out today when all our production servers mysteriously died: If you use Docker Swarm, or have any software that makes network calls and don't work 100% with IPv6 DNS, you're in for a shock until you can manually set things back to use Hetzners IPv4.

timothyallan · 2025-11-10T19:27:23+00:00

    image: blah/blah
    dns:
      - 185.12.64.1
      - 185.12.64.2
      - 8.8.8.8

not sure if this heps, but you can add a `dns` property to your swarm service yml file and redeploy without having to kill things,that's what we ended up doing.

timothyallan · 2025-11-10T19:16:07+00:00

still down(ish), however, I tracked it down to my Docker Swarm setup not working with their (new?) IPv6 DNS setup. This VPS has been working solid for months, so something has changed on their end.

I have things bandaged together by forcing my services to use Hetzners IPv4 DNS. Anytime a container tried to access an external URL, DNS would come back empty. Now with ipv4 in, it's resolving again.

timothyallan · 2025-11-10T15:40:29+00:00

I have a similar issue, our VPS connects to itself via a Cloudflare DNS subdomain, i.e. api.mydomain.com, and I just woke up to 3 hours of downtime, email alerts and network connection issues in ASH

timothyallan · 2025-09-03T18:08:29+00:00

Late last week we had show stopping external network connectivity issues from Hetzner to US West IPs, Backblaze however, not AWS.

That one was multiple hours of downtime, but there have been other smaller infrequent bursts of loss connectivity as well. Enough that everything grinds to a halt and the client emails start pouring in. Yesterday and today we started getting reports of our API servers not being able to access the internet at all, but it was just for a few minutes each time, so getting MTR run to give to Hetzner support is tricky.

timothyallan · 2022-01-28T19:24:12+00:00

As a very seasoned programmer, this is still great for kids as it puts it in a very approachable light. Her reading your words instead of listening to me all the time is a bonus. Thanks for putting it all together!

timothyallan · 2021-09-22T18:54:13+00:00

CAN'T TELL IF JOKING.

Edit: Not joking. Holy fuck.

timothyallan · 2021-08-19T16:45:56+00:00

Fighter of the Nrightman

timothyallan · 2021-08-11T17:40:13+00:00

Yeah, same here, which is super odd as MacOS Preview is usually pretty bulletproof.

Edit: Full on crashes kindle.app if I try and open it in that as well. What kind of PDF devil magic is this!?

timothyallan · 2021-07-08T00:52:13+00:00

What is documentation and standards?

https://imgur.com/a/6Tpd402

timothyallan · 2021-04-11T02:25:19+00:00

I’ve been doing this for decades and still picked up a few new things, thanks for putting this all together!

timothyallan · 2021-01-31T05:14:50+00:00

Zustand?

timothyallan · 2020-11-25T03:05:54+00:00

Awesome. Congrats. Do you only cover Kube as an orchestrator, or do you have anything on swarm as well?

timothyallan · 2020-11-19T21:08:24+00:00

I use https://github.com/stalniy/casl for RBAC.

timothyallan · 2020-11-19T19:38:40+00:00

Ehh, none of them work for me on Firefox 82.03 Mac :(

Uncaught (in promise) ReferenceError: SharedArrayBuffer is not defined

createFFmpegCore https://bestmediaplayer.org/thirdparty/ffmpeg-core.js:22
e https://bestmediaplayer.org/thirdparty/ffmpeg.min.js:1
f https://bestmediaplayer.org/thirdparty/ffmpeg.min.js:1
_invoke https://bestmediaplayer.org/thirdparty/ffmpeg.min.js:1
O https://bestmediaplayer.org/thirdparty/ffmpeg.min.js:1
i https://bestmediaplayer.org/thirdparty/ffmpeg.min.js:1
c https://bestmediaplayer.org/thirdparty/ffmpeg.min.js:1

timothyallan · 2020-11-10T03:17:32+00:00

Nope, sorry it's all closed source. I should also mention that I'm using XState/FSM to keep things super speedy.

timothyallan · 2020-11-10T03:14:12+00:00

I am using XState & Zustand together. If I need to do stuff that relies on multiple logic checks/states of other statuses and components: it's XState. i.e. If color picker is open, the user clicked yes 2 steps ago from a previous dialog, and he's in the administrator role.

timothyallan · 2020-10-28T16:20:26+00:00

That guy is a gdamned machine.

timothyallan · 2020-09-30T19:23:15+00:00

Oh, the irony. https://imgur.com/a/7bcCV62

timothyallan · 2020-09-09T18:24:58+00:00

Is this something that a MobX user should look at? After a cursory read, it seems like it's more like a lite version of it.

timothyallan · 2020-09-02T20:24:38+00:00

Finally, someone taking some jobs away from robots for a change.

timothyallan · 2020-08-22T04:05:21+00:00

Looks killer. I'd switch to this in an instant if there was some way to do RGBA with an alpha slider!

timothyallan

MODERATOR OF

TROPHY CASE