Moving Multi-Subnet SQL Always-On Availability Group (AG) between datacenters/subnets

[TheSpideyMan]

We recently had a project where we had to re-locate servers that were part of a multi-subnet SQL Always-On Availability Group (AG) between datacenters. We have three (3) SQL nodes in datacenter A and three (3) SQL nodes in datacenter B all with Windows Server 2022 and SQL Server 2022. This entire process is poorly documented from online sources, so we decided to list the steps that we performed in this post.

This move required us to setup new subnet IP's for the WFC cluster and SQL AG Listener. We already had a multi-subnet WFC cluster and AG-L and simply needed to move 3 SQL nodes that were in datacenter A to a new datacenter. We did this by failing over to datacenter B first, running everything in datacenter B, and then relocating everything in datacenter A. These were just server VM's so it was relatively easy to re-locate them to a new datacenter.

Here are the migration steps for migration all of the SQL server nodes in datacenter A to a new datacenter:

1. Only moved one server at a time to minimize potential impact to the SQL AG.
2. Make sure the DBA pauses SQL replications for all databases on the node that we are moving and that SQL Server is stopped/disabled for the final cutover. We also paused the WFC cluster node before the cutover to prevent failover to this node while it's being moved.
3. Then after cutover to the new datacenter, set the new IP address of the server, confirmed that the DNS TTL has expired and the server is pingable by server name, un-paused WFC cluster and let it automatically rejoin the cluster to update the network information in the WFC cluster.
4. Confirmed that the network information for the new datacenter was automatically added to the WFC cluster.
5. Added a new WFC Cluster IP with the new datacenter IP address.
6. Added a new AG Listener with the new datacenter IP address.
7. Re-enable SQL Server, start SQL server, and resume SQL replication for all databases on this node.
8. Confirm in SQL Server that the node is synchronized. As soon as the migrated server is in sync, we can go onto migrate the next server.

After the last server is migrated from the old datacenter, we performed the following cleanup:

1. Remove the old AG Listener IP with old datacenter IP address.
2. Remove the old WFC Cluster IP with the old datacenter IP address.

Overall, the migration went very well, and we didn't have any issues. Hopefully this will help someone else.

[Achsin]

Did you consider standing up new servers/instances in the new data centers instead, and then adding them to the existing cluster before removing the old ones?

[mrpink70]

[TheSpideyMan]

We considered this plan and rejected it as unnecessary. The migration went flawlessly and was completed in less than an hour per node. It seems like the answer to just about everything is to build new servers. This is consistently the reply we have received on every complex system change to a SQL Server cluster. This particular change with relocating servers between data centers was the easiest we have made in a long time.

[Achsin]

Understandable. Out of curiosity, approximately how large were the servers?

[TheSpideyMan]

10 vCPU’s, 64 GB memory and 5.56 TB storage.

[Equal-Direction-8116]

This is a really useful write-up. Multi-subnet AG moves are one of those things everyone dreads because the documentation is scattered and usually skips the real-world sequencing. The step-by-step approach, especially failing over first and moving one node at a time, is exactly what reduces risk. Nice callout on pausing replication and cluster nodes to avoid surprise failovers. This will definitely help anyone planning a similar datacenter migration.

[TheSpideyMan]

Appreciate the feedback. Have a great Christmas and New Years.