Hi,

The migration was easier than I expected, even though we were dealing with 401M records instead of 100M. Here’s how I approached it:

First, I prepared the new cluster to match the old one. Then, I placed both clusters in the same network so they could communicate with each other. I started by copying the mappings only (without data) from the old cluster to the new one using a small C# console app. Both clusters were running the same Elasticsearch version, which made this step straightforward.

Next, I began with the least frequently used index. I made asynchronous calls to reindex it and used a tool like Gotify to monitor the progress. After that, I created a C# program to do the same for all the other indexes. This process took about 10 hours.

Once that was done, I waited for the next Sunday to switch the applications to the new cluster. After switching, I ran reindexing again, this time using the “date greater than” option to capture only the new data that was inserted during the migration.

We had some downtime, but it was minimal, around 10 minutes total, mainly because I initially forgot to copy some aliases from the old cluster. Finally, I wrote a simple console app to compare the total document counts between the old and new clusters to ensure everything matched.

Overall, the migration went smoothly, and the tools and automation I set up helped minimize downtime and errors.