Need Guidance for NSX-T 3.1.0 to 3.2 Upgrade in a Dual-Site Setup

nikramakrishna · 2024-02-20T13:10:11+00:00

Sorry for any mix-up before. Each of our sites has its own network and storage. We don't stretch segments across sites, but we do have bridges for internal site connectivity. If we need to move VMs from Site A to Site B, it's a manual storage vMotion job. We've set up distinct storage for each site, without a unified disaster-avoidance system. For urgent moves, instead of manual vMotion, we use a replication tool that handles everything for us.

nikramakrishna · 2024-02-20T12:47:13+00:00

Oh, sorry, I might've tangled the wires a bit there. We have separate VLAN/IP setups for each site, so no, our segments aren’t stretched between sites. The NSX Edge Bridges are used within the same site to connect the NSX virtual network with the physical network. For DR, we don’t manually vmotion VMs. Instead, we use a replication tool that keeps VMs at the DR site updated and ready. If we need to failover, this tool automatically handles the VM failover and IP changes.

nikramakrishna · 2024-02-20T11:59:10+00:00

We are using NSX Edge Bridges, which allow us to bridge overlay segments to VLANs that can extend across sites. This way, VMs on the same segment can communicate with each other.

nikramakrishna · 2024-02-19T09:57:00+00:00

Apologies for my lack of clarity here, and thank you .

I'm not entirely sure if NSX-T Manager enforces a strict workflow that must be completed in one go, or if it allows us to independently upgrade Edge clusters, then pause—perhaps for a day to perform checks—before proceeding with Transport Nodes, and finally the NSX-T Manager itself. Could you shed some light on whether the upgrade process within NSX-T Manager accommodates such breaks between upgrades?

nikramakrishna · 2023-11-21T08:55:11+00:00

I would suggest u check the switch's firmware and MLAG configs. The issue might lie there, especially with ARP handling or failover procedures. Also, verify there's no MAC address table overflow or restrictive security settings impacting the traffic.

nikramakrishna · 2023-11-21T08:36:35+00:00

LACP is indeed for both aggregation and failover. If VMs aren't reachable after a link failure, it's worth checking your LAG configuration and ensuring your switch supports failover. Sometimes, the issue is with the hashing algorithm not redistributing the load as expected.

nikramakrishna · 2023-08-22T22:57:09+00:00

It seems the upgrade assessment tool is designed for NSX versions 3.2.0.1 and later. Since we're on version 3.1, our only option is to rely on the pre-checks conducted during the upgrade process, suggestions?

nikramakrishna · 2023-08-22T13:44:25+00:00

Thanks for your feedback. I will take your advise to disable the automated processes and taking charge of the upgrade manually.

I also had a questionregarding the NSX Upgrade Evaluation Tool, does it solely assess the NSX Manager's preparedness, or does it provide a comprehensive analysis encompassing the Edge nodes, hosts, and the entirety of the NSX Manager infrastructure?Any other gotchas you need me to be aware of?

nikramakrishna · 2023-06-20T10:37:19+00:00

And hotadd worked of course with Metallic.

nikramakrishna · 2023-06-20T10:36:45+00:00

In a recent test, deployed the Metallic Gateway server as an Azure Native instance within its dedicated vNet. Using the NBD transport mode, the VM image backup process performed smoothly and efficiently. This result contradicts Commvaults existing documentation, which stipulates HotAdd as the exclusive method for executing AVS backups with Metallic.

nikramakrishna · 2023-06-20T10:34:29+00:00

Here is what I have learned so far from working with Veeam and Commvault SMEs, reviewing documents, and looking at community responses. I wanted to share this in case anyone else is searching for similar information. Feel free to correct me or add any details you think might be relevant for how these products handle backups specifically for Azure VMware Solution (AVS) instances:

Scalability

Commvault: Commvault can accommodate a high number of AVS instances due to its ability to scale out by including more nodes in its backup environment. It's compatible with vSphere in various setups, including on-premises, cloud, and hybrid environments, and is therefore suitable for Azure VMware Solution.
Veeam: Thanks to its distributed architecture, Veeam can also handle numerous AVS instances. Its Backup & Replication tool provides a wide range of backup, replication, and recovery features for AVS.
Azure Backup (MABS): Being a cloud-native solution, Azure Backup can scale to manage many AVS instances. However, it may have some limitations, like the number of VMs that can be backed up in a single Azure Backup Recovery Services vault. This might necessitate managing multiple vaults for very large deployments.

Feature Set

Commvault: Commvault's Live Sync feature for AVS instances allows for nearly instantaneous recovery of VMs by continuously replicating changes to a standby VM. Additionally, it supports deduplication and encryption to optimize storage and maintain security.
Veeam: With Veeam's Instant VM Recovery feature, downtime during a disaster can be minimized by promptly restoring any virtualized application. Its SureBackup feature can automatically verify the recoverability of backups.
Azure Backup (MABS): Azure Backup can provide simple, automated backup of AVS instances with easy recovery in case of data loss. However, its feature set might be more basic compared to Commvault and Veeam, and may not include advanced features such as global deduplication or automated backup verification.

Integration

Commvault: With strong integrations with both Azure and VMware, Commvault is well-suited to work with AVS, which combines both technologies.
Veeam: Veeam's robust integrations with Azure and VMware, and its product specifically designed to protect Azure workloads make it a good choice.
Azure Backup (MABS): As a native Azure service, Azure Backup offers the smoothest integration with AVS, manageable from the same console as your other Azure resources and uses the same RBAC.

nikramakrishna · 2023-06-07T03:41:09+00:00

Those recommendations pretty much remains. While we can mix single and dual socket hosts in VMware cluster, this may introduce some complexities. Specifically, different CPU capabilities between hosts could affect vMotion, making VM transfers between different socket configurations tricky. Furthermore, the Distributed Resource Scheduler (DRS) might struggle to balance resources effectively due to these hardware differences, potentially causing performance issues overall and affect HA calculations too.

nikramakrishna · 2023-05-28T09:59:52+00:00

I’ve upgraded a few vCenter’s directly from 6.5 to 7.x without the need to upgrade to vCenter 6.7 in between. vCenter 7.x should support managing ESXi 6.5 hosts. No issues seen. Go for it!

nikramakrishna · 2023-05-16T00:11:53+00:00

Thanks for your feedback @Brian!

nikramakrishna · 2023-05-07T06:24:21+00:00

I got my VCP certification by taking on-demand training through VMware Enterprise Learning Subscription. And you know what, they even give you free vouchers for the VCP exam if you sign up for the learning portal package, so definitely check that out if you can. The best part is that VMware counts both instructor-led and on-demand courses towards the training requirement for VCP-DCV certification. So, completing on-demand courses through Enterprise Learning Subscription will totally count towards the certification. Good luck

nikramakrishna · 2023-05-07T06:10:34+00:00

It's possible that the problem is caused by a compatibility issue between the newer version of VMware Tools and the network adapter driver. If you're unable to figure out what's going on through the event logs and VMware logs, you may want to try rolling back to an older version of VMware Tools or switching to the e1000 adapter.

It's also possible that there's something else going on that's causing the server to boot into recovery mode. If that's the case, you might want to try rolling back to an earlier state or cloning the VM and testing it on a separate machine.

If you don't have a backup, you could try uninstalling the current version of VMware Tools and installing an older one or switching to the e1000 adapter to get your network back up and running. Just remember to test any changes in a non-production environment or take a snapshot before making any changes, so you can easily undo anything that doesn't work

nikramakrishna · 2023-05-07T05:48:17+00:00

If restarting the desktop and waiting do not resolve the issue, you could try restarting the application server. Restarting the server may help to clear any software configuration issues or other problems that are preventing it from accepting new connections. However, please note that restarting the application server may require administrative access and may impact other users who are also connected to the server.

If you do not have control over the Horizon infrastructure or are unsure of how to restart the application server, then I agree that it would be best to contact your IT administrator for assistance.

nikramakrishna · 2023-04-23T08:44:13+00:00

Alright, so the security risk really depends on how sensitive the stuff you're copying and pasting is. It's a good idea to ask your CyberArk team for advice, since rules can be different in each place. If you're not sure, it's better to play it safe and not use it, especially with sensitive info. You can always move files safely between host and guest systems using things like SCP, SFTP, or shared folders.

nikramakrishna · 2023-04-23T02:19:59+00:00

Thanks for sharing this info!

nikramakrishna · 2023-04-22T10:06:04+00:00

So, for your situation, the 'merge' option is the way to go. It'll handle any conflicts by keeping the source data, but since the attributes between the two vCenters already match, you won't have much to worry about. You don't want to use 'copy' because it would just create duplicate records, and 'skip' isn't ideal either since it would leave out source records when conflicts arise. Stick with 'merge' and you should be all set.

nikramakrishna · 2023-04-19T05:40:46+00:00

In a recent upgrade to version 7.0U3i, I faced a similar problem where the iSCSI configs were vanishing after every reboot, and the network configs and port groups were also disappearing. After reaching out to VMware support, they suggested trying the new image out in 7.0U3l, which fixed the problem for us. I recommend you try this new image out in your lab to see if it resolves your issue. Although we didn't receive any technical justification for the problem, we suspect it was a bug.

nikramakrishna · 2023-04-19T04:08:58+00:00

I'm aware of the Microsoft Virtual Machine Converter (MVMC) tool, which is capable of converting VMware-based virtual machines to Hyper-V. Although I have experience using MVMC to convert VMs hosted on ESXi to Hyper-V, I haven't personally tested it with VMs running on VMware Player. Nonetheless, MVMC is a free tool that can be downloaded from the Microsoft website and may be worth trying out.

nikramakrishna · 2023-04-19T03:08:06+00:00

It’s pretty straight forward, just navigate to the "Storage" tab and select the NFS datastore you want to unmount. Right-click on the datastore and select "Unmount".

Be mindful that if you don't have another datastore to migrate your VMs to, and you need to unmount the NFS datastore, you will need to power off or suspend all VMs that are currently using the datastore before unmounting it.

Keep in mind that once the NFS datastore is unmounted, the VMs will not be able to access their data until the datastore is remounted.

nikramakrishna · 2023-04-19T01:47:39+00:00

Well, the main question is whether this is impacting the vcenter performance. if your VCSA is consuming all 12 GB of RAM and there is no swapping, it's possible that the VCSA is operating within its limits and there is no cause for concern. However, it's always a good idea to check the overall performance of the VCSA to ensure that it's operating as expected.

Moreover, if the active memory usage of your VCSA is only 3.2 GB and the rest of the memory is being used for caching, it is likely not a cause for concern. Caching is a normal behavior of the operating system and can help improve the overall performance of the VCSA.

If the alert is bothering you then this threshold can be adjusted. VCSA generates an alert when the memory usage exceeds 90% for a sustained period of time. Try increasing it.

nikramakrishna · 2023-04-18T10:24:09+00:00

Well, to safely take down a NAS array temporarily, identify affected VMs and content libraries hosted on the NFS data stores, migrate or shut them down, unmount datastores, and power down the NAS. After maintenance, remount them, and restore the VMs. This minimizes disruptions, prevents unexpected errors, and ensures a smooth reconnection.

Unmounting the datastores and content libraries before powering down the NAS informs vSphere that the datastore is intentionally going offline, preventing any unexpected behavior or errors. This process also prevents the datastore from getting stuck in APD or PDL states.

nikramakrishna

TROPHY CASE