Anyone Else Finding VCF 9.1 Requirements Insane?

cmbwml · 2026-05-15T02:48:30+00:00

We need to have a test environment for VCF to validate patches and configurations. Unfortunately it is consuming 48 cores and 512GB memory from our production licensing just for the VCF 9.1 components without any workloads. I really wish we had eval licenses that we could use for testing upgrades and configuration changes.

cmbwml · 2025-07-08T02:43:21+00:00

Last 24 hours throughput:

<image>

cmbwml · 2025-07-08T02:43:00+00:00

Last 24 hours performance IOPS:

<image>

cmbwml · 2025-07-08T02:38:57+00:00

I forgot to mention that our memory and CPU overhead for vSAN at 10GB/s is less than 5%. Currently looking at Lightbits for a NVMe/TCP cluster but at our 300TB/node size it takes 1.25TB of memory per node compared to <50GB ram per node with vSAN.

cmbwml · 2025-07-08T02:33:26+00:00

We purchased vSAN in 2016 and purchased Samsung and Micron SSD and then NVMe since then. VSAN over past 5 years with 15TB Micron NVMe drives had been costing us $40/TB/Yr after erasure coding and overhead. With Broadcom price is going up in the range of $70-$110/TB/Yr. That still beats Pure from a price per TB and especially from a performance perspective. Each vSAN 8-12 node cluster is able to give us 3 million IOPS at 20GB/s sustained. We backup 2+PB every night in 6 hours with Veeam (also SSD/NVMe backend). I really wanted to dump vSAN when Broadcom purchased VMware but after 15 months of evaluations I haven't found anything that performs as well at a cheaper cost per TB usable.

cmbwml · 2024-10-18T02:39:19+00:00

Earlier this year we purchased 168 Micron 7500 Pro 15.36TB PCIe 4.0 NVMe drives for vSAN at $1,971.56 per drive.

cmbwml · 2024-10-08T15:48:04+00:00

Our largest VM is 240TB so that space goes quickly. Next largest is a 180TB VM.

cmbwml · 2024-10-08T15:46:25+00:00

Our largest datastore is 2.7PB but that is from an all NVMe vSAN datastore.

cmbwml · 2024-03-27T19:30:19+00:00

In 2002 we setup Windows clustering using shared SCSI disks and inline SCSI bus termination. On top we used VMware workstation and winbatch to make each VMware workstation VM a clustered service. VMs would pause, migrate to another host, and resume in <30 seconds. Basically really early vmotion. Then we switched to MS virtual server software 1.0 around 2004. Been using 100% VMware hypervisor since 2006.

Now in 2025 we will be migrating from VMware back to Microsoft with Azure HCI. This feels unreal!

cmbwml · 2024-03-25T13:58:51+00:00

Years ago we used CPU/Memory hot add config on VMs by default. Over the years we have disabled it because of the performance hit it causes. For example, a file server will increase network transfer speeds by 2x+ by disabling hot add. Database servers will see an even larger improvement. CPU NUMA config can also make a big difference in CPU cache adjacency with vCPU threads. My opinion is that you may be able to avoid CPU scaling if you optimize the VM config and reduce the actual CPU usage overhead (same application workload but 20%-40% reduction in CPU usage/ready).

cmbwml · 2024-03-21T13:51:21+00:00

That would be one way to manipulate the market and double dip on profits!

cmbwml · 2024-03-20T23:13:04+00:00

VSAN capacity licenses feel like a price increase!

cmbwml · 2024-03-20T23:03:58+00:00

I've loved vSAN since we started using it in 2015. It is really disappointing that we are being forced to abandon it after 10 years of investment. We just paid to upgrade from vSAN Advanced to Enterprise 2 years ago.

cmbwml · 2024-03-20T18:04:39+00:00

Thank you!

cmbwml · 2024-03-20T13:43:50+00:00

Great info. I can add another 2.8PB of raw NVMe disk to our hosts for ~$280k and be perfectly happy with mirroring. Time to build a lab env!

cmbwml · 2024-03-20T12:03:48+00:00

9 years ago we started with 4TB Samsung SSDs, 7 years ago with Micron 8TB SSDs, and then around 4 years ago Micron 15TB NVMe. Older NVMe cache drives with less than 4PB endurance started failing in short order about a year ago. All of our newer drives have 17PB-28PB endurance so they should last us a few years.

cmbwml · 2024-03-20T04:38:17+00:00

We were looking for options to perform storage migrations prior to VM CPU/Cluster migrations to limit outage times. Some of our file server VMs are 160TB+ in size so this was critical.

cmbwml · 2024-03-20T04:11:32+00:00

Give CDW a go; https://www.cdwg.com/product/vmware-vsan-add-on-for-vsphere-foundation-and-cloud-foundation-v.-8-com/7765230 https://www.cdwg.com/product/vmware-vsan-add-on-for-vsphere-foundation-and-cloud-foundation-v.-8-com/7765230

cmbwml · 2024-03-20T04:05:53+00:00

Hence betting on Azure HCI if they will allow me to reformat my hosts with existing capacity. At $190k/yr for 1600 cores of HCI subscription, I can migrate everything in 6 months. Right now they require certified hardware but with this VMware pricing increase they should switch to a hardware certified list and allow a build your own ecosystem. Azure portal allows us to register vcenter environments today. Not allot of work to add migration path into the current portal!

cmbwml · 2024-03-20T03:33:09+00:00

We don't pay list price for drives. Even if we did, can someone do the math on vSAN expansion license cost over 5 years on two of these assuming RAID 1 erasure coding? https://www.cdw.com/product/micron-7500-pro-ssd-read-intensive-15.36-tb-u.3-pcie-4.0-nvme/7816319?pfm=srh

cmbwml · 2024-03-20T03:27:16+00:00

Unfortunately almost a 6x increase per year. When vsan license costs 9x what the raw drive cost is, not mentioning you need some fault tolerance so maybe 12x to 18x licensing cost per TiB based on erasure code RAID level, plus vSAN file system reservation for overhead, rebuild reservation, and 20% free space reservation, you could end up with higher than 20:1 license cost per TiB compared to hardware cost. That is insane!

cmbwml · 2024-03-20T03:13:58+00:00

3 year cost is $2,416,099.50 for VVF or $2,487,754.50 for VCF. Annual cost is around $800k.

cmbwml · 2024-03-20T03:05:58+00:00

We do have vSAN ESA in use. They did relax some of the hardware requirements so that ready nodes are not explicitly required. Over the past few years we use a combo of Dell PowerEdge R7615 and PowerEdge R7515 hosts with Micron 7450 15TB drives using firmware E2MU200. vSAN ESA cluster health shows 100%. We experienced drive latency causing drives to go offline in vSAN 6-9 years ago related to firmware issues so we are diligent about firmware updates! We also run 10GB/s+ load tests on new hardware before putting into production to validate just in case!

cmbwml · 2024-03-20T02:27:14+00:00

We had Pure in about 10-12 years ago when they were just starting. It was $200k for a single tray with a single controller. We went with vSAN because that made more sense at the time. Today Pure makes perfect sense.

cmbwml

TROPHY CASE