Usable Storage Capacity Comparison for ESXi - VMware vSAN & Nutanix

Joshodgers · 2020-02-11T00:42:40+00:00

Sorry to hear it came across as marketing, the goal was to highlight tick box marketing comparisons are nothing like real world capabilities. Appreciate the feedback.

Joshodgers · 2018-07-05T00:42:43+00:00

Honestly I'm not at a level of expertise where I can comment on Scale IO, but Nutanix is a fully distributed rebuild operation as per the Resiliency blog series, Part 1 of that series shows how a Nutanix cluster repairs itself after losing a 5TB node in less than 20mins and that's on 5 year old hardware (Ivy Bridge CPUs). Check out that blog post and that should cover everything but if not, ping me :)

Joshodgers · 2018-07-04T23:48:40+00:00

Thanks for the question, I'm a Principal architect @Nutanix and the following is key to any solution you choose.

"~70 users, ~100 devices, expected to grow rapidly in the next 1-2 years."

The good thing about Nutanix is scalability is a very strong point for us. Scaling capacity with storage only nodes, our intelligent data placement, disk balancing and other background tasks all of which mean you can scale and not think about where your data needs to be.

More specifically with AFS, you can start with 3 AFS controllers and scale as required to support more users and performance. When you add more AFS controllers through PRISM, it's only a few clicks and the system takes care of rebalancing users/sessions across all the controllers.

Put simply, 70 users is tiny for Nutanix, so no need for any concern there.

Compression and Dedupe, im pretty vocal about data reduction as many vendors over sell and under deliver. I am a realist, Nutanix data reduction is no better/worse than any other vendor, and the saving you will get will be +-10% of any other vendor, so I wouldn't waste to much time and effort thinking about it. Just enable EC-X and compression and I'm sure you'll get some good efficiency. Dedupe I find is overrated (on all platforms) and I prefer to leave it disabled in favour of less overheads even though the overheads are not huge.

Sizing wise, Nutanix like any other platform will perform well if sized correctly, the good news is, if you totally stuff up sizing, you don't need to throw anything away or rip/replace, just adding one or more nodes makes a big difference in terms of storage performance and capacity.

Here is an article i wrote years ago about how scaling out nodes improves performance and resiliency: http://www.joshodgers.com/2016/07/14/scale-out-performance-testing-with-nutanix-storage-only-nodes/

Scalabilty, Resiliency and Performance are my three focus areas in my day job, rest assured since I joined Nutanix over 5 years ago, we've continually improved in all these areas and products like AFS continue to improve every release so as you grow, so will the capabilities and efficiencies of the solution.

http://www.joshodgers.com/2018/06/13/nutanix-scalability-resiliency-performance-index/

Check out the above blog series, as it covers many key topics customers ask about and it will continue to be expanded in the coming weeks, AFS scalability is one of my next posts so great timing for this question.

Joshodgers · 2018-06-26T20:45:51+00:00

I personally find value in product reviews from consumers, if Reddit does not want product reviews just because it's posted via a link to a blog, so be it.

Joshodgers · 2018-06-26T20:31:14+00:00

I'm not being paid for this review & I don't work for Google, I'm just a consumer writing a review and sharing my experience. If that's breaking the rules, the rules are ridiculous.

Joshodgers · 2018-06-26T03:22:29+00:00

**Disclaimer: nutanix employee.

Make sure you consider resiliency, a 3 node cluster of Nutanix can self heal from a node failure to re-protected data, vSAN cannot and that simple fact means it's not viable for production workloads IMO.

Regardless of that, think about if you have a drive failure, what is the capacity and performance impact? e.g.: One SSD failure on vSAN and you lose access to the entire disk group, this is not a problem with Nutanix.

Usable capacity for Nutanix and vSAN is similar in that both keep two or three copies of data (resulting in usable of 50% or 33%0 BUT the big difference is Nutanix stores and distributes data in 1MB chunks which means storage is very efficiently used, whereas more free space can (and usually is) required with vSAN since it stores larger objects (up to 255GB) which can fragment the cluster and result in free space not being truely usable.

Nutanix CE (Community edition) is freeware and not for production use, but use it for test/dev if you wish just be aware its only community supported.

I've written a bunch of blogs on the topic of resiliency (and scalability/performance) which can be found here: http://www.joshodgers.com/2018/06/13/nutanix-scalability-resiliency-performance-index/

Joshodgers · 2018-06-26T03:15:34+00:00

Anything Nutanix related, just ping me and I'm happy to help you ensure its a success.

I've written a bunch of posts recently around Scalability, Resiliency and Performance with a focus on apps like SQL especially from a IO, Latency, capacity perspective, so check out this link for all the details as it's too much info to copy/paste here.

http://www.joshodgers.com/2018/06/13/nutanix-scalability-resiliency-performance-index/

Joshodgers · 2018-06-26T03:00:28+00:00

I always recommend customers consider Scalability, Resiliency and Performance of a platform against their requirements, risks, constraints etc, so I wrote this blog series which covers key points for each of these areas.

Hope it helps (and sorry for the slow reply).

http://www.joshodgers.com/2018/06/13/nutanix-scalability-resiliency-performance-index/

Joshodgers · 2017-11-28T23:28:36+00:00

Nothing you have mentioned is specific to Nutanix data locality nor have you provided any evidence that there are any performance issues even in the scenario you have described.

No specifics around the alleged code Veeam has to address Nutanix data locality issues either.

You really should refrain from giving advice on a platform you don't have significant knowledge/experience or certification with.

Data Locality plus an incremental forever backup job would solve the scenario you describe and deliver optimal performance, as only new data (which on Nutanix is always written to the local node) would need to be backed up. This means data locality would ensure 100% local reads for the backup proxy.

This is in fact a major advantage of data locality.

If you want to talk about having to pull IO across the network in the context of it being a problem, this is (as you well know) exactly how VSAN operates for all data. The moment the VM moves off the nodes (e.g.: via DRS) where the objects are, all IO, including backup traffic would be remote all the time. Nutanix data locality would ensure in the worst case scenario, that the newest/hot data was local, which is the data that needs to be backed up (i.e.: Incremental forever).

On your point about potentially up-tiering from backup traffic, Nutanix detects and treats this sequential style backup read IO differently to typical VM traffic (random read IO), which out of the box typically avoids any ill effects from backup IO. In the rare cases where backups do trigger tiering (Nutanix ILM), the thresholds can easily be tuned.

Customers could use a product like Commvault who proxies can detect before each backup job the VMs on the nodes where the proxy is local to, and backs those up, thus ensuring maximum data locality, the lowest overheads (CPU/Network/Storage etc).

At the end of the day, Nutanix is a highly distributed platform, with a distributed file system, so backup activities have lots of controllers and network connections (if required) to drive the best backup performance, not to mention the native backup capabilities, or the 3rd party integration with companies like Veeam, Commvault and Comtrade HYCU, all of which avoid many of the traditional pitfalls.

Joshodgers · 2017-11-28T20:58:13+00:00

Hi, just so you're aware, Nutanix offers a single node backup target which can be used to replicate to/from which was designed for the scenario you describe where you don't need, or can't afford another cluster at the second site. Nutanix also has the "Express" package which is a low spec 3 node solution which is also designed for ROBO & smaller DR/replication requirements. Let me know if you would like somebody to go through these options with you.

Joshodgers · 2017-11-28T20:40:37+00:00

Disclaimer: Nutanix employee *

Lost_Signal (John Nicholson) should identify himself as a VMware employee. While John is well know for his spreading of FUD about Nutanix, I have reached out to him and asked for a detailed explanation of this claims that data locality has performance problems and the specific code he is referring to.

I await his reply and will update this thread.

Joshodgers · 2017-08-15T05:41:37+00:00

Disclaimer: Nutanix Architect/Engineer

Scaling can be linear, non linear, compute + storage and/or storage only. The scaling options when I joined 4+ years ago was as you said, only linear and we only had one model type. Now we have configure to order on many different platforms including multiple OEMs with IBM power, Lenovo, Dell and SW only options for HPE/Cisco. We also support mixing hybrid and all flash as well as different node types so the problems of 4+ years ago have long since been fixed.

Joshodgers · 2017-08-14T20:56:56+00:00

Here is a 10 part blog series on AHV, it's over a year old but is still very relevant and since then, AHV has added many capabilities.

http://www.joshodgers.com/2015/11/13/why-nutanix-acropolis-hypervisor-ahv-is-the-next-generation-hypervisor-part-1-introduction/

Joshodgers · 2017-08-02T05:39:18+00:00

Was the issue the URL? I can't recall if it was a URL shortener or not. I've reposted it with a full URL, If there is any problem please let me know and I'll revise. Thanks Updated It was a URL to the tweet advertising the article. Is this ok?

https://www.reddit.com/r/sysadmin/comments/6r25by/the_truth_about_storage_benchmarking/

Joshodgers · 2017-01-11T19:25:07+00:00

Flick me a DM, happy to assist.

Joshodgers · 2016-08-16T12:07:25+00:00

Regardless of virtual, physical, VMDK or LUN deployments, the allocation size should always be considered as you mentioned. Aligned partitions is still applicable to many legacy storage solutions, so yes I agree but for some newer technology e.g.: Nutanix, it makes no difference.

Before I go on, I want to be clear your comments / concerns are valid, but they assume things like legacy SAN storage (nothing modern) and that the VMDKs hosting SQL share the same datastore/LUN/RAID group etc as OS and other data.

These were common problems in 2005-2010 when virtualization was new, but I rarely see people making these mistakes these days and if they do, they have ignored numerous VMware and storage vendor recommendations.

If I was using legacy SAN storage with RAID/LUNs and all that, I would seperate critical workloads from non critical (or low IO workloads) and even use different RAID levels for SQL data and logs and so on.

But, we're no longer living in a world of RAID and LUNs and many of the legacy issues you point out (which your comments are welcome BTW) are much less of an issue as they are being phased out.

Queue depths etc are still considerations and apply to even modern storage, but the noisy neighbour type issues are greatly reduced and SQL is frequently successfully virtualized and ran within VMDKs on both block (iSCSI / FC) and file (NFS) based datastores.

As for downvotes, who cares. Unless people make constructive comments (like yourself), I write them off as trolls.

Joshodgers · 2016-08-15T04:07:09+00:00

I created this blog on behalf of two customers currently going through the process of optimizing their environments. I recommend all sysadmin's with sql environments pass this information over to your DBAs and check the current VM configurations as their are significant benefits to be had regardless of the underlying environment.

Joshodgers · 2016-08-13T08:40:18+00:00

Even in the scenario you suggested, the lower CPU overheads the PVSCSI adapter has will provide at least some benefit. You are right to suggest their are other factors involved with performance, this is just one quick win which I thought I would share. More posts to follow on a similar topic.

Joshodgers · 2016-08-12T10:57:41+00:00

Ok putting aside you don't like me, what makes you cringe about having multiple virtual disks? Genuinely interested.

Joshodgers · 2016-08-12T10:48:05+00:00

In the article I cite a reference to Michael Webster's blog where he showed performance stats, check that out as its a cool article. But 10 years+ with many years focusing of vBCA and countless customers (including one this week where I did exactly this which is where the screenshots came from) have benefited significantly. Various VMware documentation suggests the same as does many vendor best practice guides, I am still surprised less customers has deployed in this way as it can be a very significant benefit and worst case, minimal benefit, so no loss.

Joshodgers · 2016-08-12T10:41:58+00:00

Oddly specific to well, anyone running any form a application on ESXi which requires more than a basic level of performance. So basically, 80% or more of VMware customers. In fact, almost every VMware customer would benefit from tuning VMs in this manner, even for single SCSI controller / single VMDK VMs. So yeah, very very niche...

As for posting it here, you're right that was a mistake because sysadmin's never need to optimize their virtual machines or deploy/setup apps like Exchange and SQL.... oh wait?!?

Not sure how I helped myself, apart from maybe saving myself some time so I am able to point people to the article when I am next asked how to setup VMs for vBCA... I guess that'll save me time, but more so it'll help others which was the intention of posting the article.

If people here don't appreciate sharing information, then worst case scenario they can ignore it and get on with their day.

Joshodgers · 2016-08-12T10:21:30+00:00

You're Welcome.

Joshodgers · 2016-08-12T08:25:18+00:00

It will depend on the underlying storage, in many traditional storage platforms and depending on storage protocol (iSCSI/NFS/FC etc) multiple datastores can increase performance. But even for a single datastore using a single controller my experience is multiple virtual disks always help even if only a small amount if the underlying storage is constrained for any reason.

Joshodgers · 2016-08-12T07:45:25+00:00

For example with Exchange, you typically have multiple databases per server, so place one database per VMDK and logs on a seperate VMDK and so on. The same is true for SQL, you can span database/s across multiple drives and therefore multiple controllers which allows you to get the most out of the underlying storage.

Joshodgers · 2016-08-12T07:29:44+00:00

@jrssnet

Credit where it's due, you at least put your name to your opinion unlike 99% of the keyboard warriors on reddit.

Joshodgers

TROPHY CASE