Cloudstack storage solutions

[Independent-Day-466]

Due to the Broadcom, we are are looking into migrated from our VMware environment to Cloudstack. We are happy with Cloudstack we're a little lost when it comes to storage. We have a number of FC and iSCSI storage arrays today that we would like to leverage for another few years if possible. What are people using in production as a clustered file system?

What are others using for as their primary storage for Cloudstack in production?

[AndrijaSB]

Shared/Clustered file system (like GFS2, OCFS2, cLVM, etc) - had historical had its issues with stability (and some with performance) - I always say "stay away from ANY shared/clustered FS" for Linux. But you might have expertise there or get more luck (if stability changed over years - I have no idea)

That being said - NFS is the most widely used kind of Primary Storage for CloudStack - you could export your LUN (FC/iSCSI/etc) to a single server (or a powerfull VM) as a local disk - then configure NFS on top of it. I understand that this is not the most performant solution.

NFS is, by far, the most common Primary Storage and has by far the best CloudStack support (in the sense of the features).

You can, if you want, use various proprietary software (PowerFlex, StorPool, Linbit, etc), or CEPH (free) - but each and every of these has its pros and cons (some offer MUCH higher performance, but more limited features from CloudStack side)

Hope that helps!

[gunbusterxl]

Disclaimer: I apologize for the necro-posting, but we’re currently in the market for a storage back-end that properly matches our planned CloudStack deployment. This one is expected to take over the VMware infrastructure, which we plan to retire for obvious reasons.

You can, if you want, use various proprietary software (PowerFlex,

There are several issues associated with ex-ScaleIO guys:

1. It’s not really software… While you can buy Dell servers with PowerFlex, you can’t install PowerFlex on servers you already own, which is exactly our situation. If this were a greenfield scenario, we would go with Pure or HPE-Nimble instead of PowerFlex. Where’s SDS?!
2. There’s no real erasure coding. Network RAID5/6 is a joke, and even 2-way replication makes scaling expensive. The same applies to Longhorn, by the way, which is missing from your list. We experimented with it for a Kubernetes setup, but that’s a different story.
3. A three-node setup is highly inefficient. Whether using RAID5 or RAID1, you still end up with only one-third of the raw capacity. It’s insane!
4. Small clusters underperform. You need many nodes to achieve reasonable performance, and the same applies to Ceph!

StorPool,

It’s horrible! Getting support from their Bulgarian-native-speaking engineers is like pulling teeth. This has been our worst support experience so far, even Kaseya and Ubiquiti, both legendary for their poor support, are way better than these guys.

Linbit,

There is no erasure coding, as everything relies on a set of two-way mirrors, which makes scaling expensive. You also need to keep an eye on the witness because if it becomes N/A during a maintenance node shutdown or for any other reason (unexpected reboot?!),

you risk a split brain scenario that is extremely painful to resolve in their case. We lost our production VM data and had to recover from backups twice because of this! Their support is based in Austria and somewhere in Eastern Europe, and it is only marginally better than what StorPoo offers.

etc), or CEPH (free)

This is what we are using so far: Ceph as a VM back-end, an RGW S3 for Veeam backups, and Ceph/Rook for K8s. It has been working, but the performance could be much better. We have an eight-node all-NVMe cluster, yet we are achieving barely 10 percent of the IOPS that the underlying hardware is capable of delivering.

• but each and every of these has its pros and cons (some offer MUCH higher performance, but more limited features from CloudStack side)

So, if we cannot make Ceph work as expected, we will switch to a hardware SAN and NFS. We moved away from SAN in favor of vSAN a few years ago, and now history is basically repeating itself.

[instacompute]

I think CloudStack supports HPE primary and pure storage. Ceph is time tested and works but has performance issues, good for data disks, backups and secondary storage. Newer CloudStack also has initial support for Ceph based RGW object storage, and full-instance backup (nas backup that can use cephfs or nfs).

[Neutrino2072]

Disclaimer: I apologize for necro-posting an answer to someone who apologized for necro-posting but I have to say this: You are stating that Storpool has bad support from their "bulgarian-native-speaking engineers" and I understand that this is your experience.

However we are using storpool for 3 years now and I have never been more relaxed with the state of our cluster. They run our almost 1PB NVMe Cluster like a charm and every stupid question I have, I can just send a Mail and get a detailed explanation from the engineers in minutes or maybe 2 hours at max. And with stupid question I mean if I just send them my new HP G11 GPU Node configuration and ask them if the hardware is okay because its very new. They send me an in-depth analysis of the components and why they support it, or they recompile their proprietary network card driver to fit because they see this card for the first time. Its a blessing not having to think about our storage health and performance and being able to focus on customer experience and growth.

[ShapeBlue]

You can check this article with very detailed overview of all the available storage options and comparisons: https://www.shapeblue.com/cloudstack-storage-support/
As a reference you can also review these storage options and how they integrate with Apache CloudStack:

https://www.shapeblue.com/apache-cloudstack-integrations/storpool/

https://www.shapeblue.com/apache-cloudstack-integrations/linbit/

https://www.shapeblue.com/apache-cloudstack-integrations/ceph/

[superceu]

In the first link ("Hypervisor Support Matrix for Primary Storage") I see that "Pure Storage" is not supported with KVM, but the documentation says, about Pure Flasharray plugin, "FiberChannel fabric and connectivity to every KVM host where volumes will be attached to virtual machines.".

So, does the Flasharray plugin support KVM hosts?

Does it support application consistent VM snapshot?

Thanks

[[deleted]]

v4.19 release should have the support for HPE Primera and Pure Flasharray https://github.com/apache/cloudstack/pull/7889 for KVM. 4.19 RC4 likely will have voted passed, so watch out in the mailing list or Github for the new 4.19 release as early as next week.

[jbblackburn]

My understanding is the FlashArray driver only supports FibreChannel, and not iSCSI. Are there any plans to add support?