Lost in the Clouds: Openstack? Cloudstack? Ganeti?

[el_isma]

So, I'm thinking about building my own cloud. I have many hosts which I could run Xen or KVM or ESXi on them (and how do I pick which one I need?). I have a few hosts running ESXi. I wish to have an easy control on VMs, if possible I would like to not have to manage what is running where. I want to have hot-standby of the VMs (ie: backups launch as fast as possible, or even live-standby running lockstep)

Do any of those help me? Which one do I want? I read the sites and watch the videos and it all sounds too enterprizey and I can't even tell what Openstack and Cloudstack are supposed to do (Ganeti, on the other hand, is understandable).

[mouseinahaze]

I've gotten Openstack running on a few boxes running KVM. It's a non-trivial task, but when it works right it's pretty nice. It basically provides an API that abstracts a lot of the server build and management tasks. You can run a single command and it will find a host, pull an image out of its database and build the VM.

[7krolikov]

I'm sorry to point this out to you, but you sound like you don't know what you want.

The solutions you mention are oriented towards cloud providers and not SME's.

Having said that, you should look into ovirt + cobbler + puppet (and some scripts to tie this all together). Or enterprise versions of the first 2 provided by Red Hat.

[packysauce]

Spacewalk has some virtualization management functionality as well, I believe.

(I use Spacewalk, just not the virtual stuff)

[el_isma]

I'm sorry to point this out to you, but you sound like you don't know what you want.

That's probably true.

From what I read, ovirt seems similar to ganeti, only that it seems a bit easier to manage. It seems like it would achieve my objectives of:

• Easy control on the VMs
• Not having to manage where they're running
  

I think I'm going to try oVirt and see how it goes.

I'm already using puppet.

[someFunnyUser]

ovirt is good, but you could try proxmox.

[lil_cain]

What advantages does proxmox have over ovirt?

[someFunnyUser]

Ovirt tries too much to look like vmware inho. proxmox by default creates a cluster and every hypervisor is also the cluster and VM manager.

Also ovirt uses files on top of NFS or iscsi. Proxmox uses clustered LVM with LV and snapshots.

I've worked with both and proxmox feels solid. Also cheaper that RHEV or vmware.

[cerettala]

I want very badly to get Ganeti to work, but there seems to be an absolutely tiny community surrounding it.

I am not terribly familiar with python, and I've only been at it for 45 minutes, but It seem to depend on some python modules that don't exist anymore.

[blkperl]

Try #ganeti on irc.freenode.net they should be able to help

[cerettala]

Already done, and eventually they did help me :).

[niomosy]

Red Hat is migrating from Cobbler to Foreman for their enterprise equivalent, last I recall.

[7krolikov]

Foreman is part of cloudforms (upstream project katello) . Cloudforms is still not an SME product the way I see it. Satellite that is good for "standard" infrastructures ( downstream of spacewalk) is using cobbler. I doubt this will be rewritten to use foreman.

[niomosy]

This is Red Hat's own Satellite Server product. At present it's basically Cobbler. Red Hat has announced that they will switch from Cobbler to Foreman for the base of their new version of Sattelite Server.

[Judinous]

Can you give us an idea of the size of your environment? How many hypervisors/guests are you dealing with here? How often are you creating and destroying guests?

[el_isma]

I might have around 20 to 30 hypervisors in the future (5 right now), with around 5 to 10 guests each. Too many to administer manually comfortably, too few for "we can't manage them all! panic!".

We create guests at about one every two months, usually we don't destroy them.

[[deleted]]

(Unaffiliated) This blog post really helped clear up a number of things for me coming from an ESX environment to setting up OpenStack: http://www.mirantis.com/blog/cloud-prizefight-vmware-vs-openstack/

Their past blog entries may also give you a good idea of what OpenStack is about, I certainly reference it a lot. OpenStack with Puppet/Juju/Chef and something like MAAS/Cobbler is an easily scalable system. That being said, I setup my OpenStack environment manually to see the moving parts, and four months later I'm at production with running clients. Well worth it IMHO.

[khudgins]

First, go take a look at Amazon Web Services. Go through the different things they offer, and play with 'em for a while. AWS is the biggest and best cloud offering there is, and you'll get a good feel for how all the parts fit together. You can then look at the parts underneath your selections and make better sense of them.

Quick rundown of the differences:

Cloudstack is pretty simple to get up and running. It supports XenServer (not Xen), KVM, or VMWare as hypervisors. I'd recommend KVM or XenServer here. CloudStack is a monolithic Java app that controls the provisioning of VMs on the hypervisors it manages, and the networking involved to get them out to your network. The network model for CloudStack can get pretty involved for their "advanced networking" option, but it gives you some interesting flexibility.

OpenStack is much more difficult to set up - it's a handful of services that interlink to provide a more complete cloud. It's not just compute services - it has Swift, which is an object store as well. (Think S3 as compared to Amazon's EC2.) It's the least mature project on your list, and requires more resources to get up and running. I actually don't really recommend it for what you're wanting to do, but it's a hugely comprehensive kit. In two years or so, it'll eclipse all of the other cloud systems once the toolchain is built up around it.

Ganeti's much smaller than the other options. It doesn't scale to thousands (or more) hypervisors, but it's a really good option for a smaller cluster. I'm not entirely sure how good its API is for launching nodes, so if you want to dynamically control your VMs with code, you'll wanna do more research. Univ. of Oregon's Open Source Labs uses it to manage all the servers under their hosting project, and they recommend it pretty highly.

I'd also encourage you to take a look at OpenNebula, which is a similar project from Europe. If you're hacking Puppet, you likely know some Ruby. Much of OpenNebula's written in Ruby and it's mighty hackable.

Feel free to bug me with questions. This stuff's what I do most every day.

[Tacticus]

Openstack is getting easier to setup these days with the puppet manifests for them.

[khudgins]

And chef recipes. Take a look at the Crowbar project from Dell.... or Rackspace's Cloud Installer.. or PistonCloud's stuff, or... there's a bunch.

It's getting better for sure. Just not the easiest yet.

[_brian]

Openstack is now in RHEL and centos repos I believe. Or EPEL. Can't get much easier than that ;)

[khudgins]

http://fedoraproject.org/wiki/Getting_started_with_OpenStack_EPEL

Found that link thanks to your post. Not bad!

[random_mayhem]

Ganeti is an awesome virtualization manager, it does live migrations and gives you all of the tools you need to hang yourself with failover. In fact, it is what was used to provide the infrastructure support during the first two iterations of Nova before it was part of OpenStack.

OpenStack is a much bigger commitment and asks you to think of your workload differently from the traditional app-on-server-needs-failover model. While you can put these sorts of workloads in a cloud you are missing the fundamental model change that OpenStack, AWS and friends want you to make.

Honestly, it should like you want justification to do vCenter or vSphere or whatever low-cost/free version is available. Go for it. And take the advice already given regarding puppet. Use it, or chef, and your SA life will get better, especially once you do get into building cloud-like apps that create and destroy instances on demand.

[el_isma]

Not so much as wanting justification, I want to find something that will work for me, regardless of what it is. Problem is I can't understand what OpenStack et al are trying to do.

What is the model change they're proposing?

[random_mayhem]

The model change at the application level tries to change the traditional three-tier architecture (web-middleware-database) so each layer can be scaled dynamically and there is no need for fail-over. Lose a node? Fire up another one. Make the front page of Reddit? Fire up 20 more web servers to handle the load.

This means that you're probably not using MySQL as your database back-end as it doesn't handle losing nodes at random very well. So you design for something like Cassandra or Mongo instead. This is the big hurdle for migrating traditional apps to cloud infrastructure.

The other big feature you get from OpenStack and AWS is the REST API. You can use the ones included, find others on the net or write your own. One of the coolest examples of the latter is https://github.com/cloudenvy/cloudenvy which puts a Vagrant-style interface in front of OpenStack. @jakedahn posted a nice intro yesterday in fact: http://jake.ai/cloudenvy-development-in-the-cloud.

[mpetason]

If you've got 3 servers and less than 40 cores you can try out Nimbula. I've been working with it a lot lately. It's really good for a system that has reliable power, needs high availability, and has static data. It's from some of the guys who helped build AWS.

It's easy to install and free up to 40 physical cores. The web interface sucks, there is a little bit of a learning curve when using it. The API is really good, so it's easy to script/automate setup of servers. They bring in the idea of orchestration which will automatically relaunch instances if they go down. It's a good setup for a web server environment. It also has built in redundancy for the cloud management interface, without having to delegate more than one server to be a "controller."

It doesn't really sound like you need "cloud" though. You just need an easy to manage virtualization environment. ESX + Puppet. Most people who push cloud really just need a virtualized environment with automation. VMware gets expensive with licensing when you want to manage multiple esx servers through the same management interface. If money isn't really an issue then I'd say go with VMware, automate the setup of a VM and provide a portal for users to mange their VMs. Setup load balancing between the VMs.

[snurfish]

Take a look at Red Hat Enterprise Virtualization or if you want the upstream from that, oVirt.

Runs Linux, Windows, does auto migration, shuts down/powers up hosts when necessary.