This post is intended to help someone learn more about three of the most popular open source virtualization management platforms for large IaaS deployments (in alphabetical order): CloudStack, OpenNebula, and OpenStack. You are reading this article on the LINBIT® blog because LINBIT develops integrations for these platforms that you can use to provide software-defined storage (SDS), high availability (HA), and disaster recovery (DR) capabilities for the storage that you use with these platforms.
There are other popular open source virtualization management platforms, Proxmox VE and Xen Orchestra come to mind, but they are generally better suited for medium or small deployments.
This article will give you a starting point for exploring CloudStack, OpenNebula, and OpenStack by giving an overview of each platform’s history, hypervisor support, system requirements, and installation methods. I will pepper this article with links to each platform’s official documentation at relevant places to provide you with more opportunities to explore. Along the way, you will also learn how LINBIT open source software can integrate with each platform to give you enhanced capabilities for the persistent data storage that you use with these platforms, to back your virtual machine (VM) images, for example. This article will also include links to other resources that can guide you through installation and setup steps.
History of the platforms
Before diving into more technical comparison points, it might first help to give some background history on the three platforms, if you are new to them. If history does not interest you, at least know that all three projects roughly originated at the same time, the late 2000s to early 2010s, and have shown good longevity, adoption, development, and support. There are no newcomers here. This contributes to each platform’s popularity and is a reason why major organizations worldwide trust these platforms to build solutions on.
CloudStack history
The software that would come to be known as CloudStack was first developed in 2008 by a private start-up company. Citrix bought the company in 2011 and released GNU Public License open source versions of the CloudStack project’s code in 2011 and 2012. Later in 2012, Citrix released the code under the Apache Software License (ASLv2) and gave the project to the Apache Software Foundation (ASF) under the foundation’s Incubator project governance. In 2013, Apache CloudStack graduated from Incubator status to a full-fledged “Top-Level Project” under the ASF and the foundation released CloudStack version 4.2.0 later that year. Nearing the end of 2024, Apache CloudStack is at version 4.19.1.
OpenNebula history
OpenNebula was first released by its creators publicly in 2008 as open source software. Its roots however go back a few years earlier where it was an academic research project, sharing a similar history with LINBIT’s own DRBD® software. In 2010, the project creators started the company that would eventually become OpenNebula Systems to provide professional development and support services for OpenNebula software users and enterprises. Nearing the end of 2024, OpenNebula is at version 6.10 with its long-term support (LTS) version at 6.4.6.
OpenStack history
The software that would become the OpenStack platform began life in 2010, when two companies, Rackspace and Anso Labs (then working on a project for NASA), combined their efforts to create a cloud computing platform that could scale massively. The first open source official release happened later that year. In 2012, the OpenStack Foundation (now the Open Infrastructure Foundation) was created to govern the development of OpenStack as an open source software project. OpenStack, as the name suggests, is actually a stack of different but complimentary software that combine to make a platform that you can use to manage compute, storage, virtualization, containerization, networking, and other building blocks that you might need to create IaaS solutions. Nearing the end of 2024, OpenStack is at version 2024.2, code named Dalmatian.
A disclaimer before comparing the platforms
Creating an IaaS offering involves bringing together many areas of technology: compute, data storage, virtualization, networking, user management, and others. For this reason, it is difficult to write about a specific focus or strength of a platform that manages many things. All three platforms have a baseline of excellency, as evidenced by the household name companies and organizations that have used them to build solutions and offerings upon. There are some differences though, either in hypervisor support, platform architecture or implementation, and system requirements, that I will try to overview in the coming sections.
Comparing hypervisor and container support
One of the easiest comparisons to make between these three platforms is their support for different hypervisor and containerization technologies. If you have a strict requirement for using a specific hypervisor or containerization type, this comparison point is a quick way to rule out a particular platform for your intended use case.
Each of the three platforms supports managing KVM hypervisor and VMware hosts and VMs. Beyond these, CloudStack and OpenStack support more hypervisors than OpenNebula. However, before committing to a platform based on its stated hypervisor support, you will likely want to read the fine print in each platform’s documentation. The way that each platform supports managing hypervisors, might be different.
Hypervisor and container support in CloudStack
According to its documentation, “CloudStack supports three hypervisor families, KVM, XenServer/XCP-ng with XAPI, and VMware with vSphere.”
CloudStack nominally supports managing LXC containers run from Red Hat Enterprise Linux (RHEL) 8 or 9 nodes. However, CloudStack documentation disclaims that LXC host containers are not tested to work with the latest CloudStack releases.
CloudStack also supports using Kubernetes for container management. This is done by using a CloudStack Kubernetes integration plugin.
For the latest information about hypervisor and container support within CloudStack, refer to the CloudStack Compatibility Matrix.
Hypervisor and container support in OpenNebula
According to OpenNebula documentation, “The main […] distribution provides full support for KVM, […] as well as LXC system containers.”
What this means is that the OpenNebula project supports the KVM hypervisor and LXC container platforms by default, but you can add support for different hypervisors on your own, within the OpenNebula deployment. Besides the native support for running KVM hosts in OpenNebula, you can add VMware or Xen (or XCP-ng) hypervisor hosts. To use either of these hypervisors in OpenNebula, you will need to install drivers or add-ons.
VMware hypervisor support in OpenNebula is more of a side-by-side affair. With a concurrently running VMware vCenter deployment and proper OpenNebula drivers, you can use OpenNebula to manage VMware hypervisors and guest VMs.
Support for the Xen or XCP-ng hypervisor within OpenNebula depends on a third-party OpenNebula add-on. At one time, this add-on actively lived in this GitHub project but it has not received an update in over eight years and nominally is only compatible with OpenNebula versions 5.0, and a couple of 4.x versions. For context, the current OpenNebula versions in 2024 are 6.x.
With all of that said, if you require a hypervisor technology other than KVM or managing containers other than LXC, you might want to evaluate a different platform.
Hypervisor and container support in OpenStack
OpenStack supports managing the same hypervisors as CloudStack: KVM, Xen (and XAPI-based variants such as XCP-ng), and VMware vSphere. Additionally, OpenStack supports managing Hyper-V and other hypervisors listed in OpenStack documentation. Similarly to CloudStack and OpenNebula, OpenStack also supports managing LXC containers.
For more details, refer to a features support matrix about the capabilities of some of these hypervisors and system architecture that you can run them on within OpenStack.
Comparing system architecture
After exploring hypervisor and container management support within CloudStack, OpenNebula, and OpenStack, another way to compare these platforms is by examining their system architecture and hardware requirements.
CloudStack system architecture
A CloudStack deployment can be very complicated but its documentation does a good job of introducing a newcomer to CloudStack system architecture design in a gentle way, starting with a very simple diagram of a 2-node CloudStack deployment. This basic deployment consists of a management node and a hypervisor node. From that foundation point, the documentation builds to a more sophisticated design diagram that shows the CloudStack concepts of clusters, pods, zones, and regions, all of which you can have multiples of.
Connecting these CloudStack conceptual components in a real deployment involves compute, management, networking, and storage components. The CloudStack documentation has a section that shows how you can bring these components together in a few different deployment architectures, such as a small-scale or large-scale deployment.
LINBIT developed an open source CloudStack plugin that integrates LINSTOR® storage with CloudStack to back CloudStack primary storage. CloudStack primary storage stores the virtual disks for VMs running on hosts in CloudStack. Integrating LINSTOR with CloudStack to back this storage provides benefits such as high availability for live VM migrations, software-defined storage, and disaster recovery capabilities. You can learn more about the LINSTOR CloudStack integration within the chapter in the LINSTOR User Guide.
OpenNebula system architecture
You can find an overview of OpenNebula design architecture within the OpenNebula documentation. The overview also has an OpenNebula architecture diagram, and an OpenNebula components diagram. Compared to either CloudStack or OpenStack, OpenNebula’s system architecture is perhaps easier to comprehend and contributes to it being the least intimidating of the platforms to understand and get started with.
Beyond OpenNebula’s base architecture, you can extend its capabilities by using third-party add-ons. To extend OpenNebula’s storage capabilities, you can use the LINSTOR OpenNebula add-on. The LINSTOR add-on allows you to deploy VMs with highly available images backed by DRBD and attached across the network through one of DRBD’s own high performance transport protocols, either TCP or RDMA based.
You can find more details about integrating LINSTOR with OpenNebula within a dedicated chapter in the LINSTOR User Guide.
OpenStack system architecture
You can learn more about OpenStack system architecture from details within the OpenStack design and architecture documentation including this OpenStack architecture diagram. Elsewhere in OpenStack documentation, you can find a conceptual architecture diagram and a logical architecture diagram.
If these diagrams cause some head spinning, do not be alarmed. OpenStack is a highly sophisticated collection of different components working together to create a platform for building virtualization, IaaS, and cloud solutions by some large enterprises and organizations. It might be the case that you do not need to fully understand or even use all of the many components of OpenStack, to do what you might need to do with it.
As the design documentation describes, OpenStack’s main components are: compute, storage, network, image, control plane, and cloud management. These components have code names within OpenStack. For example, Nova is the OpenStack compute component.
Within the OpenStack storage component, there are components for managing block, file, and object storage. The OpenStack component responsible for provisioning and managing block storage is called Cinder. Other OpenStack services such as the compute component, Nova, can request volumes from Cinder. Cinder will then make a volume accessible to the requesting service.
You can integrate LINSTOR with Cinder by using a volume driver. The volume driver translates calls to the Cinder API to LINSTOR commands. For example, requesting a volume from Cinder will create new resources in LINSTOR, Cinder volume snapshots translate to snapshots in LINSTOR and you can use snapshots and LINSTOR’s snapshot shipping feature as part of a backup or disaster recovery plan.
The LINSTOR User Guide has a chapter devoted to the LINSTOR OpenStack integration where you can learn more about installing and using LINSTOR with OpenStack to provide block storage high availability, software-defined storage, and disaster recovery capabilities for persistent storage.
CloudStack, OpenNebula, and OpenStack system requirements and installation
The follow sections itemize CloudStack, OpenNebula, and OpenStack system requirements and describe how you can install the platforms to evaluate them.
CloudStack system requirements
CloudStack minimum system requirements, as described in CloudStack documentation, are as follows.
CloudStack Management server, database, and storage system requirements
- Operating system:
- Preferred: EL8+ or Ubuntu 22.04 or higher
- 64-bit x86 CPU (more cores results in better performance)
- 4 GB of memory
- 250 GB of local disk (more results in better capability; 500 GB recommended)
- At least one network interface controller (NIC)
- Statically allocated IP address
- Fully qualified domain name as returned by the
hostnamecommand
CloudStack Hypervisor host system requirements
- Must support hardware-assisted virtualization
- Intel Virtualization Technology or AMD Virtualization enabled
- 64-bit x86 CPU (more cores results in better performance)
- Hardware virtualization support required
- 4 GB of memory
- 36 GB of local disk
- At least one NIC
- Latest hotfixes applied to hypervisor software
- When you deploy CloudStack, the hypervisor host must not have any VMs already running
- All hosts within a cluster must be homogeneous. The CPUs must be of the same type, count, and feature flags.
Installing CloudStack
Steps to install CloudStack are outlined in CloudStack documentation. The outlined steps seem deceptively simple. However, even a basic test deployment in a virtual environment can take a few hours to set up, while a production-ready deployment can takes weeks to months to prepare.
A faster way to install CloudStack (with LINBIT software automatically integrated) to evaluate a deployment is by using the LINBIT® CloudStack® HCI Appliance. You can find guided instructions to deploying CloudStack by using this freely downloadable software appliance built on open source software within this LINBIT blog article.
OpenNebula system requirements
OpenNebula system requirements, as described in OpenNebula documentation, are as follows:
Supported operating systems:
- RHEL/AlmaLinux 8 or 9
- Debian 11 or 12
- Ubuntu 22.04 or 24.04
Minimum hardware:
- 4 GiB RAM
- 80 GiB free disk space
Configuration:
- Access to the privileged user (root) account
- A public IP address
- An SSH server running on port 22
- Open ports:
- 22 (SSH)
- 80 (for the Ruby Sunstone GUI)
- 2616 (for the FireEdge GUI)
- 5030 (for the OneGate service)
Installing OpenNebula
To install OpenNebula, you can use the miniONE deployment tool. Using miniONE to install OpenNebula on a single node, without at least one hypervisor node can be a simple but limited way to evaluate OpenNebula, as described in the “Deploy an OpenNebula Front-end On-premises” tutorial.
After installing OpenNebula on a single node, you might want to explore OpenNebula by using a more robust installation. You can use instructions within a LINBIT blog article, “Setting Up LINBIT SDS with OpenNebula for Resilient & Flexible Storage”, to guide you through installing LINSTOR within a 3-node cluster of OpenNebula nodes.
OpenStack system requirements
Because of the many diverse components that make up the OpenStack platform, it is difficult to neatly summarize OpenStack’s system requirements in a single itemized list. You can find an overview of the installation process for various OpenStack components within the “Install OpenStack Services” documentation with some details and hints about system requirements.
One way that you might deploy OpenStack is by using Kolla. According to the Kolla project’s GitHub page:
Kolla has out of the box defaults for a working basic deployment, and also implements complete customization. This model permits operators with minimal experience to deploy OpenStack quickly and as the operator’s experience grows modify the OpenStack configuration to suit the operator’s exact requirements.
You can find some system recommendations and requirements for Kolla deployment method within OpenStack documentation. While these recommendations are somewhat fuzzy, you might find some better defined recommendations within older OpenStack documentation, such as described in some 2019 documentation related to installing OpenStack-Ansible.
Installing OpenStack
To install OpenStack in a cluster of nodes, and for some more reading to potentially spin your head, you can refer to the 2024.2 version of the OpenStack Installation guide. As mentioned earlier, an easier though not as flexible way to install OpenStack is by using Kolla. You can find a Kolla quick start deployment guide within the OpenStack documentation site. Another way to install an OpenStack environment rather than by hand, is by using the set of scripts in the DevStack project. Both the Kolla and DevStack installation methods can be ways that you can relatively quickly deploy OpenStack to evaluate its suitability for your intended purpose.
If you want to explore integrating LINSTOR with OpenStack, for enhanced persistent storage capabilities and features, refer to the installation instructions within the “LINSTOR Volumes in OpenStack” chapter in the LINSTOR User Guide.
Conclusion
Hopefully this overview of three major open source virtualization management platforms, CloudStack, OpenNebula, and OpenStack, has inspired you to explore one or more of them. Household name enterprises and organizations have built solid solutions that continue to run on these platforms. Explore any of these platforms with the confidence that you are not traveling down obscure roads. These roads might be sophisticated and often complex, but you can expect massive scalability and longevity in return.
If you want any help on your journey integrating LINBIT data storage software with any of these platforms, reach out to the LINBIT team. You can also search for answers or ask questions within the LINBIT Community Forum.
