Meeting Your HA, DR, & SDS Goals with LINBIT Software

LINBIT® software covers DRBD® and LINSTOR®, together with DRBD Reactor and its promoter plugin, LINSTOR Gateway, WinDRBD, and LINBIT SDS for Windows. Teams evaluating high availability (HA), disaster recovery (DR), and software-defined storage (SDS) can use these tools to replicate block storage reliably across Linux and Windows nodes, automate failover with or without specialized fencing hardware, and integrate with platforms from Apache CloudStack to Kubernetes to Proxmox VE, and many others.

📝 NOTE: DRBD refers to DRBD 9, the latest major version of DRBD developed by LINBIT.

At a glance

Priorities differ by use case, but a few requirements come up in nearly every HA, DR, and SDS evaluation. The LINBIT software stack and the support behind it address these.

Keeping data safe and recoverable

DRBD confirms each write on every replica before sending an acknowledgment to the application, and LINSTOR places those replicas on standard LVM or ZFS devices. The DRBD quorum feature prevents replicas from diverging during a network partition, and partial resynchronization supports faster recovery time objectives after a node returns.

Replication protocols

DRBD supports both synchronous and asynchronous replication. In protocol C mode, the default and most commonly used protocol, a write is confirmed to the application only after the data has reached persistent storage on all connected replicas, so a node taking over a resource has an up-to-date copy of data that the application expects. For higher-latency links, protocol A (asynchronous) or protocol B (memory synchronous) acknowledge writes earlier. With protocols A and B, you trade some protection for lower latency.

Direct data access and recovery

LINBIT describes DRBD and LINSTOR as not “hijacking” data. At all times, a storage administrator can access and recover data with common Linux tools independently of whether DRBD or LINSTOR is running, if the underlying physical storage is healthy. DRBD stores data from the start of the backing device and its own metadata at the end, while LINSTOR places that data on standard LVM logical volumes or ZFS data sets. On Windows nodes, LINBIT SDS for Windows places data on Windows Storage Spaces virtual disks, thick or thin-provisioned, and replicates those volumes with WinDRBD. When storage is healthy, an administrator can stop DRBD and mount the lower-level device directly, ideally read-only, without using DRBD or LINSTOR. If a device is failing, standard recovery practices such as taking a best-effort image with a tool such as GNU ddrescue, or other standard and familiar tools, can be used.

Encryption at rest and in transit

LINSTOR can add a LUKS layer to the storage stack it creates, for encryption at rest on an LVM or ZFS backing device. The layer uses dm-crypt and requires cryptsetup on each satellite node. A LINSTOR passphrase encrypts the LUKS passphrases in the LINSTOR database, and the controller needs that passphrase again after each restart, either entered by an administrator or supplied from linstor.toml or an environment variable. Encryption in transit is a separate configuration. DRBD 9.2.6 and later can encrypt replication traffic between nodes by using in-kernel TLS, and SSL/TLS can encrypt traffic between the LINSTOR controller and its satellite nodes. For broader cluster hardening guidance, see security best practices for LINSTOR software-defined storage clusters.

Failover and resynchronization

In a high-availability cluster, the impact of a failing service on a single node is absorbed through resource failover to a healthy node with an up-to-date replica. Upon returning to the cluster, after an administrator addresses the underlying issue behind the node failure, DRBD does a partial resynchronization rather than a full synchronization to bring the returning node up-to-date with data written to the DRBD device on the healthy node. DRBD uses a quick-sync bitmap to record changes that happened since the node failed and left the cluster. This makes for faster cluster recovery times.

DRBD quorum and split-brain prevention

DRBD quorum permits writes only on a node that can communicate with a majority of the nodes replicating a data set. A cluster partition without a majority blocks writes rather than accepting writes that would diverge, which avoids a split-brain scenario without requiring STONITH. To meet the odd node count requirement for quorum, you can use a diskless arbitrator running on low-powered hardware without physical storage as the nth node. DRBD Reactor and its promoter plugin promote a resource and start its associated services on only one node at a time, relying on the state management that DRBD enforces to keep a single DRBD instance promoted and able to write data.

Quorum loss policies

An administrator can configure how a loss of quorum reaches the application layer. The io-error policy, which LINSTOR applies by default, returns an error, and the suspend-io policy blocks I/O until quorum returns. Returning an I/O error suits a service that a cluster manager controls, such as the highly available LINSTOR controller database, where a write error causes the application to exit which lets the promoter service release the resource so that a node with quorum takes over. Suspending I/O suits a virtual machine, because the guest file system remounts read-only after an I/O error and then needs an unmount and remount, whereas suspended I/O resumes when quorum returns.

Control plane independence

The LINSTOR control plane is separate from the DRBD data plane. If the controller node stops, storage on the satellite nodes remains accessible and continues to serve and replicate data, though management operations pause until a controller is available again. You can make the controller highly available with DRBD Reactor and the promoter plugin, or you can let the LINBIT deployment tool do this for you.

Performance and scale

DRBD adds little demand on resources beyond the backing device and network beneath it, so throughput and latency are determined by the underlying physical storage devices, the network round-trip time, and the DRBD replication protocol in use. Scale comes from spreading many independently replicated volumes across nodes rather than from combining the capacity of many nodes.

Throughput and latency

DRBD sits low in the Linux I/O stack and LINBIT prioritizes high throughput and low latency, limiting system resource overhead to the underlying backing device and network. DRBD performs as fast as its slowest component, so throughput and latency are typically limited by physical storage devices and the network. LINBIT has published independent performance testing of DRBD by E4. With synchronous replication in protocol C, the network round-trip time becomes the floor for write latency, which is why LINBIT recommends low-latency links for local high availability and asynchronous protocol A for remote sites. For latency-sensitive workloads, tuning options include an RDMA transport and pinning DRBD to specific CPUs with a CPU mask. LINBIT also documents hardware considerations for DRBD and LINSTOR.

Caching and tiered storage

LINSTOR can also place a cache layer in front of the backing device, so that SSD or NVMe devices accelerate HDD-backed storage. LINSTOR works with dm-cache, dm-writecache, and bcache. A cache layer draws from a separate LINSTOR storage pool backed by the faster devices. Capacity stays on the slower media while frequently accessed or recently written blocks sit on the faster media, and the volume presented to the application does not change.

Tiered storage combines a similar mix of media in a different way. Rather than putting a slower device in front of a faster one, LINSTOR keeps a separate storage pool on each class of device, for example, one storage pool backed by NVMe drives and another backed by spinning disks. A LINSTOR resource group acts as a template for the resources created from it and selects the storage pool that backs these resources. An administrator can then assign a volume to a tier when provisioning it. This fits workloads with known requirements, such as keeping active data on fast media and backups or archives on slower and less expensive media. It can also give service providers a basis for offering different storage plans to customers.

Some LINSTOR integrations use tiered storage directly. In Apache CloudStack, each LINSTOR resource group becomes a separate CloudStack primary storage entry, and a storage tag links that primary storage to a CloudStack disk offering. A user then selects a tier by choosing a disk offering when creating a volume, and the LINSTOR CloudStack plugin creates the resource from the resource group that backs it.

Scaling model

DRBD replication mirrors a data set across nodes rather than striping it, so a replicated volume is limited by the capacity of the nodes that hold it rather than by the sum of all nodes. A DRBD resource is replicated across two nodes by default, with a maximum recommended replica count of three, and a technical maximum of 32 replicas per resource. LINSTOR scales by placing many independent replicated volumes across a larger set of nodes, and diskless nodes can access a volume without holding a local copy. This design favors predictable performance and straightforward recovery over combining the storage capacity of many nodes into a single large pool.

Protecting data across sites

Multisite protection combines asynchronous replication over a WAN, synchronous replication where latency between sites allows it, and snapshot shipping for point-in-time backup copies. Link latency and link bandwidth together determine which replication mode best fits, while snapshots cover the disaster recovery cases that using replication alone does not.

WAN replication with DRBD Proxy

For local high availability, DRBD uses synchronous replication (protocol C), and for a remote site it uses protocol A (asynchronous) so that the WAN round-trip time stays out of the write acknowledgment path. Protocol A removes link latency from that path, but the average write rate remains limited by the bandwidth of the link, and the writing application blocks as soon as the socket output buffer fills. Write bursts pass without slowing the application only when they fit within that buffer, so sizing the link to the sustained write rate of the workload is as important as choosing an asynchronous protocol.

DRBD Proxy buffers the replication stream in memory to absorb variations in bandwidth and latency on the link, and it can compress the stream to reduce bandwidth use between sites. The buffer size is configurable and limited only by available memory on the node, which is what lets it absorb bursts that a socket buffer cannot. DRBD Proxy enables the on-congestion pull-ahead policy, which suspends replication when the link congests and lets the primary node pull ahead of its peers. This policy records changes for a later resynchronization, rather than blocking the application, so the remote replica falls behind the primary node until bandwidth allows it to catch up. A common deployment architecture adds a third, off-site node in a stacked configuration on top of a locally replicated resource. DRBD Proxy carries a per-node license and is one of the few LINBIT components that is not open source.

Stretched clusters and backup strategy

For campus or metro networks where inter-site links offer near-LAN latency, a stretched cluster provides synchronous multisite replication as a distinct alternative to asynchronous WAN replication, and the same approach extends to platforms such as Proxmox VE. However, real-time replication is complementary to, not a replacement for, snapshots and backups. Use replication for high availability and snapshots and backups for disaster recovery.

Snapshot shipping

LINSTOR snapshot shipping addresses the disaster recovery side. It takes point-in-time snapshots of thin-provisioned LVM or ZFS volumes and ships them, in full or as deltas, to a remote defined in LINSTOR. The remote is either another LINSTOR cluster or S3-compatible object storage such as Amazon S3, MinIO, RustFS, or Storj. You can run snapshot shipping on-demand or on a schedule. Snapshot shipping is asynchronous and point-in-time, so you can use it to recover data from logical corruption that DRBD replication copies to every replica.

Operating without the complexity

LINBIT covers cluster creation, provisioning, and observation with its own tools, so basic operation needs no third-party components. Deployment has a graphical installer tool option which uses Ansible behind the scenes, or you can use the LINBIT Ansible collections directly. Management uses LINSTOR with a web or terminal interface, and monitoring uses Prometheus and Grafana.

Deployment and management tools

Deployment and day-to-day operation are handled by separate tools.

Metrics and monitoring

Monitoring integrates with an existing Prometheus and Grafana deployment. The LINSTOR controller publishes node, storage pool, and resource metrics on its REST port, and DRBD Reactor publishes per-device metrics such as out-of-sync bytes, resource roles, and connection states. LINBIT publishes Grafana dashboards for both endpoints. In Kubernetes, the Prometheus Operator and the kube-prometheus-stack collection scrape the same metrics.

Running everywhere you run

LINBIT maintains integrations for virtualization, container, and cloud platforms, and it supports Windows Server alongside Linux. LINBIT support subscribers install regularly updated, precompiled, signed kernel modules, including on hosts that enforce Secure Boot.

Platform integrations

LINSTOR provides LINBIT-maintained integrations for Apache CloudStack, Docker, Kubernetes, OpenNebula, OpenShift, OpenStack, oVirt and OLVM, Proxmox VE, and XCP-ng and Xen Orchestra. The Kubernetes integration includes a CSI driver with support for RWO and RWX volumes, a LINSTOR Operator for simple deployment and features such as QoS. LINSTOR in Kubernetes is downstream of the open source, CNCF project also developed by LINBIT, Piraeus. The XCP-ng and Xen Orchestra integration (XOSTOR) is developed and supported by the LINBIT partner, Vates.

Windows Server and Hyper-V

WinDRBD is a native Windows kernel driver, a port of DRBD, that presents replicated volumes as regular SCSI disks so that applications such as Microsoft SQL Server and Hyper-V can use them. LINBIT SDS for Windows combines WinDRBD with the Windows ports of LINSTOR and DRBD Reactor to deliver replicated, highly available storage and automated failover, including live migration of Hyper-V virtual machines. LINBIT SDS for Windows uses Windows Storage Spaces, thick or thin-provisioned, as a storage provider, in the same role that LVM and ZFS fill on Linux nodes. It runs on Windows Server 2016–2025 for production deployments, and on Windows 10 and 11 for testing. LINBIT SDS for Windows is an early production stage product and LINBIT recommends engaging the LINBIT support team for deployments. WinDRBD, the bedrock of LINBIT SDS for Windows, has been relied upon in production deployments for years.

NVMe over Fabrics

LINSTOR Gateway can create and manage highly available NVMe/TCP, iSCSI, and NFS targets by using the NVMe target implementation built into the Linux kernel. NVMe/TCP is the default transport, and RDMA transports such as RoCE are also possible at the kernel level. NVMe over Fabrics is the client access protocol that presents LINSTOR volumes to initiators. DRBD replication between nodes uses the DRBD protocol over TCP or RDMA, a path independent from client access. For a first iSCSI target, see getting started with LINSTOR Gateway.

Kernel module maintenance and Secure Boot

DRBD has been in Linux since 2.6.33, currently at DRBD version 8.4. LINBIT also maintains an out-of-tree module, DRBD 9, with the newest features. Subscribers can install precompiled kernel module packages from the LINBIT certified repositories rather than compiling on every kernel update, covering many common Linux distributions. On RPM-based systems, LINBIT ships its own Secure Boot signing key with the module package, so that after key enrollment the signed modules load under Secure Boot without generating a per-host key. On Debian and Ubuntu, the drbd-dkms package builds and signs the module for each host through the distribution shim-signed and Machine Owner Key mechanism. The LINBIT team is working to bring DRBD 9 to the mainline Linux kernel.

Licensing, support, and getting started

DRBD and LINSTOR are open source and free to use with community support, and subscriptions add certified packages and professional support up to 24×7. An evaluation can start from public repositories, a minikube proof of concept using Piraeus, or other free and open entry points, before any commercial engagement.

Licensing and support

DRBD is GPL and open source, and has been part of the mainline Linux kernel since version 2.6.33. LINSTOR is published on GitHub under an open source license. Both are free to use with community support. LINBIT subscriptions add signed and certified prebuilt packages, repository access, and tiered professional support up to 24×7. Piraeus is the open source upstream distribution of LINSTOR for Kubernetes. DRBD Proxy is the exception. It is not published under an open source license.

Getting started

To get started with the open source software:

• Install from the Ubuntu PPA, the public Debian repository at packages.linbit.com, or the public Proxmox VE repository, which carries the full stack of DRBD, LINSTOR, and the Proxmox plugin.
• Deploy Piraeus Datastore on minikube for a Kubernetes proof of concept ready in ~20–30 min, following the step-by-step blog post.
• Bring existing, manually created DRBD resources under LINSTOR management without starting over, by using the documented migration path.
• Set up a home lab to evaluate the stack, following Homelabbing LINBIT storage solutions.

For production use, an enterprise evaluation, or access to certified customer repositories and expert support, contact the LINBIT team.

Further reading

For the foundational documentation and product overviews behind this article:

• DRBD 9.0 User Guide and the LINSTOR User Guide, the reference documentation for the replication and storage-management layers.
• Exploring LINSTOR features for SDS, HA, and DR, an overview of the capabilities in this article.
• The LINSTOR product page and the LINBIT disaster recovery page.