DRBD的核心功能是通过Linux内核模块实现的。具体来说，DRBD构成虚拟块设备的驱动程序，因此DRBD位于系统的I/O堆栈的底部附近。因此，DRBD非常灵活和通用，这使得它成为一个复制解决方案，适合为几乎任何应用程序添加高可用性。

DRBD is, by definition and as mandated by the Linux kernel architecture, agnostic of the layers above it. Therefore, it is impossible for DRBD to miraculously add features to upper layers that these do not possess. For example, DRBD cannot auto-detect file system corruption or add active-active clustering capability to file systems like ext3 or XFS.

DRBD includes a set of administration tools which communicate with the kernel module to configure and administer DRBD resources. From top-level to bottom-most these are:

The high-level administration tool of the DRBD-utils program suite. Obtains all DRBD configuration parameters from the configuration file /etc/drbd.conf and acts as a front-end for drbdsetup and drbdmeta. drbdadm has a dry-run mode, invoked with the -d option, that shows which drbdsetup and drbdmeta calls drbdadm would issue without actually calling those commands.

Configures the DRBD module that was loaded into the kernel. All parameters to drbdsetup must be passed on the command line. The separation between drbdadm and drbdsetup allows for maximum flexibility. Most users will rarely need to use drbdsetup directly, if at all.

允许创建、转储、还原和修改DRBD元数据结构。与 drbdsetup 一样，大多数用户很少直接使用 drbdmeta。

在DRBD中，资源 是指特定复制数据集的所有方面的集合术语。其中包括：

This can be any arbitrary, US-ASCII name not containing white space by which the resource is referred to.

任何资源都是一个复制组，由共享同一复制流的多个 卷 之一组成。DRBD确保了资源中所有卷的写保真度。卷的编号以 0 开头，一个资源中最多可以有65535个卷。卷包含复制的数据集和一组供DRBD内部使用的元数据。

在 drbdadm 级别，可以通过资源名称和卷号将资源中的卷寻址为 resource/volume。

这是一个由DRBD管理的虚拟块设备。它的设备主编号为147，次编号从0开始，这是Linux中的惯例。每个DRBD设备对应于资源中的一个卷。关联的块设备通常命名为`/dev/drbdX`，其中`X`是设备的次要编号。udev`通常还会创建包含资源名和卷号的符号链接，如/dev/drbd/by-res/resource/vol-nr`。

A connection is a communication link between two hosts that share a replicated data set. With DRBD 9 each resource can be defined on multiple hosts; with the current versions this requires a full-meshed connection setup between these hosts (that is, each host connected to every other for that resource).

在 drbdadm 级别，连接由资源和连接名（后者默认为对等主机名）寻址，如 resource:connection。

In DRBD, every resource has a role, which may be Primary or Secondary.

当然，资源的角色可以通过manual intervention，调用集群管理应用程序的一些自动算法，或者automatically来更改。将资源角色从次要角色更改为主要角色称为 promotion (升级)，而反向操作称为 demotion (降级)。

DRBD’s hardware and environment requirements and limitations are mentioned below. DRBD can work with just a few KiBs of physical storage and memory, or it can scale up to work with several TiBs of storage and many MiBs of memory.

Since DRBD version 9.2.6, it is possible to encrypt DRBD traffic by using the TLS feature. However, DRBD itself does not contain cryptographic modules. DRBD uses cryptographic modules that are available in the ktls-utils package (used by the tlshd daemon), or that are referenced by the Linux kernel crypto API. In either case, the cryptographic modules that DRBD uses to encrypt traffic will be FIPS compliant, so long as you are using a FIPS mode enabled operating system.

If you have not enabled the TLS feature, then DRBD does not use any cryptographic modules.

In DRBD versions before 9.2.6, it was only possible to use encryption with DRBD if it was implemented in a different block layer, and not by DRBD itself. Linux Unified Key Setup (LUKS) is an example of such an implementation. You can refer to details in the LINSTOR User’s Guide about using LINSTOR as a way that you can layer LUKS below the DRBD layer.

In single-primary mode, a resource is, at any given time, in the primary role on only one cluster member. Since it is guaranteed that only one cluster node manipulates the data at any moment, this mode can be used with any conventional file system (ext3, ext4, XFS, and so on).

在单一主模式下部署DRBD是高可用性（支持故障转移）集群的规范方法。

In dual-primary mode a resource can be in the primary role on two nodes at a time. Since concurrent access to the data is therefore possible, this mode usually requires the use of a shared cluster file system that uses a distributed lock manager. Examples include GFS and OCFS2.

Deploying DRBD in dual-primary mode is the preferred approach for load-balancing clusters which require concurrent data access from two nodes, for example, virtualization environments with a need for live-migration. This mode is disabled by default, and must be enabled explicitly in DRBD’s configuration file.

See 启用双主模式 for information about enabling dual-primary mode for specific resources.

DRBD支持三种不同的复制模式，允许三种程度的复制同步性。

Asynchronous replication protocol. Local write operations on the primary node are considered completed as soon as the local disk write has finished, and the replication packet has been placed in the local TCP send buffer. In case of forced fail-over, data loss may occur. The data on the standby node is consistent after fail-over; however, the most recent updates performed prior to the fail-over could be lost. Protocol A is most often used in long distance replication scenarios. When used in combination with DRBD Proxy it makes an effective disaster recovery solution. See Long-distance Replication through DRBD Proxy, for more information.

Memory synchronous (semi-synchronous) replication protocol. Local write operations on the primary node are considered completed as soon as the local disk write has occurred, and the replication packet has reached the peer node. Normally, no writes are lost in case of forced fail-over. However, in case of simultaneous power failure on both nodes and concurrent, irreversible destruction of the primary’s data store, the most recent writes completed on the primary may be lost.

Synchronous replication protocol. Local write operations on the primary node are considered completed only after both the local and the remote disk write(s) have been confirmed. As a result, loss of a single node is guaranteed not to lead to any data loss. Data loss is, of course, inevitable even with this replication protocol if all nodes (respective of their storage subsystems) are irreversibly destroyed at the same time.

到目前为止，DRBD设置中最常用的复制协议是protocol C。

复制协议的选择影响部署的两个因素： 保护 和 延迟 。相比之下， 吞吐量 在很大程度上独立于所选的复制协议。

有关演示复制协议配置的资源配置示例，请参见配置资源。

使用DRBD 9，可以将数据同时存储在两个以上的集群节点上。

While this has been possible before through stacking, in DRBD 9 this is supported out-of-the-box for (currently) up to 16 nodes. (In practice, using three-, four- or perhaps five-way redundancy through DRBD will make other things the leading cause of downtime.)

与stacking解决方案的主要区别在于性能损失更小，因为只使用了一个级别的数据复制。

在DRBD 9之前，可以使用 drbdadm primary 命令提升资源。在DRBD 9下，当启用 auto promote 选项时，DRBD将自动将资源提升为主角色，并装入或打开其中一个卷进行写入。一旦卸载或关闭所有卷，资源的角色就变回次要角色。

只有在群集状态允许时（即，如果显示 drbdadm primary 命令成功），自动提升才会成功。否则，安装或打开设备将失败，就像在DRBD 9之前一样。

DRBD支持多种网络传输。到目前为止，有两种传输实现可用：TCP和RDMA。每个传输都由对应的内核模块来实现。

DRBD allows configuring multiple paths per connection. The TCP transport uses only one path at a time for a connection, unless you have configured the TCP load balancing feature. The RDMA transport is capable of balancing the network traffic over multiple paths of a single connection. see Configuring multiple paths for more details.

（重新）同步不同于设备复制。当对主角色中的资源的任何写入事件发生复制时，同步与传入的写操作是分离的。否则，它会影响到设备整体。

如果复制链路由于任何原因中断（无论是由于主节点故障、辅助节点故障还是复制链路中断）时，则必须进行同步。同步是有效的，因为DRBD不会按照修改后的块最初写入的顺序，而是按照线性顺序同步，这会产生以下后果：

后台同步正在进行时，服务继续在活动节点上不间断地运行。

如果配置正确，DRBD可以检测复制网络是否拥塞，在这种情况下可以暂停复制。在这种模式下，主节点会”超前”于次节点 即 暂时不同步，但仍在次节点上保留一致的副本。当有更多带宽可用时，复制将自动恢复并进行后台同步。

挂起复制通常在具有可变带宽的链接上启用，例如在数据中心或云实例之间的共享连接上启用广域复制。

有关拥塞策略和挂起复制的详细信息，请参见Configuring Congestion Policies and Suspended Replication。

Online device verification enables users to do a block-by-block data integrity check between nodes in a very efficient manner.

Note that efficient refers to efficient use of network bandwidth here, and to the fact that verification does not break redundancy in any way. Online verification is still a resource-intensive operation, with a noticeable impact on CPU utilization and load average.

It works by one node (the verification source) sequentially calculating a cryptographic digest of every block stored on the lower-level storage device of a particular resource. DRBD then transmits that digest to the peer node(s) (the verification target(s)), where it is checked against a digest of the local copy of the affected block. If the digests do not match, the block is marked out-of-sync and may later be synchronized. Because DRBD transmits just the digests, not the full blocks, online verification uses network bandwidth very efficiently.

The process is termed online verification because it does not require that the DRBD resource being verified is unused at the time of verification. Therefore, though it does carry a slight performance penalty while it is running, online verification does not cause service interruption or system down time — neither during the verification run nor during subsequent synchronization.

It is a common use case to have online verification managed by the local cron daemon, running it, for example, once a week or once a month. See Using Online Device Verification for information about how to enable, invoke, and automate online verification.

DRBD可以选择使用加密消息摘要算法（如MD5、SHA-1或CRC-32C）执行端到端消息完整性检查。

These message digest algorithms are not provided by DRBD, but by the Linux kernel crypto API; DRBD merely uses them. Therefore, DRBD is capable of using any message digest algorithm available in a particular system’s kernel configuration.

With this feature enabled, DRBD generates a message digest of every data block it replicates to the peer, which the peer then uses to verify the integrity of the replication packet. If the replicated block can not be verified against the digest, the connection is dropped and immediately re-established; because of the bitmap the typical result is a retransmission. Therefore, DRBD replication is protected against several error sources, all of which, if unchecked, would potentially lead to data corruption during the replication process:

See Configuring Replication Traffic Integrity Checking for information about how to enable replication traffic integrity checking.

Split brain is a situation where, due to temporary failure of all network links between cluster nodes, and possibly due to intervention by a cluster management software or human error, both nodes switched to the Primary role while disconnected. This is a potentially harmful state, as it implies that modifications to the data might have been made on either node, without having been replicated to the peer. Therefore, it is likely in this situation that two diverging sets of data have been created, which cannot be trivially merged.

DRBD split brain is distinct from cluster split brain, which is the loss of all connectivity between hosts managed by a distributed cluster management application such as Pacemaker. To avoid confusion, this guide uses the following convention:

DRBD允许在检测到脑裂时自动通知管理员（通过电子邮件或其他方式）。有关如何配置此功能的详细信息，请参见Split Brain Notification。

虽然在这种情况下，建议的操作过程是manually resolve处理脑裂，然后消除其根本原因，但在某些情况下，可能需要自动化该过程。DRBD有几种可用的解决方案列举如下：

Whether or not automatic split brain recovery is acceptable depends largely on the individual application. Consider the example of DRBD hosting a database. The discard modifications from host with fewer changes approach may be fine for a web application click-through database. By contrast, it may be totally unacceptable to automatically discard any modifications made to a financial database, requiring manual recovery in any split brain event. Consider your application’s requirements carefully before enabling automatic split brain recovery.

请参阅Automatic Split Brain Recovery Policies以了解有关配置DRBD的自动裂脑恢复策略的详细信息。

当本地块设备（如硬盘驱动器或RAID逻辑磁盘）启用了写缓存时，对这些设备的写入在到达易失性缓存后即被视为完成。控制器制造商通常将此称为回写模式，反之则为直写模式。如果控制器在回写模式下断电，则最后一次写入永远不会提交到磁盘，可能会导致数据丢失。

为了解决这个问题，DRBD使用了磁盘刷新。磁盘刷新是一种写入操作，仅当关联的数据已提交到稳定（非易失性）存储时才完成，也就是说，它已有效地写入磁盘，而不是缓存。DRBD使用磁盘刷新对其复制的数据集和元数据进行写操作。实际上，DRBD在其认为必要的情况下绕过写缓存，如在activity log更新或强制隐式写后依赖项中。这意味着即使在停电的情况下，也会有额外的可靠性。

It is important to understand that DRBD can use disk flushes only when layered on top of backing devices that support them. Most reasonably recent kernels support disk flushes for most SCSI and SATA devices. Linux software RAID (md) supports disk flushes for RAID-1 provided that all component devices support them too. The same is true for device-mapper devices (LVM2, dm-raid, multipath).

带电池备份的写缓存（BBWC）的控制器使用电池备份其易失性存储。在这些设备上，当断电后恢复供电时，控制器会从电池支持的高速缓存中清除所有挂起的写入到磁盘的操作，确保提交到易失性高速缓存的所有写入实际上都转移到稳定存储中。在这些设备上运行DRBD时，可以禁用磁盘刷新，从而提高DRBD的写入性能。有关详细信息，请参见Disabling Backing Device Flushes。

Trim and Discard are two names for the same feature: a request to a storage system, telling it that some data range is not being used anymore^[2] and can be erased internally.
This call originates in Flash-based storages (SSDs, FusionIO cards, and so on), which cannot easily rewrite a sector but instead have to erase and write the (new) data again (incurring some latency cost). For more details, see for example, the wikipedia page.

从8.4.3版开始， DRBD包含了对 Trim/Discard 的支持。您不需要配置或启用任何东西；如果DRBD检测到本地（底层）存储系统允许使用这些命令，它将透明地启用这些命令并传递这些请求。

The effect is that for example, a recent-enough mkfs.ext4 on a multi-TB volume can shorten the initial sync time to a few seconds to minutes – just by telling DRBD (which will relay that information to all connected nodes) that most/all of the storage is now to be seen as invalidated.

稍后连接到该资源的节点将不会看到 Trim/Discard 请求，因此将启动完全重新同步；不过，依赖于内核版本和文件系统，对`fstrim`的调用可能会给出所需的结果。

如果被用作其中一个节点上DRBD的备份块设备的硬盘发生故障，DRBD可以将I/O错误传递到上层（通常是文件系统），也可以屏蔽上层的I/O错误。

If DRBD is configured to pass on I/O errors, any such errors occurring on the lower-level device are transparently passed to upper I/O layers. Therefore, it is left to upper layers to deal with such errors (this may result in a file system being remounted read-only, for example). This strategy does not ensure service continuity, and is therefore not recommended for most users.

如果DRBD被配置为在低层I/O错误时分离, 即 detach ，DRBD将在第一个低层I/O错误发生时自动这样做。当DRBD通过网络从对等节点透明地获取受影响的块时，I/O错误从上层被屏蔽。从那时起，DRBD被称为在无盘模式下操作，并且仅在对等节点上执行所有后续的读写I/O操作。此模式下的性能将降低，但服务不会中断，并且可以在方便的时候经过验证后再移动到对等节点。

See Configuring I/O Error Handling Strategies for information about configuring I/O error handling strategies for DRBD.

DRBD distinguishes between inconsistent and outdated data. Inconsistent data is data that cannot be expected to be accessible and useful in any manner. The prime example for this is data on a node that is currently the target of an ongoing synchronization. Data on such a node is part obsolete, part up to date, and impossible to identify as either. Therefore, for example, if the device holds a file system (as is commonly the case), that file system would be unexpected to mount or even pass an automatic file system check.

Outdated data, by contrast, is data on a secondary node that is consistent, but no longer in sync with the primary node. This would occur in any interruption of the replication link, whether temporary or permanent. Data on an outdated, disconnected secondary node is expected to be clean, but it reflects a state of the peer node some time past. To avoid services using outdated data, DRBD disallows promoting a resource that is in the outdated state.

DRBD has interfaces that allow an external application to outdate a secondary node as soon as a network interruption occurs. DRBD will then refuse to switch the node to the primary role, preventing applications from using the outdated data. A complete implementation of this functionality exists for the Pacemaker cluster management framework (where it uses a communication channel separate from the DRBD replication link). However, the interfaces are generic and may be easily used by any other cluster management application.

每当过时的资源重新建立其复制链接时，其过时标志将自动清除。参阅background synchronization。

When using three-way replication, DRBD adds a third node to an existing 2-node cluster and replicates data to that node, where it can be used for backup and disaster recovery purposes. This type of configuration generally involves Long-distance Replication through DRBD Proxy.

三向复制通过在保存生产数据的现有资源上添加另一个堆叠 stacked 的DRBD资源来工作，如下图所示：

堆叠(stack)资源使用异步复制（DRBD Protocol A）进行复制，而生产数据通常使用同步复制（DRBD Protocol C）。

三路复制可以永久使用，其中第三个节点将使用生产集群中的数据不断更新。或者，它也可以按需使用，其中生产集群通常与备份站点断开连接，并且定期执行站点到站点的同步，例如通过运行夜间cron作业。

DRBD的protocol A是异步的，但是一旦套接字输出缓冲区满了，写入应用程序就会阻塞（请参见 DRBD.conf 手册页中的 sndbuf size 选项）。在这种情况下，写入应用程序必须等到某些写入的数据, 这些数据可能要通过可能很小的带宽网络，而使得链路耗尽。

平均写入带宽受网络链路可用带宽的限制。只有配置好受限的套接字输出缓冲区时，突发写入才能被优雅的处理。

You can mitigate this by DRBD Proxy’s buffering mechanism. DRBD Proxy will place changed data from the DRBD device on the primary node into its buffers. DRBD Proxy’s buffer size is freely configurable, only limited by the address room size and available physical RAM.

Optionally DRBD Proxy can be configured to compress and decompress the data it forwards. Compression and decompression of DRBD’s data packets might slightly increase latency. However, when the bandwidth of the network link is the limiting factor, the gain in shortening transmit time outweighs the added latency of compression and decompression.

Compression and decompression were implemented with multi core SMP systems in mind, and can use multiple CPU cores.

事实上，大多数块I/O数据压缩得非常好，因此对带宽的使用更加有效,这证明了即使使用DRBD Protocol B和C也可以使用DRBD代理。

See 使用DRBD代理 for information about configuring DRBD Proxy.

Truck-based replication, also known as disk shipping, is a means of preseeding a remote site with data to be replicated, by physically shipping storage media to the remote site. This is particularly suited for situations where

In such situations, without truck-based replication, DRBD would require a very long initial device synchronization (on the order of weeks, months, or years). Truck based replication allows shipping a data seed to the remote site, and so drastically reduces the initial synchronization time. See 使用基于卡车的复制 for details on this use case.

DRBD的一个有点特殊的用例是 浮动节点 。在浮动节点设置中，DRBD节点不绑定到特定的命名主机（如在传统配置中），而是能够在多个主机之间浮动。在这种配置中，DRBD通过IP地址而不是主机名来标识对方。

有关管理浮动对等配置的更多信息，请参见Configuring DRBD to Replicate Between Two SAN-backed Pacemaker Clusters。

如果公司的政策规定需要3路冗余，则安装时至少需要3台服务器。

Now, as your storage demands grow, you will encounter the need for additional servers. Rather than having to buy 3 more servers at the same time, you can rebalance your data across a single additional node.

In the figure above you can see the before and after states: from 3 nodes with three 25TiB volumes each (for a net 75TiB), to 4 nodes, with net 100TiB.

DRBD 9使在线实时迁移数据成为可能；请参见Data Rebalancing了解所需的确切步骤。

With the multiple-peer feature of DRBD, several interesting use cases have been added, for example the DRBD client.

The basic idea is that the DRBD back end can consist of three, four, or more nodes (depending on the policy of required redundancy); but, as DRBD 9 can connect more nodes than that. DRBD works then as a storage access protocol in addition to storage replication.

在主 DRBD客户端 上执行的所有写请求都会被发送到配备了存储设备的所有节点。读取请求仅传送到其中一个服务器节点。DRBD客户端 将在所有可用的服务器节点之间平均分配读取请求。

See Permanently Diskless Nodes for more information.

DRBD quorum is a feature that can help you avoid split-brain situations and data divergence in high-availability clusters. By using DRBD quorum, you do not have to resort to using fencing or STONITH solutions, although you can also use these if you want to. DRBD quorum requires at least three nodes but a third node which acts as an arbitrator can be diskless. An arbitrator node does not need to have the same hardware specifications as diskful storage nodes in your cluster. For example, a low-powered single-board computer such as a Raspberry Pi might even be sufficient.

The functional concept behind DRBD quorum is that a cluster node can only modify the DRBD replicated data set if the number of DRBD-running nodes for the data set that the node can communicate with, including the node itself, meets the requirement that you specify when you enable the quorum option. For most configurations, this will be a majority number of nodes. A majority number is more than half of the total number of DRBD-running nodes for the data set in the cluster. By only allowing data writes on a node that has network access to more than half the nodes in a given partition, including the node itself, the DRBD quorum feature helps you avoid a situation that would create a diverging data set.

It is not however the case that you always need two or more running nodes in a 3-node cluster for a DRBD primary node to be able to write to the data set. There is an exception, for example, if you disconnect all secondary nodes gracefully, then DRBD will mark their data as outdated as they leave the cluster. In this way, a single remaining DRBD primary node would “know” that it is safe to continue to write data. This situation could arise, for example, while performing system maintenance on nodes in your cluster. This way, you can maintain nodes in your cluster without causing downtime for your applications and services.

Using DRBD quorum is compatible with running a Pacemaker cluster. Pacemaker gets informed about quorum or loss-of-quorum through the master score of the DRBD resource.

There are different options and behaviors that you can configure related to the DRBD quorum feature, such as how you define quorum in your cluster and what actions DRBD might take when a node loses quorum. Refer to the Configuring Quorum section for information about this.

DRBD runs all its necessary resync operations in parallel so that nodes are reintegrated with up-to-date data as soon as possible. This works well when there is one DRBD resource per backing disk.

However, when DRBD resources share a physical disk (or when a single resource spans multiple volumes), resyncing these resources (or volumes) in parallel results in a nonlinear access pattern. Hard disks perform much better with a linear access pattern. For such cases you can serialize resyncs using the resync-after keyword within a disk section of a DRBD resource configuration file.

See here for an example.

In many scenarios it is useful to combine DRBD with a failover cluster resource manager. DRBD can integrate with a cluster resource manager (CRM) such as DRBD Reactor and its promoter plug-in, or Pacemaker, to create failover clusters.

DRBD Reactor is an open source tool that monitors DRBD events and reacts to them. Its promoter plug-in manages services using systemd unit files or OCF resource agents. Since DRBD Reactor solely relies on DRBD’s cluster communication, no configuration for its own communication is needed.

DRBD Reactor requires that quorum is enabled on the DRBD resources it is monitoring, so a failover cluster must have a minimum of three nodes. A limitation is that it supports ordering of services only for collocated services. One of its advantages is that it makes possible fully automatic recovery of clusters after a temporary network failure. This, together with its simplicity, make it the recommended failover cluster manager. Furthermore, DRBD Reactor is perfectly suitable for deployments on clouds as it needs no STONITH or redundant networks in deployments with three or more nodes (for quorum).

Pacemaker is the longest available open source cluster resource manager for high-availability clusters. It requires its own communication layer (Corosync) and it requires STONITH to deal with various scenarios. STONITH might require dedicated hardware and it can increase the impact radius of a service failure. Pacemaker probably has the most flexible system to express resource location and ordering constraints. However, with this flexibility, setups can become complex.

Finally, there are also proprietary solutions for failover clusters that work with DRBD, such as SIOS LifeKeeper for Linux, HPE Serviceguard for Linux, and Veritas Cluster Server.

Veritas Cluster Server (or Veritas InfoScale Availability) is a commercial alternative to the Pacemaker open source software. In case you need to integrate DRBD resources into a VCS setup please see the README in drbd-utils/scripts/VCS on github.

LINBIT, the DRBD project’s sponsor company, provides binary packages to its commercial support customers. These packages are available through repositories and package manager commands (for example, apt, dnf), and when reasonable through LINBIT’s Docker registry. Packages and images from these sources are considered “official” builds.

这些版本可用于以下发行版：

Refer to the LINBIT Kernel Module Signing for Secure Boot section for information about which specific DRBD kernel modules have signed packages for which distributions.

一些其他发行版的软件包也已经构建好，但是未经充分测试。

LINBIT与DRBD在source release中也同时发布了二进制版本。

Package installation on RPM-based systems (SLES, RHEL, AlmaLinux) is done by simply using dnf install (for new installations) or dnf update (for upgrades).

For DEB-based systems (Debian GNU/Linux, Ubuntu) systems, drbd-utils and drbd-module-`uname -r` packages are installed by using apt install,

LINBIT provides a Docker registry for its commercial support customers. The registry is accessible through the host name ‘drbd.io’.

Before you can pull images, you have to log in to the registry:

成功登录后，可以拉取镜像。要测试登录名和registry，请首先输入以下命令：

Several Linux distributions provide DRBD, including prebuilt binary packages. Support for these builds, if any, is being provided by the associated distribution vendor. Their release cycle may lag behind DRBD source releases.

Releases generated by Git tags on github are snapshots of the Git repository at the given time. You most likely do not want to use these. They might lack things such as generated man pages, the configure script, and other generated files. If you want to build from a tar file, use the ones provided by us.

我们所有的项目都包含标准的构建脚本（例如，Makefile、configure）。维护每个发行版的特定信息（例如，编译文档时损坏的宏等）太麻烦了，而且从历史上看，本节中提供的信息很快就过时了。如果你不知道如何以标准的方式构建软件，请考虑使用LINBIT提供的软件包。

The source tar files for both current and historic DRBD releases are available for download from https://pkg.linbit.com/. Source tar files, by convention, are named drbd-x.y.z.tar.gz, for example, drbd-utils-x.y.z.tar.gz, where x, y and z refer to the major, minor and bug fix release numbers.

DRBD’s compressed source archive is less than half a megabyte in size. After downloading a tar file, you can decompress its contents into your current working directory, by using the tar -xzf command.

For organizational purposes, decompress DRBD into a directory normally used for keeping source code, such as /usr/src or /usr/local/src. The examples in this guide assume /usr/src.

DRBD’s source code is kept in a public Git repository. You can browse this online at https://github.com/LINBIT. The DRBD software consists of these projects:

Source code can be obtained by either cloning Git repositories or downloading release tar files. There are two minor differences between an unpacked source tar file and a Git checkout of the same release:

After cloning the DRBD and related utilities source code repositories to your local host, you can proceed to building DRBD from the source code.

Provided your DRBD build completed successfully, you will be able to install DRBD by issuing the command:

The DRBD userspace management tools (drbdadm, drbdsetup, and drbdmeta) will now be installed in the prefix path that was passed to configure, typically /sbin/.

Note that any kernel upgrade will require you to rebuild and reinstall the DRBD kernel module to match the new kernel.

Some distributions allow to register kernel module source directories, so that rebuilds are done as necessary. See e.g. dkms(8) on Debian.

The DRBD userspace tools, in contrast, need only to be rebuilt and reinstalled when upgrading to a new DRBD version. If at any time you upgrade to a new kernel and new DRBD version, you will need to upgrade both components.

The DRBD build system contains a facility to build RPM packages directly out of the DRBD source tree. For building RPMs, Checking Build Prerequisites applies essentially in the same way as for building and installing with make, except that you also need the RPM build tools, of course.

Also, see Preparing the Kernel Source Tree if you are not building against a running kernel with precompiled headers available.

The build system offers two approaches for building RPMs. The simpler approach is to simply invoke the rpm target in the top-level Makefile:

This approach will auto-generate spec files from pre-defined templates, and then use those spec files to build binary RPM packages.

The make rpm approach generates several RPM packages:

The other, more flexible approach is to have configure generate the spec file, make any changes you deem necessary, and then use the rpmbuild command:

The RPMs will be created wherever your system RPM configuration (or your personal ~/.rpmmacros configuration) dictates.

After you have created these packages, you can install, upgrade, and uninstall them as you would any other RPM package in your system.

Note that any kernel upgrade will require you to generate a new kmod-drbd package to match the new kernel; see also Kernel Application Binary Interface warning for some distributions.

The DRBD userland packages, in contrast, need only be recreated when upgrading to a new DRBD version. If at any time you upgrade to a new kernel and new DRBD version, you will need to upgrade both packages.

The DRBD build system contains a facility to build Debian packages directly out of the DRBD source tree. For building Debian packages, Checking Build Prerequisites applies essentially in the same way as for building and installing with make, except that you of course also need the dpkg-dev package containing the Debian packaging tools, and fakeroot if you want to build DRBD as a non-root user (highly recommended). All DRBD sub-projects (kernel module and drbd-utils) support Debian package building.

Also, see Preparing the Kernel Source Tree if you are not building against a running kernel with precompiled headers available.

The DRBD source tree includes a debian subdirectory containing the required files for Debian packaging. That subdirectory, however, is not included in the DRBD source tar files — instead, you will need to create a Git checkout of a tag associated with a specific DRBD release.

Once you have created your checkout in this fashion, you can issue the following commands to build DRBD Debian packages:

This build process will create the following Debian packages:

After you have created these packages, you can install, upgrade, and uninstall them as you would any other Debian package in your system.

The drbd-utils packages supports Debian’s dpkg-reconfigure facility, which can be used to switch which versions of the man-pages are shown by default (8.3, 8.4, or 9.0).

Building and installing the actual kernel module from the installed module source package is easily accomplished via Debian’s module-assistant facility:

You can also use the shorthand form of the above command:

Note that any kernel upgrade will require you to rebuild the kernel module (with module-assistant, as just described) to match the new kernel. The drbd-utils and drbd-module-source packages, in contrast, only need to be recreated when upgrading to a new DRBD version. If at any time you upgrade to a new kernel and new DRBD version, you will need to upgrade both packages.

Starting from DRBD9, automatic updates of the DRBD kernel module are possible with the help of dkms(8). All that is needed is to install the drbd-dkms Debian package.

DRBD allows you to reconfigure resources while they are operational. To that end,

drbdadm adjust 然后切换到 drbdsetup 对配置进行必要的调整。与往常一样，您可以通过使用 -d （dry-run）选项运行 drbdadm 来查看挂起的 drbdsetup 调用。

Manually switching a resource’s role from secondary to primary (promotion) or vice versa (demotion) is done using one of the following commands:

In single-primary mode (DRBD’s default), any resource can be in the primary role on only one node at any given time while the connection state is Connected. Therefore, issuing drbdadm primary <resource> on one node while the specified resource is still in the primary role on another node will result in an error.

A resource configured to allow dual-primary mode can be switched to the primary role on two nodes; this is, for example, needed for online migration of virtual machines.

If not using a cluster manager and looking to handle failovers manually in a passive/active configuration, the process is as follows.

On the current primary node, stop any applications or services using the DRBD device, unmount the DRBD device, and demote the resource to secondary.

Now on the node you want to make primary promote the resource and mount the device.

If you’re using the auto-promote feature, you don’t need to change the roles (Primary/Secondary) manually; only stopping of the services and unmounting, respectively mounting, is necessary.

Included with drbd-utils versions since 9.26.0, there is a “graceful shutdown” service, drbd-graceful-shutdown.service. This service ensures that the DRBD “last man standing” behavior applies to your cluster when shutting down nodes.

The graceful shutdown service is started automatically by the udev service at the moment the first DRBD device is created. At system shutdown, the graceful shutdown service ensures that all nodes hosting DRBD resources shut down services in a proper sequence so that the last node keeps quorum.

During a normal system shutdown sequence without this intervention, networking often stops before a system unmounts file systems and takes down DRBD devices. Without the graceful shutdown service, to make it so that the last node to shut down keeps DRBD quorum, you would need to manually unmount file systems and down DRBD resources before shutting down nodes.

By virtue of the graceful shutdown service running on a node, when shutting down, a node marks its DRBD resources outdated, stops its running DRBD services, and then networking services are allowed to stop. This shutdown sequence makes it so that the DRBD peer nodes leaving the cluster are able to communicate their outdated status over the network to the last node, and in this way the last node to leave the cluster will keep quorum.

Upgrading DRBD is a fairly simple process. This section contains warnings or important information regarding upgrading to a particular DRBD 9 version from another DRBD 9 version.

If you are upgrading DRBD from 8.4.x to 9.x, refer to the instructions within the Appendix.

双主模式允许资源在多个节点上同时承担主角色。这样做可以是永久性的，也可以是暂时性的。

Normally, one tries to ensure that background synchronization (which makes the data on the synchronization target temporarily inconsistent) completes as quickly as possible. However, it is also necessary to keep background synchronization from hogging all bandwidth otherwise available for foreground replication, which would be detrimental to application performance. Therefore, you must configure the synchronization bandwidth to match your hardware — which you may do in a permanent fashion or on-the-fly.

同样，出于同样的原因，设置高于复制网络上可用带宽的同步速率是没有意义的。

Checksum-based synchronization is not enabled for resources by default. To enable it, add the following lines to your resource configuration in /etc/drbd.conf:

<algorithm> 可能是系统内核配置中内核加密API支持的任何消息摘要算法。通常，您至少可以从 sha1 , md5 和 crc32c 中进行选择。

如果对现有资源进行此更改，请一如既往地将 drbd.conf 同步到对等节点，并在两个节点上运行 drbdadm adjust <resource> 。

在复制带宽高度可变的环境中（这在广域网复制设置中是典型的），复制链路有时可能会变得拥挤。在默认配置中，这将导致主节点上的I/O阻塞，这有时是不可取的。

相反，在这种情况下，您可以将DRBD配置为挂起正在进行的复制，从而使主数据集位于次数据集的 前拉 pull ahead 位置。在这种模式下，DRBD保持复制通道打开 – 它从不切换到断开连接的模式 – 但直到有足够的带宽再次可用时才真正进行复制。

以下示例适用于DRBD代理配置：

通常明智的做法是将 congestion-fill 和 congestion-extents 与 pull-ahead 选项一起设置。

congestion-fill 的理想值是90%

congestion-extents 的理想值是受影响资源配置的 al-extents 的90%。

DRBD’s strategy for handling lower-level I/O errors is determined by the on-io-error option, included in the resource disk configuration in /etc/drbd.conf:

当然，如果要为所有资源定义全局I/O错误处理策略，也可以在 common 部分中设置此值。

The on-io-error option is independent from the on-no-data-accessible option. Some on-io-error strategies involve retrying I/O requests on peer disks. The on-no-data-accessible option setting dictates DRBD behavior when I/O requests are unsuccessful on all disks.

You can set the on-io-error <strategy> to one of the following:

这是默认和推荐的选项。在发生较低级别的I/O错误时，节点将丢弃其备份设备，并继续以无盘模式运行。

This causes DRBD to change the disk status to Inconsistent, mark the failed block as inconsistent in the DRBD quick-sync bitmap, and retry the I/O operation on a peer (secondary node) disk. If the I/O operation succeeds on at least one peer, then a write operation is considered successful. DRBD will retry a read operation until there are no more peer disks to try to read from.

调用定义为本地I/O错误处理程序的命令。这要求在资源的 handlers 部分中定义相应的 local-io-error 命令调用。完全由管理员自行决定使用 local-io-error 调用的命令（或脚本）来实现I/O错误处理。

您可以按照此过程重新配置正在运行的资源的I/O错误处理策略：

Replication traffic integrity checking is not enabled for resources by default. To enable it, add the following lines to your resource configuration in /etc/drbd.conf:

<algorithm> 可能是系统内核配置中内核加密API支持的任何消息摘要算法。通常，您至少可以从 sha1 , md5 和 crc32c 中进行选择。

如果对现有资源进行此更改，请一如既往地将 drbd.conf 同步到对等节点，并在两个节点上运行 drbdadm adjust <resource> 。

When growing (extending) DRBD volumes, you need to grow from bottom to top. First, you need to extend the backing block devices on all nodes. Then you can tell DRBD to use the new space.

Once the DRBD volume is extended, you still need to propagate that change into whatever upper layers might be using the DRBD volume, for example, by extending the file system, or making a VM running with this volume attached aware of the new “disk size”.

Doing this typically means taking the following steps:

Note that different file systems have different capabilities and different sets of management tools. For example XFS can only grow. You point its tool to the active mount point: xfs_growfs /where/you/have/it/mounted.

While the EXT family can both grow (even online), and also shrink (only offline; you have to unmount it first). To resize an ext3 or ext4, you would point the tool not to the mount point, but to the (mounted) block device: resize2fs /dev/drbd#

Obviously use the correct DRBD (as displayed by mount or df -T, while mounted), and not the backing block device. If DRBD is up, that’s not supposed to work anyways (resize2fs: Device or resource busy while trying to open /dev/mapper/VG-LV Couldn’t find valid filesystem superblock.). If you tried to do that offline (with DRBD stopped), you may corrupt DRBD metadata if you ran the file system tools directly against the backing LV or partition. So don’t.

You do the file system resize only once on the Primary, against the active DRBD device. DRBD replicates the changes to the file system structure. That is what you have it for.

Also, don’t use resize2fs on XFS volumes, or XFS tools on EXT, or …​ but the right tool for the file system in use.

resize2fs: Bad magic number in super-block while trying to open /dev/drbd7 is probably just trying to tell you that this is not an EXT file system, and you should try an other tool instead. Maybe xfs_growfs? But as mentioned, that does not take the block device, but the mount point as argument.

When shrinking (reducing) DRBD volumes, you need to shrink from top to bottom. So first verify that no one is using the space you want to cut off. Next, shrink the file system (if your file system supports that). Then tell DRBD to stop using that space, which is not so easy with DRBD internal metadata, because they are by design “at the end” of the backing device.

Once you are sure that DRBD won’t use the space anymore either, you can cut it off from the backing device, for example using lvreduce.

See also Shrinking Online, Shrinking Offline.

在没有BBWC的情况下运行或在电池耗尽的BBWC上运行时，禁用DRBD的刷新 可能会导致数据丢失 ，不应尝试。

DRBD allows you to enable and disable backing device flushes separately for the replicated data set and DRBD’s own metadata. Both of these options are enabled by default. If you want to disable either (or both), you would set this in the disk section for the DRBD configuration file, /etc/drbd.conf.

要禁用复制数据集的磁盘刷新，请在配置中包含以下行：

要禁用DRBD元数据上的磁盘刷新，请包括以下行：

在修改了资源配置（当然，在节点之间同步了 /etc/drbd.conf ）之后，可以通过在两个节点上输入以下命令来启用这些设置：

如果只有一个服务有BBWC^[7]，应将设置移动到主机部分，如下所示：

三个节点的设置包括一个堆叠在另一个设备上的DRBD设备。

在DRBD中，节点可能是永久无盘的。下面是一个配置示例，显示一个具有3个diskful节点（服务器）和一个永久无磁盘节点（客户端）的资源。

For permanently diskless nodes no bitmap slot gets allocated. For such nodes the diskless status is displayed in green color since it is not an error or unexpected state. See The Client Mode for internal details.

Given the (example) policy that data needs to be available on 3 nodes, you need at least 3 servers for your setup.

Now, as your storage demands grow, you will encounter the need for additional servers. Rather than having to buy 3 more servers at the same time, you can rebalance your data across a single additional node.

In the figure above you can see the before and after states: from 3 nodes with three 25TiB volumes each (for a net 75TiB), to 4 nodes, with net 100TiB.

To redistribute the data across your cluster you have to choose a new node, and one where you want to remove this DRBD resource.
Please note that removing the resource from a currently active node (that is, where DRBD is Primary) will involve either migrating the service or running this resource on this node as a DRBD client; it’s easier to choose a node in Secondary role. (Of course, that might not always be possible.)

This section describes how you can configure the DRBD quorum feature to avoid split-brain situations and data divergence in your high-availability clusters.

You enable quorum for a DRBD resource by setting the quorum option within the options section of a DRBD resource configuration file to either majority, all, or a numerical value. By default, DRBD quorum is not enabled and the option is set to off.

An example DRBD resource configuration that enables DRBD quorum is as follows:

You can also enable quorum globally, by setting the quorum option within the options subsection of the common section within the global DRBD configuration file, /etc/drbd.d/global_common.conf. Enabling quorum and any other related options within the global configuration file will affect all DRBD resources, unless the same option is set within a specific DRBD resource configuration file. In that case, the option value set within a DRBD resource configuration file will take precedence.

An example global configuration that enables DRBD quorum is as follows:

For the unlikely case that you want to remove DRBD, here are the necessary steps.

DRBD Proxy进程可以直接位于设置drbd的机器上，也可以放置在不同的专用服务器上。一个DRBD代理实例可以作为分布在多个节点上的多个DRBD设备的代理。

DRBD Proxy is completely transparent to DRBD. Typically you will expect a high number of data packets in flight, therefore the activity log should be reasonably large. Since this may cause longer re-sync runs after the failure of a primary node, it is recommended to enable the DRBD csums-alg setting.

有关DRBD代理的基本原理的更多信息，请参见功能说明Long-distance Replication through DRBD Proxy。

The DRBD Proxy 3 uses several kernel features that are only available since 2.6.26, so running it on older systems (for example, RHEL 5) is not possible. Here we can still provide DRBD Proxy 1 packages, though^[9].

To obtain DRBD Proxy, please contact your LINBIT sales representative. Unless instructed otherwise, please always use the most recent DRBD Proxy release.

要在基于Debian和Debian的系统上安装DRBD Proxy，请使用dpkg工具，如下所示（用DRBD Proxy版本替换版本，用目标体系结构替换体系结构）：

To install DRBD Proxy on RPM based systems (like SLES or RHEL) use the RPM tool as follows (replace version with your DRBD Proxy version, and architecture with your target architecture):

同时安装DRBD管理程序drbdadm，因为需要配置DRBD代理。

This will install the DRBD Proxy binaries as well as an init script which usually goes into /etc/init.d. Please always use the init script to start/stop DRBD proxy since it also configures DRBD Proxy using the drbdadm tool.

When obtaining a license from LINBIT, you will be sent a DRBD Proxy license file which is required to run DRBD Proxy. The file is called drbd-proxy.license, it must be copied into the /etc directory of the target machines, and be owned by the user/group drbdpxy.

DRBD Proxy can be configured using LINSTOR as described in the LINSTOR User’s Guide.

DRBD Proxy can also be configured by editing resource files. It is configured by an additional section called proxy and additional proxy on sections within the host sections.

下面是直接在DRBD节点上运行的代理的DRBD配置示例：

inside IP地址用于DRBD和DRBD代理之间的通信，而 outside IP地址用于代理之间的通信。后一个通道可能必须在防火墙设置中被允许。

drbdadm 提供了 proxy-up 和 proxy-down 子命令，用于配置或删除与命名DRBD资源的本地DRBD代理进程的连接。这些命令由 /etc/init.d/drbdproxy 实现的 start 和 stop 操作使用。

DRBD代理有一个低级的配置工具，称为 drbd-proxy-ctl。在没有任何选项的情况下调用时，它以交互模式运行。

要避免交互模式，直接传递命令，请在命令后面使用 -c 参数。

要显示可用的命令，请使用：

注意传递的命令周围的双引号。

Here is a list of commands; while the first few ones are typically only used indirectly (via drbdadm proxy-up resp. drbdadm proxy-down), the latter ones give various status information.

Since DRBD Proxy version 3 the proxy allows to enable a few specific plugins for the WAN connection. The currently available plugins are zstd, lz4, zlib, and lzma (all software compression).

zstd (Zstandard) is a real-time compression algorithm, providing high compression ratios. It offers a very wide range of compression / speed trade-off, while being backed by a very fast decoder. Compression rates are dependent on “level” parameter which can be arranged between 1 to 22. Over level 20, DRBD Proxy will require more memory.

lz4 是一种非常快速的压缩算法；数据通常被压缩1:2到1:4，可以节省一半到三分之二的带宽。

The zlib plugin uses the GZIP algorithm for compression; it uses a bit more CPU than lz4, but gives a ratio of 1:3 to 1:5.

The lzma plugin uses the liblzma2 library. It can use dictionaries of several hundred MiB; these allow for very efficient delta-compression of repeated data, even for small changes. lzma needs much more CPU and memory, but results in much better compression than zlib — real-world tests with a VM sitting on top of DRBD gave ratios of 1:10 to 1:40. The lzma plugin has to be enabled in your license.

Contact LINBIT to find the best settings for your environment – it depends on the CPU (speed, number of threads), available memory, input and available output bandwidth, and expected I/O spikes. Having a week of sysstat data already available helps in determining the configuration, too.

DRBD Proxy logs events through syslog using the LOG_DAEMON facility. Usually you will find DRBD Proxy events in /var/log/daemon.log.

可以使用以下命令在DRBD Proxy中启用调试模式。

例如，如果代理连接失败，它将记录类似 Rejecting connection because I can’t connect on the other side 的内容。在这种情况下，请检查两个节点上的DRBD是否都在运行（不是独立模式），以及两个代理是否都在运行。还要仔细检查您的配置。

DRBD and the DRBD administrative tool, drbdadm, return POSIX error codes. If you need to get more information about a specific error code number, you can use the following command, provided that Perl is installed in your environment. For example, to get information about error code number 11, enter:

How to deal with hard disk failure depends on the way DRBD is configured to handle disk I/O errors (see Disk Error Handling Strategies), and on the type of metadata configured (see DRBD Metadata).

When DRBD detects that its peer node is down (either by true hardware failure or manual intervention), DRBD changes its connection state from Connected to Connecting and waits for the peer node to reappear. The DRBD resource is then said to operate in disconnected mode. In disconnected mode, the resource and its associated block device are fully usable, and may be promoted and demoted as necessary, but no block modifications are being replicated to the peer node. Instead, DRBD stores which blocks are being modified while disconnected, on a per-peer basis.

DRBD detects split brain at the time connectivity becomes available again and the peer nodes exchange the initial DRBD protocol handshake. If DRBD detects that both nodes are (or were at some point, while disconnected) in the primary role, it immediately tears down the replication connection. The tell-tale sign of this is a message like the following appearing in the system log:

After split brain has been detected, one node will always have the resource in a StandAlone connection state. The other might either also be in the StandAlone state (if both nodes detected the split brain simultaneously), or in Connecting (if the peer tore down the connection before the other node had a chance to detect split brain).

At this point, unless you configured DRBD to automatically recover from split brain, you must manually intervene by selecting one node whose modifications will be discarded (this node is referred to as the split brain victim). This intervention is made with the following commands:

On the other node (the split brain survivor), if its connection state is also StandAlone, you would enter:

如果节点已处于 Connecting 状态，则可以省略此步骤；然后将自动重新连接。

连接后，裂脑受害者立即将其连接状态更改为 SyncTarget，并由其他节点覆盖其修改。

在重新同步完成后，裂脑被视为已解决，节点再次形成完全一致的冗余复制存储系统。

The DRBD administration tool, drbdadm, includes a force secondary option, secondary --force. If DRBD quorum was configured to suspend DRBD resource I/O operations upon loss of quorum, the force secondary option will allow you to gracefully recover the node that lost quorum and reintegrate it with the other nodes.

Requirements:

You can use the command drbdadm secondary --force <all|resource_name> to demote a primary node to secondary, in cases where you are trying to recover a primary node that lost quorum. The argument to this command can be either a single DRBD resource name or all to demote the node to a secondary role for all its DRBD resources.

By using this command on the primary node that lost quorum with suspended I/O operations, all the suspended I/O requests and newly submitted I/O requests will terminate with I/O errors. You can then usually unmount the file system and reconnect the node to the other nodes in your cluster. An edge case is a file system opener that does not do any I/O and just idles around. Such processes need to be removed manually before unmounting will succeed or with the help of external tools such as fuser -k, or the OCF file system resource agent in clustered setups.

Along with the DRBD administration tool’s force secondary option, you can also add the on-suspended-primary-outdated option to a DRBD resource configuration file and set it to the keyword value force-secondary. You will also need to add the resource role conflict (rr-conflict) option to the DRBD resource configuration file’s net section, and set it to retry-connect. This enables DRBD to automatically recover a primary node that loses quorum with suspended I/O operations. With these options configured, when such a node connects to a cluster partition that has a more recent data set, DRBD automatically demotes the primary node that lost quorum and has suspended I/O operations. Additional configurations, for example in a handlers section of the resource configuration file, as well as additional configurations within a cluster manager, may also be necessary to complete a fully automatic recovery setup.

Settings within a DRBD resource configuration file’s options section that cover this scenario could look like this:

DRBD Reactor can be installed from source files found within the project’s GitHub repository. See the instructions there for details and any prerequisites.

Alternatively, LINBIT customers can install DRBD Reactor from prebuilt packages, available from LINBIT’s drbd-9 packages repository.

Once installed, you can verify DRBD Reactor’s version number by using the drbd-reactor --version command.

Because DRBD Reactor has many different uses, it was split into two components: a core component and a plugin component.

Before you can run DRBD Reactor, you must configure it. Global configurations are made within a main TOML configuration file, which should be created here: /etc/drbd-reactor.toml. The file has to be a valid TOML (https://toml.io) file. Plugin configurations should be made within snippet files that can be placed into the default DRBD Reactor snippets directory, /etc/drbd-reactor.d, or into another directory if specified in the main configuration file. An example configuration file can be found in the example directory of the DRBD Reactor GitHub repository.

For documentation purposes only, the example configuration file mentioned above contains example plugin configurations. However, for deployment, plugin configurations should always be made within snippet files.

You can use the DRBD Reactor CLI utility, drbd-reactorctl, to control the DRBD Reactor daemon and its plugins.

With the drbd-reactorctl utility, you can:

For greater control of some of the above actions, there are additional options available. The drbd-reactorctl man page has more details and syntax information.

In this example, you will use DRBD Reactor and the promoter plugin to create a highly available file system mount within a cluster.

Prerequisites:

The DRBD resource, ha-mount, should have the following settings configured in its DRBD resource configuration file:

First, make one of your nodes Primary for the ha-mount resource.

Then create a file system on the DRBD backed device. The ext4 file system is used in this example.

Make the node Secondary because after further configurations, DRBD Reactor and the Promoter plugin will control promoting nodes.

On all nodes that should be able to mount the DRBD backed device, create a systemd unit file:

Next, on the same nodes as the previous step, create a configuration file for the DRBD Reactor promoter plugin:

To apply the configuration, enable and start the DRBD Reactor service on all nodes. If the DRBD Reactor service is already running, reload it instead.

Next, verify which cluster node is in the Primary role for the ha-mount resource and has the backing device mounted.

Test a simple failover situation on the Primary node by using the DRBD Reactor CLI utility to disable the ha-mount configuration.

Run the DRBD Reactor status command again to verify that another node is now in the Primary role and has the file system mounted.

After testing failover, you can enable the configuration on the node you disabled it on earlier.

As a next step, you may want to read the LINSTOR User’s Guide section on creating a highly available LINSTOR cluster. There, DRBD Reactor is used to manage the LINSTOR Controller as a service so that it is highly available within your cluster.

DRBD Reactor’s Prometheus monitoring plugin acts as a Prometheus compatible endpoint for DRBD resources and exposes various DRBD metrics. You can find a list of the available metrics in the documentation folder in the project’s GitHub repository.

Prerequisites:

To enable the Prometheus plugin, create a simple configuration file snippet on all DRBD Reactor nodes that you are monitoring.

Reload the DRBD Reactor service on all nodes that you are monitoring.

Add the following DRBD Reactor monitoring endpoint to your Prometheus configuration file’s scrape_configs section. Replace “node-x” in the targets lines below with either hostnames or IP addresses for your DRBD Reactor monitoring endpoint nodes. Hostnames must be resolvable from your Prometheus monitoring node.

Then, assuming it is already enabled and running, reload the Prometheus service by entering sudo systemctl reload prometheus.service.

Next, you can open your Grafana server’s URL with a web browser. If the Grafana server service is running on the same node as your Prometheus monitoring service, the URL would look like: http://<node_IP_address_or_hostname>:3000.

You can then log into the Grafana server web UI, add a Prometheus data source, and then add or import a Grafana dashboard that uses your Prometheus data source. An example dashboard is available at the Grafana Labs dashboards marketplace. An example dashboard is also available as a downloadable JSON file here, at the DRBD Reactor GitHub project site.

Pacemaker is a sophisticated, feature-rich, and widely deployed cluster resource manager for the Linux platform. It includes a rich set of documentation. To understand this chapter, reading the following documents is highly recommended:

在本节中，您将看到使用自主DRBD存储看起来像本地存储；因此，在Pacemaker集群中集成是通过将挂载点指向DRBD来完成的。

First of all, we will use the auto-promote feature of DRBD, so that DRBD automatically sets itself Primary when needed. This will probably apply to all of your resources, so setting that as a default “yes” in the common section makes sense:

Now you just need to use your storage, for example, through a filesystem:

实际上，所需要的只是一个挂载点（在本例中为`/var/lib/mysql`），DRBD资源在这里挂载。

Provided that Pacemaker has control, it will only allow a single instance of that mount across your cluster.

See also Importing DRBD’s Promotion Scores into the CIB for additional information about ordering constraints for system startup and more.

本节介绍如何在Pacemaker集群中启用DRBD支持的服务。

ocf:linbit:drbd ocf资源代理提供主/从功能，允许Pacemaker启动和监视多个节点上的drbd资源，并根据需要进行升级和降级。但是，您必须了解，DRBD RA在Pacemaker关闭时，以及在为节点启用待机模式时，会断开并分离它管理的所有DRBD资源。

To enable a DRBD-backed configuration for a MySQL database in a Pacemaker CRM cluster with the drbd OCF resource agent, you must create both the necessary resources, and Pacemaker constraints to ensure your service only starts on a previously promoted DRBD resource. You may do so using the crm shell, as outlined in the following example:

在此之后，应启用配置。Pacemaker现在选择一个节点，在该节点上提升DRBD资源，然后在同一节点上启动DRBD支持的资源组。

See also Importing DRBD’s Promotion Scores into the CIB for additional information about location constraints for placing the Master role.

本节概述了在DRBD复制链接中断时防止Pacemaker升级DRBD主/从资源所需的步骤。这使得Pacemaker不会使用过时的数据启动服务，也不会在启动过程中造成不必要的 时间扭曲 。

To enable any resource-level fencing for DRBD, you must add the following lines to your resource configuration:

You will also have to make changes to the handlers section depending on the cluster infrastructure being used.

Corosync-based Pacemaker clusters can use the functionality explained in Resource-level Fencing Using the Cluster Information Base (CIB).

堆叠资源允许DRBD用于多节点集群中的多级冗余，或建立非现场灾难恢复能力。本节描述如何在这种配置中配置DRBD和Pacemaker。

This is a somewhat advanced setup usually employed in split-site configurations. It involves two separate Pacemaker clusters, where each cluster has access to a separate Storage Area Network (SAN). DRBD is then used to replicate data stored on that SAN, across an IP link between sites.

考虑下面的插图来描述这个概念。

每个站点中当前充当DRBD对等点的单个节点中的哪一个没有明确定义 – DRBD对等点被称为是 浮动的,；也就是说，DRBD绑定到没有绑定到特定物理机的虚拟IP地址。

由于这种类型的设置处理共享存储，因此对STONITH进行配置和测试对于它的正常工作至关重要。

Every DRBD resource exposes a promotion score on each node where it is configured. It is a numeric value that might be 0 or positive. The value reflects how desirable it is to promote the resource to master on this particular node. A node that has an UpToDate disk and two UpToDate replicas has a higher score than a node with an UpToDate disk and just one UpToDate replica.

During startup, the promotion score is 0. E.g., before the DRBD device has its backing device attached, or, if quorum is enabled, before quorum is gained. A value of 0 indicates that a promotion request will fail, and is mapped to a pacemaker score that indicates must not run here.

The drbd-attr OCF resource agent imports these promotion scores into node attributes of a Pacemaker cluster. It needs to be configured like this:

These are transient attributes (have a lifetime of reboot in pacemaker speak). That means, after a reboot of the node, or local restart of pacemaker, those attributes will not exist until an instance of drbd-attr is started on that node.

You can inspect the generated attributes with crm_mon -A -1.

These attributed can be used in constraints for services that depend on the DRBD devices, or, when managing DRBD with the ocf:linbit:drbd resource agent, for the Master role of that DRBD instance.

Here is an example location constraint for the example resource from Using DRBD as a Background Service in a Pacemaker Cluster

This means, provided that the attribute is not defined, the fs_mysql file system cannot be mounted here. When the attribute is defined, its value becomes the score of the location constraint.

This can also be used to cause Pacemaker to migrate a service away when DRBD loses a local backing device. Because a failed backing block device causes the promotion score to drop, other nodes with working backing devices will expose higher promotion scores.

The attributes are updated live, independent of the resource-agent’s monitor operation, with a dampening delay of 5 seconds by default.

The resource agent has these optional parameters, see also its man page ocf_linbit_drbd-attr(7):

LVM2是Linux device mapper框架上下文中逻辑卷管理的实现。它实际上与原始LVM实现没有任何共同点，除了名称和缩写。旧的实现（现在追溯命名为 “LVM1″）被认为是过时的；本节不涉及它。

在使用LVM时，必须了解其最基本的概念：

A PV is an underlying block device exclusively managed by LVM. PVs can either be entire hard disks or individual partitions. It is common practice to create a partition table on the hard disk where one partition is dedicated to the use by the Linux LVM.

A VG is the basic administrative unit of the LVM. A VG may include one or more several PVs. Every VG has a unique name. A VG may be extended during runtime by adding additional PVs, or by enlarging an existing PV.

LVs may be created during runtime within VGs and are available to the other parts of the kernel as regular block devices. As such, they may be used to hold a file system, or for any other purpose block devices may be used for. LVs may be resized while they are online, and they may also be moved from one PV to another (provided that the PVs are part of the same VG).

Snapshots are temporary point-in-time copies of LVs. Creating snapshots is an operation that completes almost instantly, even if the original LV (the origin volume) has a size of several hundred GiByte. Usually, a snapshot requires significantly less space than the original LV.

Because an existing LVM logical volume is simply a block device in Linux terms, you can use one as a DRBD backing device. To use an LVM logical volume in this way, you simply create one, and then reference it in a DRBD resource configuration just as you would a physical backing disk.

The following example assumes that an LVM volume group named foo already exists on two nodes of in your LVM-enabled cluster, and that you want to create a DRBD resource named r0 using a logical volume in that volume group.

First, you create the logical volume on both nodes in your cluster:

After this, you will have a block device named /dev/foo/bar on both nodes.

Then, you can simply enter the newly created volumes in your resource configuration:

现在您可以continue to bring your resource up，就像使用非LVM块设备一样。

当DRBD正在同步时，同步目标的状态是不一致的，直到同步完成。如果在这种情况下，SyncSource发生故障（无法修复），这将使您处于一个不幸的位置：具有良好数据的节点已死亡，而幸存的节点具有不良（不一致）数据。

当从LVM逻辑卷上服务DRBD时，可以通过在同步启动时创建自动快照，并在同步成功完成后自动删除同一快照来缓解此问题。

To enable automated snapshotting during resynchronization, add the following lines to your resource configuration:

这两个脚本解析DRBD自动传递给它调用的任何 handler 的 $DRBD_RESOURCE 环境变量。然后， snapshot-resync-target-lvm.sh 脚本为资源包含的任何卷创建一个lvm快照，然后开始同步。如果脚本失败，则同步不会开始。

同步完成后， unsnapshot-resync-target-lvm.sh 脚本将删除不再需要的快照。如果取消快照失败，快照将继续徘徊。

如果您的同步源在任何时候确实出现无法修复的故障，并且您决定还原到对等机上的最新快照，则可以通过输入 lvconvert -M 命令来执行此操作。

To prepare a DRBD resource for use as a Physical Volume, it is necessary to create a PV signature on the DRBD device. To do this, issue one of the following commands on the node where the resource is currently in the primary role:

或

Now, it is necessary to include this device in the list of devices LVM scans for PV signatures. To do this, you must edit the LVM configuration file, normally named /etc/lvm/lvm.conf. Find the line in the devices section that contains the filter keyword and edit it accordingly. If all your PVs are to be stored on DRBD devices, the following is an appropriate filter option:

此筛选器表达式接受在任何DRBD设备上找到的PV签名，同时拒绝（忽略）所有其他签名。

If you want to use stacked resources as LVM PVs, then you will need a more explicit filter configuration. You need to verify that LVM detects PV signatures on stacked resources, while ignoring them on the corresponding lower-level resources and backing devices. This example assumes that your lower-level DRBD resources use device minors 0 through 9, whereas your stacked resources are using device minors from 10 upwards:

此筛选器表达式接受仅在DRBD设备 /dev/drbd10 到 `/dev/drbd19′ 上找到的PV签名，同时拒绝（忽略）所有其他签名。

After modifying the lvm.conf file, you must run the vgscan command so LVM discards its configuration cache and re-scans devices for PV signatures.

You may of course use a different filter configuration to match your particular system configuration. What is important to remember, however, is that you need to:

此外，应通过设置以下内容禁用LVM缓存：

After disabling the LVM cache, remove any stale cache entries by deleting /etc/lvm/cache/.cache.

您还必须在对等节点上重复上述步骤。

When you have configured your new PV, you may proceed to add it to a Volume Group, or create a new Volume Group from it. The DRBD resource must, of course, be in the primary role while doing so.

When you have created your VG, you may start carving Logical Volumes out of it, using the lvcreate command (as with a non-DRBD-backed Volume Group).

有时，您可能希望向卷组中添加新的DRBD支持的物理卷。无论何时执行此操作，都应将新卷添加到现有资源配置中。这将保留复制流并确保VG中所有pv的写入保真度。

扩展资源配置以包括附加卷，如下例所示：

Verify that your DRBD configuration is identical across nodes, then issue:

这将隐式调用 drbdsetup new-minor r0 1，以启用资源 r0 中的新卷 1。将新卷添加到复制流后，可以初始化并将其添加到卷组：

这将把新的PV /dev/drbd/by res/<resource>/1 添加到 <name> VG中，从而在整个VG中保持写保真度。

在对等机之间传输卷组并使相应的逻辑卷可用的过程可以自动化。PacemakerLVM资源代理正是为此目的而设计的。

To put an existing, DRBD-backed volume group under Pacemaker management, run the following commands in the crm shell:

提交此配置后，Pacemaker将自动使 r0 卷组在当前具有DRBD资源主（Master）角色的节点上可用。

The typical high availability use case for DRBD is to use a cluster resource manager (CRM) to handle the promoting and demoting of resources, such as DRBD replicated storage volumes. However, it is possible to use DRBD without a CRM.

You might want to do this in a situation when you know that you always want a particular node to promote a DRBD resource and you know that the peer nodes are never going to take over but are only being replicated to for disaster recovery purposes.

In this case, you can use a couple of systemd unit files to handle DRBD resource promotion and make sure that back-end LVM logical volumes are activated first. You also need to make the DRBD systemd unit file for your DRBD resource a dependency of whatever file system mount might be using the DRBD resource as a backing device.

To set this up, for example, given a hypothetical DRBD resource named webdata and a file system mount point of /var/lib/www, you might enter the following commands:

In this example, the X in drbdX is the volume number of your DRBD backing device for the webdata resource.

The drbd-wait-promotable@<DRBD-resource-name>.service is a systemd unit file that is used to wait for DRBD to connect to its peers and establish access to good data, before DRBD promotes the resource on the node.

Red Hat全局文件系统（GFS）是redhat对并发访问共享存储文件系统的实现。与任何此类文件系统一样，GFS允许多个节点以读/写方式同时访问同一存储设备，而不会造成数据损坏的风险。它通过使用分布式锁管理器（DLM）来管理来自集群成员的并发访问。

GFS was designed, from the outset, for use with conventional shared storage devices. Regardless, it is perfectly possible to use DRBD, in dual-primary mode, as a replicated storage device for GFS. Applications may benefit from reduced read/write latency due to the fact that DRBD normally reads from and writes to local storage, as opposed to the SAN devices GFS is normally configured to run from. Also, of course, DRBD adds an additional physical copy to every GFS filesystem, therefore adding redundancy to the concept.

GFS makes use of a cluster-aware variant of LVM, termed Cluster Logical Volume Manager or CLVM. As such, some parallelism exists between using DRBD as the data storage for GFS, and using DRBD as a Physical Volume for conventional LVM.

GFS file systems are usually tightly integrated with Red Hat’s own cluster management framework, the Red Hat Cluster. This chapter explains the use of DRBD in conjunction with GFS in the Red Hat Cluster context.

GFS, CLVM, and Red Hat Cluster are available in Red Hat Enterprise Linux (RHEL) and distributions derived from it, such as CentOS. Packages built from the same sources are also available in Debian GNU/Linux. This chapter assumes running GFS on a Red Hat Enterprise Linux system.

Since GFS is a shared cluster file system expecting concurrent read/write storage access from all cluster nodes, any DRBD resource to be used for storing a GFS filesystem must be configured in dual-primary mode. Also, it is recommended to use some of DRBD’s features for automatic recovery from split brain. To do all this, include the following lines in the resource configuration:

Once you have added these options to your freshly-configured resource, you may initialize your resource as you normally would. Since the allow-two-primaries option is set to yes for this resource, you will be able to promote the resourceto the primary role on two nodes.

GFS uses CLVM, the cluster-aware version of LVM, to manage block devices to be used by GFS. To use CLVM with DRBD, ensure that your LVM configuration

创建新的DRBD资源并 completed your initial cluster configurations后，必须在GFS集群的两个节点上启用并启动以下系统服务：

To create a GFS filesystem on your dual-primary DRBD resource, you must first initialize it as a Logical Volume for LVM.

Contrary to conventional, non-cluster-aware LVM configurations, the following steps must be completed on only one node due to the cluster-aware nature of CLVM:

CLVM will immediately notify the peer node of these changes; issuing lvs (or lvdisplay) on the peer node will list the newly created logical volume.

Now, you may proceed by creating the actual filesystem:

或者，对于GFS2文件系统：

The -j option in this command refers to the number of journals to keep for GFS. This must be identical to the number of nodes with concurrent Primary role in the GFS cluster; since DRBD does not support more than two Primary nodes the value to set here is always 2.

The -t option, applicable only for GFS2 filesystems, defines the lock table name. This follows the format <cluster>:<name>, where <cluster> must match your cluster name as defined in /etc/cluster/cluster.conf. Therefore, only members of that cluster will be permitted to use the filesystem. By contrast, <name> is an arbitrary file system name unique in the cluster.

创建文件系统后，可以将其添加到 /etc/fstab ：

对于GFS2文件系统，只需更改文件系统类型：

Do not forget to make this change on both cluster nodes.

After this, you may mount your new filesystem by starting the gfs service (on both nodes):

From then on, if you have DRBD configured to start automatically on system startup, before the Pacemaker services and the gfs service, you will be able to use this GFS file system as you would use one that is configured on traditional shared storage.

Oracle集群文件系统（OCFS2）是Oracle公司开发的一个并发访问共享存储文件系统。与它的前身OCFS不同，OCFS是专门设计的，只适用于Oracle数据库有效负载，OCFS2是实现大多数POSIX语义的通用文件系统。OCFS2最常见的用例可以说是Oracle Real Application Cluster（RAC），但是OCFS2也可以用于实现比如负载平衡的NFS集群。

虽然OCFS2最初设计用于传统的共享存储设备，但它同样非常适合部署在dual-Primary DRBD上。从文件系统中读取的应用程序可能会受益于减少的读取延迟，这是因为DRBD从本地存储中读取和写入，而不是在正常情况下运行的SAN设备OCFS2。此外，DRBD通过向每个文件系统映像添加一个额外的副本来增加OCFS2的冗余，而不是仅仅共享一个文件系统映像。

与其他共享群集文件系统（如GFS）一样，OCFS2允许多个节点以读/写模式同时访问同一存储设备，而不会导致数据损坏。它通过使用分布式锁管理器（DLM）来管理来自集群节点的并发访问。DLM本身使用一个虚拟文件系统（ocfs2_dlmfs），它独立于系统上的实际ocfs2文件系统。

OCFS2可以使用一个内在的集群通信层来管理集群成员和文件系统的装载和卸载操作，或者将这些任务推迟到Pacemaker集群基础设施。

OCFS2在SUSE Linux企业服务器（它是主要受支持的共享集群文件系统）、CentOS、Debian GNU/Linux和Ubuntu服务器版本中可用。Oracle还为Red Hat Enterprise Linux（RHEL）提供了软件包。本章假设在SUSE Linux企业服务器系统上运行OCFS2。

Since OCFS2 is a shared cluster file system expecting concurrent read/write storage access from all cluster nodes, any DRBD resource to be used for storing a OCFS2 filesystem must be configured in dual-primary mode. Also, it is recommended to use some of DRBD’s features for automatic recovery from split brain. To do all this, include the following lines in the resource configuration:

不建议在初始配置时将 allow-two-primaries 选项设置为 yes 。您应该在初始资源同步完成后执行此操作。

Once you have added these options to your freshly-configured resource, you may initialize your resource as you normally would. After you set the allow-two-primaries option to yes for this resource, you will be able to promote the resourceto the primary role on both nodes.

Now, use OCFS2’s mkfs implementation to create the file system:

这将在 /dev/drbd0 上创建一个具有两个节点插槽的OCFS2文件系统，并将文件系统标签设置为 ocfs2_drbd0。您可以在 mkfs 调用中指定其他选项；有关详细信息，请参阅 mkfs.ocfs2 系统手册页。

Xen是一个虚拟化框架，最初在剑桥大学（英国）开发，后来由XenSource，Inc.维护（现在是Citrix的一部分）。它包含在大多数Linux发行版的最新版本中，如Debian GNU/Linux（4.0版以后）、SUSE Linux Enterprise Server（10版以后）、Red Hat Enterprise Linux（5版以后）和许多其他发行版。

Xen使用半虚拟化（一种虚拟化方法，涉及虚拟化主机和来宾虚拟机之间的高度协作）和选定的来宾操作系统，与传统的虚拟化解决方案（通常基于硬件仿真）相比，可提高性能。Xen还支持在支持适当虚拟化扩展的cpu上进行完全硬件仿真；用Xen的话说，这就是HVM（ “硬件辅助虚拟机” ）。

Xen supports live migration, which refers to the capability of transferring a running guest operating system from one physical host to another, without interruption.

When a DRBD resource is used as a replicated Virtual Block Device (VBD) for Xen, it serves to make the entire contents of a DomU’s virtual disk available on two servers, which can then be configured for automatic failover. That way, DRBD does not only provide redundancy for Linux servers (as in non-virtual DRBD deployment scenarios), but also for any other operating system that can run virtually under Xen — which, in essence, includes any operating system available on 32- or 64-bit Intel compatible architectures.

对于Xen Domain-0内核，建议加载DRBD模块，并将参数 disable_sendpage 设置为 1。为此，创建（或打开）文件 /etc/modprobe.d/drbd.conf ，并输入以下行：

Configuring a DRBD resource that is to be used as a Virtual Block Device (VBD) for Xen is fairly straightforward — in essence, the typical configuration matches that of a DRBD resource being used for any other purpose. However, if you want to enable live migration for your guest instance, you need to enable dual-primary modefor this resource:

启用dual-primary模式是必要的，因为Xen在启动实时迁移之前，会检查资源配置为在源主机和目标主机上用于迁移的所有VBD上的写访问。

To use a DRBD resource as the virtual block device, you must add a line like the following to your Xen DomU configuration:

This example configuration makes the DRBD resource named resource available to the DomU as /dev/xvda in read/write mode (w).

Of course, you may use multiple DRBD resources with a single DomU. In that case, simply add more entries like the one provided in the example to the disk option, separated by commas.

在这些情况下，必须使用传统的 phy: 设备语法和与资源关联的DRBD设备名，而不是资源名。但是，这要求您在Xen之外管理DRBD状态转换，这是一种比 DRBD 资源类型提供的更不灵活的方法。

Once you have configured your DRBD-backed DomU, you may start it as you would any other DomU:

在此过程中，您配置为VBD的DRBD资源将提升为主要角色，并按预期让Xen访问。

这同样直截了当：

Again, as you would expect, the DRBD resource is returned to the secondary role after the DomU is successfully shut down.

这也是使用常用的Xen工具完成的：

在这种情况下，会自动连续快速地执行几个管理步骤: * 资源将提升为目标主机上的主要角色。 * Live migration of DomU is initiated on the local host. * 迁移到目标主机完成后，资源将在本地降级为次要角色。

两个资源必须在两台主机上以主角色短暂运行，这是首先必须以dual-primary模式配置资源的原因。

Xen本机支持两种虚拟块设备类型：

This device type is used to hand “physical” block devices, available in the host environment, off to a guest DomU in an essentially transparent fashion.

This device type is used to make file-based block device images available to the guest DomU. It works by creating a loop block device from the original image file, and then handing that block device off to the DomU in much the same fashion as the phy device type does.

If a Virtual Block Device configured in the disk option of a DomU configuration uses any prefix other than phy:, file:, or no prefix at all (in which case Xen defaults to using the phy device type), Xen expects to find a helper script named block-prefix in the Xen scripts directory, commonly /etc/xen/scripts.

DRBD发行版为 drbd 设备类型提供了这样一个脚本，名为 /etc/xen/scripts/block-drbd 。该脚本处理必要的DRBD资源状态转换，如本章前面所述。

To fully capitalize on the benefits provided by having a DRBD-backed Xen VBD’s, it is recommended to have Pacemaker manage the associated DomUs as Pacemaker resources.

You may configure a Xen DomU as a Pacemaker resource, and automate resource failover. To do so, use the Xen OCF resource agent. If you are using the drbd Xen device type described in this chapter, you will not need to configure any separate drbd resource for use by the Xen cluster resource. Instead, the block-drbd helper script will do all the necessary resource transitions for you.

When measuring the impact of using DRBD on a system’s I/O throughput, the absolute throughput the system is capable of is of little relevance. What is much more interesting is the relative impact DRBD has on I/O performance. Therefore, it is always necessary to measure I/O throughput both with and without DRBD.

I/O吞吐量估计的工作原理是将大量数据写入块设备，并测量系统完成写入操作所需的时间。这可以使用相当普遍的实用程序 dd 轻松完成，该实用程序的最新版本包括内置的吞吐量估计。

一个简单的基于 dd 的吞吐量基准，假设您有一个名为 test 的临时资源，该资源当前已连接并且在两个节点上都处于次要角色，如下所示：

此测试只需将512MiB的数据写入DRBD设备，然后写入其备份设备进行比较。这两项测试各重复5次，以便进行一些统计平均。相关结果是由 dd 生成的吞吐量测量。

有关一些性能数据，请参阅我们的 Optimizing DRBD Throughput 一章。

延迟测量的目标与吞吐量基准完全不同：在I/O延迟测试中，一个人会写入非常小的数据块（理想情况下是系统可以处理的最小数据块），并观察完成该写入所需的时间。这个过程通常重复几次，以解释正常的统计波动。

与吞吐量测量一样，可以使用无处不在的 dd 实用程序执行I/O延迟测量，尽管设置不同，观察的焦点完全不同。

下面提供了一个简单的基于 dd 的延迟微基准测试，假设您有一个名为 test 的临时资源，该资源当前已连接，并且在两个节点上都处于次要角色：

这个测试用4kiB分别将1000个数据块写入DRBD设备，然后写入其备份设备进行比较。4096字节是Linux系统（除了s390以外的所有体系结构）、现代硬盘和固态硬盘预计要处理的最小块大小。

重要的是要了解由 dd 生成的吞吐量测量与此测试完全无关；重要的是在完成所述1000次写入过程中经过的时间。将此时间除以1000可得出单个块写入的平均延迟。

此外，有关一些典型的性能值，请参阅我们的Optimizing DRBD Latency一章。

DRBD吞吐量受底层I/O子系统（磁盘、控制器和相应缓存）的带宽和复制网络的带宽的影响。

I/O subsystem throughput is determined, largely, by the number and type of storage units (disks, SSDs, other Flash storage [like FusionIO], …​) that can be written to in parallel. A single, reasonably recent, SCSI or SAS disk will typically allow streaming writes of roughly 40MiB/s to the single disk; an SSD will do 300MiB/s; one of the recent Flash storages (NVMe) will be at 1GiB/s. When deployed in a striping configuration, the I/O subsystem will parallelize writes across disks, effectively multiplying a single disk’s throughput by the number of stripes in the configuration. Therefore, the same 40MiB/s disks will allow effective throughput of 120MiB/s in a RAID-0 or RAID-1+0 configuration with three stripes, or 200MiB/s with five stripes; with SSDs, NVMe, or both, you can easily get to 1GiB/sec.

一个带有RAM和BBU的RAID控制器可以加速短峰值（通过缓冲它们），因此太短的基准测试可能也会显示类似于1GiB/s的速度；对于持续的写操作，它的缓冲区只会满负荷运行，然后就没有多大帮助。

Network throughput is usually determined by the amount of traffic present on the network, and on the throughput of any routing/switching infrastructure present. These concerns are, however, largely irrelevant in DRBD replication links which are normally dedicated, back-to-back network connections. Therefore, network throughput may be improved either by switching to a higher-throughput hardware (such as 10 Gigabit Ethernet, or 56GiB InfiniBand), or by using link aggregation over several network links, as one may do using the Linux bonding network driver.