As I am still involved in DRBD development, this newsletter edition includes insights into how we test DRBD.
Sometimes, testing begins with a bug in DRBD that embarrasses the developer. Other times, the developer takes pride in understanding a newly discovered issue and finding the mistake in the code. Whichever the case, the developer is motivated to ensure that the error never occurs again. Next, the developer writes a test. Our Python framework lets us easily specify the number of nodes, network interfaces, and storage volumes and automatically generates the necessary DRBD configuration files.
Then, we write the test as a Python program that invokes DRBD commands, expresses events in the environment (e.g., network link failure, disk failure, etc.), and describes the expected DRBD state changes as code.
As of writing this newsletter, we have 106 such tests, and 94 of those are in the CI loop. That means our GitLab instance will run all 94 tests on every proposed code change. Only if all tests pass do we release it for peer review.
In addition, we run all 106 tests in multiple variants (with TCP and RDMA transports, LVM, ZFS, and many different Linux distributions) in 50 iterations every night and plot the success rate. We call those the stability tests. One of our developers, Joel, watches these graphs and points out whether a test’s success rate is declining.
All of the above concerns testing DRBD state handling, error code, and control paths. In addition, we have endurance tests that stress the data path.
Over to LINBIT content, we have two new YouTube videos since I last wrote. ‘Create a Highly Available iSCSI Target With DRBD & Pacemaker’ complements our recently updated tech guide.
We also posted a YouTube short from the recent Red Hat Summit. In the video, RT, one of our business development reps at LINBIT, is challenged to give a 27-second elevator pitch about LINBIT. I think he did a great job. Check it out for yourself by clicking the link above.
Additionally, we have updated two entries on our blog with new information. ‘Controlling Data Replication with Snapshot Shipping Using LINSTOR’ outlines snapshot shipping in a few simple commands, and ‘Abstracting Persistent Storage Across Environments With LINBIT SDS’ discusses how our software helps you remain in control of your data.
Regarding software releases, we recently shared WinDRBD 1.1.14, which contains a minor bug fix for the drbdmeta utility. For external metadata drbdadm, create-md now clears the entire block device, ensuring there are no leftover bits set in the bitmaps.
