We recently received a support request from a user who believed that DRBD was causing high I/O latency. We quickly determined that DRBD was not to blame. The customer relied on his favorite hardware vendor for hardware raid controllers on his servers.
This particular big-brand hardware raid controller presents the RAID volume as a SCSI device to the Linux kernel. However, the physical storage devices are NVMe drives. I need to mention that the SCSI command queue depth is 256. Meanwhile, the NVMe allows a queue depth of 65636 and an additional 65635 parallel queues, which gives an astronomical high theoretical maximum of outstanding commands.
The other hard fact is that SSDs achieve their astronomical I/O rates only when you give them many I/O requests in parallel so that they can process them in parallel writing/reading to/from different banks. That is the reason all high-performance SSDs come with an NVMe interface.
Consequently, it becomes clear that introducing a virtual SCSI device on the path from the machine to the built-in NVMe drives is a bad idea performance-wise. The SCSI bus limits the number of I/O requests the drives can process in parallel.
With the presence of NVMe drives, I think the time of hardware RAID controllers is over. RAID1 below DRBD does not make sense. Set the replica count on the DRBD level to fit your trade-off between cost and resilience against drive failures.
Be mindful of the trade-off between volume group (thin pool) sizes. They are your impact radius when a physical drive fails, limiting the maximum virtual volume size your LINBIT SDS cluster can provide.
Moving onto LINBIT news, we held the LINBIT Q1 2025 Community Meeting yesterday. We welcomed LINBIT friend, Cloud Systems Architect and Ænix founder, Andrei Kvapil, as our guest speaker. He prepared a presentation titled ‘LINSTOR on Talos Linux: A robust base for Cozystack.’
In addition, LINBIT Developers Joel Colledge and Johannes Khoshnazar-Thoma presented ‘DRBD resync without replication’ and ‘WinDRBD 1.2 news’ respectively. Both presentations provided valuable insight into our software.
Regarding LINBIT content, ‘Introducing DRBD Proxy version 4’ is for LINBIT customers who use Linux distributions that package kernel version 5.1 or newer, demonstrating what’s new in DRBD Proxy version 4.
Additionally, we have released a new Knowledge Base article, which outlines how to select suitable hardware to match workload needs when using LINSTOR and DRBD. The comprehensive article covers a wide range of considerations.
Turning our attention to our latest software updates, we released a new version of the LINSTOR GUI, in which we have fixed the display issue of the SP chart on the dashboard and the display issue of aux properties on the resource overview. You can access v1.9.2 here.
