The Linux kernel’s interface towards device drivers is not set in stone. It evolves with the evolvement of Linux. Sometimes driven by hardware improvements, sometimes driven by general evolution of the code base. In tree drivers this is not a big issue since they get changed as the interfaces are modified, therefore are called “tree wide changes”.
DRBD® development happens out-of-tree before code gets sent upstream. We at LINBIT® need to track these tree wide changes and follow them in the DRBD device driver. But, not all of our users are using the same kernel.
Some are strictly sticking to the kernel that was shipped with the distribution, running a kernel that is years behind Linus’ version of Linux.
That creates a problem. DRBD should be compatible with both: Years old, carefully maintained “Vendor kernels”, and the latest and greatest Linus kernel.
In order to do that we have a kernel compatibility layer in DRBD. It contains two main parts:
- Detecting the capabilities of the kernel we want to build against. The background is that a “Vendor kernel” is not just a random old Linus kernel. It starts as some release of the vanilla kernel, and then the vendors cherry-pick upstream changes and fixes they deem to be relevant to their vendor kernel.
- The compatibility layer itself. Up to DRBD-9.0.19 this was a huge file containing many #IFDEFs. It became a maintenance nightmare. It was hard to extend, hard to understand and debug, and hard to remove old compat. Everything was ugly.
Coccinelle is French for Ladybug
The Coccinelle project from INRIA contains a tool to apply semantic patches to the source code, or expand the semantic patches to conventional patches. A few of us DRBD developers practically fell into love with that tool. It allows us to express how some code that is compatible with the upstream kernel needs to be changed in order to be compatible with some older versions of the kernel.
This allows our DRBD upstream code to be in a form that has clean Linux upstream code, containing no compatibility hacks.
This allows us to automatically transform DRBD to be compatible with random old kernels or vendor kernels. The result, after the transformation, is clean C code without confusing macros and #IFs. It is wonderful.
The new kernel compatibility mechanism:
- Detect kernel capabilities (as before)
- Create a compat patch using spatch (from Coccinelle)
- Apply the compat patch and compile DRBD
Where there is light there must be shadow
The spatch tool is not available on all Linux distributions. For a little older kernels we even require a very recent version of spatch, which is even less available. The researchers at INRIA write the tool in a programming language, “OCaml”, which is right for them and the challenge, but not familiar to many in the open source community.
This complex build dependency makes it harder for community members to build drbd-9.0.20 and higher compared to how it was before.
The shortcut through the maze
For a number of well known vendor kernels (RHEL/Centos, Ubuntu-LTS, SLES, Oracle linux, Amazon Linux) we include the complete compat patches in the distribution source tar.gz. Meaning, during the DRBD build process it executes all the COMPAT tests and calculates a hash value out of that.
If it finds a pre-generated compat.patch for that hash value, the build process can continue without a call to spatch! Complex build dependency avoided!
The hard route through the maze
When you are building from a GIT-checkout, or you are building for a kernel for which we did not include the pre-generated compat.patch you need spatch.
If necessary you can run step 2 (using spatch) on a different machine then approach step 1 (testing kernel capabilites) and step 3 (compiling the drbd kernel module).
Use ‘make’ to start the compilation process. If it fails just after this output:
[...]
COMPAT sock_create_kern_has_five_parameters
COMPAT sock_ops_returns_addr_len
COMPAT submit_bio_has_2_params
CHK /home/phil/src/drbd09/drbd/compat.4.15.18.h
UPD /home/phil/src/drbd09/drbd/compat.4.15.18.h
CHK /home/phil/src/drbd09/drbd/compat.h
UPD /home/phil/src/drbd09/drbd/compat.h
GENPATCHNAMES 4.15.0-48-generic
SPATCH 27e10079afbff16b2b82fae9f7dbe676
Please take note of the hash value after “SPATCH”. That is like a fingerprint containing all the results of the countless “COMPAT” tests that were executed just before.
Then you need to copy the results of the COMPAT tests to machine/VM/container that has the same drbd source directory and a recent spatch.
rsync -rv drbd/drbd-kernel-compat/cocci_cache/27e10079afbff16b2b82fae9f7dbe676 \
user@somewhere:src/drbd-9.0.20/drbd/drbd-kernel-compat/cocci_cache/
Then you run the spatch part of the build process there:
ssh user@somewehre "make -C src/drbd-9.0.20/drbd compat"
After that you copy the resulting compat.patch back:
rsync -rv \ user@somewhere:src/drbd-9.0.20/drbd/drbd-kernel-compat/cocci_cache/ \
drbd/drbd-kernel-compat/cocci_cache/
Call ‘make’ to restart the build process. If you did it right, it will find the generated compat.patch and finish the compilation process.
Get a Ladybug
If you’d like to get a spatch that is recent enough for building the DRBD driver, use a docker container we published on dockerhub https://hub.docker.com/r/linbit/coccinelle.
docker pull linbit/coccinelle
Then put the following shell script under the name ‘spatch’ into your $PATH.
#!/bin/bash
docker run -it --rm -v "$PWD:$PWD" -w "$PWD" coccinelle:latest spatch "$@"
DRBD compatible to the Linux kernel
All of this is great for making the code more readable, easier to understand, and more likely to contain less bugs. And, having the DRBD code, without backward compatibility clutter, is an important milestone on the path to getting DRBD-9 into Linus’ vanilla kernel and replacing drbd-8.4 with drbd-9 from there.
