Rancher’s K3s and LINBIT SDS

K3s is a very popular Kubernetes distribution from Rancher Labs designed to be both simple to set up and lightweight (in terms of system resources, not features). This makes it a great fit for edge computing, IoT (Internet of Things), and CI environments. LINBIT SDS for Kubernetes is LINBIT’s software suite for provisioning low overhead, high performance, feature-rich block storage for the popular cloud platform. This makes the combination of LINBIT SDS and Rancher’s K3s a great choice for unattended, resource-constrained, environments.

K3s deployment of Kubernetes is far simpler than even kubeadm, which is already relatively simple. K3s is packaged as a single binary which can be stood up using a single command. The launcher handles the complexities of setting up TLS certificates, as well as configuring and installing a CNI and container runtime. The default CNI setup by K3s is flannel, which does support using IPSEC or WireGuard for encryption, but canal and calico are also supported. For container runtimes, you have the option of using Docker, but containerd is used by default. I personally haven’t found the need to venture outside of these defaults, but who doesn’t like options?

As someone who spins up and tears down test clusters on my workstation multiple times a day, the combination of simplicity of setup and smaller footprint is a huge win. LINBIT’s internal CI spins up around 10000 VMs per day for testing, so obviously any simplifications or reduction of resources needed there is very welcome. Testing and CI use cases aside, the simplicity and smaller footprint apply to production-grade, multiple master (HA control plane), deployments as well.

If you’re an Ansible user, you’ll appreciate how simple this is. Here’s a snippet from my personal playbook for standing up K3s and LINBIT SDS:

- hosts: k3s_master
  any_errors_fatal: true
  gather_facts: yes
  become: yes

  tasks:
  - name: k3s init master
    shell: "curl -sfL https://get.k3s.io | sh -"

  - name: get k3s token as stdout
    command: "cat /var/lib/rancher/k3s/server/node-token"
    register: k3s_server_token

  - name: set k3s token as a fact on master
    set_fact: join_token={{ k3s_server_token.stdout }}
    retries: 5
    delay: 10

- hosts: k3s_worker
  any_errors_fatal: true
  gather_facts: yes
  become: yes

  environment:
    JOIN_URL: "https://{{ hostvars[groups['k3s_master'][0]]['inventory_hostname'] }}:6443"
    JOIN_TOKEN: "{{ hostvars[groups['k3s_master'][0]]['join_token'] }}"
  tasks:
  - name: k3s join worker
    shell: "curl -sfL https://get.k3s.io | K3S_URL=$JOIN_URL K3S_TOKEN=$JOIN_TOKEN sh -"

Those are the only plays needed for a single master Kubernetes cluster. Couldn’t be easier, and getting the token from the master node could be a task shorter if I tried harder. If you’re not an Ansible user, the installation is as easy as copying and pasting two commands. For the complete Ansible playbook I use for testing K3s with LINBIT SDS deployed on Ubuntu Focal hosts, check out my GitHub. There’s also a GIF running through the playbook at the bottom of this post.

Rancher’s K3s aims to support most modern Linux distributions, while LINBIT officially supports most major Linux distributions. LINBIT’s Kubernetes operator ships pre-built DRBD kernel modules for the most popular distributions (Ubuntu/RHEL/SLES), but if your Kubernetes/K3s host systems have build tools installed (build-essentials/kernel-devel) it can also be set to compile a module when a pre-built one isn’t available. ARM support in LINBIT SDS isn’t here yet (as of 2/26/21), so host support between K3s and LINBIT SDS doesn’t completely overlap.

If you already set up the K3s, you can follow the LINSTOR user’s guide for deploying LINBIT SDS into your K3s cluster.

According to the K3s documentation, there’s no difference in functionality for storage providers between K3s deployed Kubernetes and ‘plain-old’ Kubernetes. So, whether you’re interested in a lightweight and easy-to-manage Kubernetes cluster in your edge environments, or just want something to easily test LINBIT SDS with before deploying into a full-blown Kubernetes cluster, I definitely recommend checking out Rancher’s K3s.

Here’s a quick recording of LINBIT SDS running in K3s from the master node after running through my Ansible playbook: