Managing AWS EBS Volumes Using LINSTOR

Cloud storage is a critical component of running persistent workloads in the cloud, especially in hybrid cloud environments. LINSTOR®, the open source storage orchestration tool from LINBIT®, integrates seamlessly with Amazon Web Services (AWS) Elastic Block Store (EBS) to simplify the management of EBS volumes. LINSTOR also layers DRBD®, LINBIT’s block device replication driver, over EBS volumes adding cross availability zone (AZ) or even cross region replication capabilities to EBS backed storage. In this blog, I’ll explore how you can use LINSTOR to create, attach, and replicate EBS volumes between EC2 instances spread across multiple AZs within a single AWS region.

❗ IMPORTANT: LINSTOR version 1.30.0 (or newer) is required for managing EBS volumes by using LINSTOR. Changes to EC2 required changes to LINSTOR, therefore making previous versions of LINSTOR unable to attach EBS volumes.

Benefits of using LINSTOR for EBS management

While this blog will only focus on creating and attaching EBS volumes in AWS and replicating their contents between AZs using DRBD, there are many benefits to using LINSTOR to manage storage in both AWS and on-premise architectures. LINSTOR allows you to abstract storage management across on-premise and cloud environments, enabling:

• Unified Orchestration: Centrally control EBS backed storage across EC2 instances and AZs using a single interface.
• High Availability: Extend EBS’s native durability with DRBD’s replication enabling high availability (HA) and disaster recovery (DR).
• Cost Efficiency: Optimize storage configurations by dynamically allocating resources to meet workload demands.
• Scalability: Expand storage as your workload grows, without the complexity of manual provisioning.
• Flexibility: Snapshot shipping, or streamed replication, between LINSTOR clusters enables migration between on-premise and cloud infrastructure.

Teaser alert: Another blog demonstrating how one could use LINSTOR in hybrid clouds will come soon.

How to configure and use LINSTOR to manage EBS volumes

The following section assumes that you have already created three EC2 instances, one in each of the us-west-2a, us-west-2b, andus-west-2c. It also assumes you’ve installed and enabled LINSTOR and DRBD v9 on all nodes.

1. Setting up encryption for AWS credentials

LINSTOR uses its encryption passphrase to store AWS credentials securely. Start by creating the encryption passphrase if you haven’t already configured one:

linstor encryption create-passphrase -p <passphrase>

This passphrase ensures your AWS credentials are securely encrypted within LINSTOR’s database, and is required.

2. Configuring LINSTOR remote objects for each AWS AZ

You need to create LINSTOR remotes for each AWS AZ to interact with the EBS service. The LINSTOR remote must be configured with an AWS IAM user’s access_key and secret_key, and the user must have at least the following actions allowed on their permission policies:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:CreateVolume",
        "ec2:AttachVolume",
        "ec2:DetachVolume",
        "ec2:DeleteVolume",
        "ec2:DescribeVolumes",
        "ec2:ModifyVolume",
        "ec2:CreateSnapshot",
        "ec2:DescribeSnapshots",
        "ec2:DeleteSnapshot",
        "ec2:DescribeInstances",
        "ec2:DescribeAvailabilityZones",
        "ec2:CreateTags",
        "ec2:DeleteTags"
      ],
      "Resource": "*"
    }
  ]
}

When entering the following commands, be sure to enclose both your access_key and secret_key within single quotation marks, to avoid the Linux shell from trying to interpret any special characters they might contain:

linstor remote create ebs ebs-rem-us-west-2a us-west-2a '<access_key>' '<secret_key>'
linstor remote create ebs ebs-rem-us-west-2b us-west-2b '<access_key>' '<secret_key>'
linstor remote create ebs ebs-rem-us-west-2c us-west-2c '<access_key>' '<secret_key>'

These commands link LINSTOR to EBS endpoints in each specific AZ, ensuring EBS volumes can be managed regionally.

3. Creating LINSTOR special satellites for AWS integration

LINSTOR’s “special satellites” serve as intermediaries between LINSTOR and AWS EBS. Because LINSTOR has a concept of LINSTOR satellites providing storage to LINSTOR storage pools, these special satellites can be thought of as “dummy” nodes in each AZ that provision EBS volumes as targets from their resulting storage pools. Later you will see that the provisioned EBS volumes, or “targets”, are then attached to LINSTOR satellite EC2 instances, similar to “initiators” in other client/server storage technologies, such as iSCSI. The special satellites need to be created relative to the LINSTOR remote in their AZ:

linstor node create-ebs-target ebs-specsat-us-west-2a ebs-rem-us-west-2a
linstor node create-ebs-target ebs-specsat-us-west-2b ebs-rem-us-west-2b
linstor node create-ebs-target ebs-specsat-us-west-2c ebs-rem-us-west-2c

Special satellites will manage EBS volume creation and deletion in their specific AZs.

4. Adding LINSTOR satellites (EC2 instances)

Next, register your EC2 instances as LINSTOR satellites using the instances’s private IP addresses:

linstor node create ec2-linstor-a <private_ip_from_instance1> --node-type satellite
linstor node create ec2-linstor-b <private_ip_from_instance2> --node-type satellite
linstor node create ec2-linstor-c <private_ip_from_instance3> --node-type satellite

These satellites represent the EC2 instances where EBS volumes will be attached.

💡 TIP: If your LINSTOR satellites are managed by the LINSTOR controller over a different network, you may want to add the LINSTOR satellites to the LINSTOR cluster using one network address, and then configure their private IP addresses as additional interfaces in LINSTOR to be used for DRBD’s replication.

5. Creating storage pools for EBS initiators

LINSTOR ebs_initiator storage pools enable you to manage volume attachments. These storage pools must also be created relative to the LINSTOR remote in their AZ:

linstor storage-pool create ebs_initiator ec2-linstor-a EbsInitPool ebs-rem-us-west-2a
linstor storage-pool create ebs_initiator ec2-linstor-b EbsInitPool ebs-rem-us-west-2b
linstor storage-pool create ebs_initiator ec2-linstor-c EbsInitPool ebs-rem-us-west-2c

6. Provisioning an EBS volume across multiple AZs

Create an EBS backed LINSTOR resource group with redundancy across two AZs. On the resource group use the --storage-pool option to reference the EbsPool LINSTOR storage pool that was automatically created when registering the LINSTOR special satellite in each AZ. Then, create the necessary LINSTOR volume group referencing the ebs-rg resource group without any options that would configure additional properties. Later, if you need to, you can set properties on the volume group level. And finally, create (“spawn”) a resource from the resource group:

linstor resource-group create ebs-rg --storage-pool EbsPool --place-count 2
linstor volume-group create ebs-rg
linstor resource-group spawn ebs-rg ebs-res-0 80G

💡TIP: LINSTOR will provision EBS volumes of the gp2 type by default. To configure other volume types, such as gp3, io2, or, sc1 for example, you can set the EBS/VolumeType property on the LINSTOR resource-group or controller level.

You should see some “SUCCESS” outputs from LINSTOR, but you will also see two new volumes within the EBS portion of the AWS EC2 dashboard. Each of the newly provisioned EBS volumes should exist within a different AZ, enabling high availability (HA) across AZs.

7. Attaching the EBS volumes to EC2 instances

Finally, before you can use the EBS volumes, you must attach the EBS volumes to the EC2 instances via LINSTOR “initiator” resource assignment.:

linstor resource create ec2-linstor-a ec2-linstor-b ebs-res-0 \
        --ebs-initiator --storage-pool EbsInitPool

After running the above command, you should see the in-use EBS volumes on your LINSTOR special satellites, and the resource synchronization process between your EC2 LINSTOR satellites:

linstor resource list
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────╮
┊ ResourceName ┊ Node                   ┊ Port ┊ Usage  ┊ Conns ┊              State ┊ CreatedOn           ┊
╞══════════════════════════════════════════════════════════════════════════════════════════════════════════╡
┊ ebs-res-0    ┊ ebs-specsat-us-west-2a ┊ 7000 ┊ InUse  ┊ Ok    ┊             in-use ┊ 2024-12-04 14:09:54 ┊
┊ ebs-res-0    ┊ ebs-specsat-us-west-2b ┊ 7000 ┊ InUse  ┊ Ok    ┊             in-use ┊ 2024-12-04 14:09:54 ┊
┊ ebs-res-0    ┊ ec2-linstor-a          ┊ 7000 ┊ Unused ┊ Ok    ┊           UpToDate ┊ 2024-12-04 16:01:41 ┊
┊ ebs-res-0    ┊ ec2-linstor-b          ┊ 7000 ┊ Unused ┊ Ok    ┊ SyncTarget(62.57%) ┊ 2024-12-04 16:03:53 ┊
┊ ebs-res-0    ┊ ec2-linstor-c          ┊ 7000 ┊ Unused ┊ Ok    ┊         TieBreaker ┊ 2024-12-04 16:03:47 ┊
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────╯

This step finalizes the process, making each EBS volume accessible to its respective EC2 instance as a DRBD device. You can use the linstor volume list command to get the DRBD device path, found in the DeviceName column of the command output.

Anything written to the DRBD device will be replicated to all peer nodes, and written to their underlying EBS volume.

8. Creating snapshots of LINSTOR managed EBS volumes

The typical LINSTOR commands for creating snapshots work for EBS backed LINSTOR resources, for example:

linstor snapshot create ebs-res-0 ebs-snap-0

After entering this command, you will find that you have an EBS snapshot in your AWS console that is also managed by LINSTOR.

9. Deleting EBS backed LINSTOR resources

LINSTOR treats EBS volumes and how they are attached to satellites similarly to “targets” and “initiators”, which should be familiar from other client/server types of network storage. This means deletion of the “initiator” resources (the EBS volume attachment to the LINSTOR satellite) must occur before the “target” resources (EBS volume provisioned on the LINSTOR special satellite) are deleted. For completeness, the LINSTOR resource definition should also be deleted from the LINSTOR database:

linstor resource delete ec2-linstor-a ec2-linstor-b ec2-linstor-c ebs-res-0
linstor resource delete ebs-specsat-us-west-2a ebs-specsat-us-west-2b ebs-res-0
linstor resource-definition ebs-res-0

📝 NOTE: If any LINSTOR snapshots of the resource were created, you must delete those before deleting the “target” resources.

Conclusion

Managing AWS EBS volumes with LINSTOR enables mulit-AZ resiliency without sacrificing performance in the cloud. Also, by abstracting the EBS provisioning and attachment, LINSTOR empowers you to build highly available and flexible storage solutions regardless of environmental differences between on-premise data centers or public clouds. Whether you’re running a hybrid cloud or purely AWS based workloads, LINSTOR’s integration with EBS provides an efficient and flexible way to manage your storage infrastructure.

For more details and advanced configurations, check out LINBIT’s official documentation, engage with other users on LINBIT’s forums, or reach out to us directly.