LINSTOR is an SDS technology with snapshot shipping integrated into the code, capable of real-time replication using DRBD. However, real-time replication may not always be the method we want to use. When you replicate data in real-time, you want your connection to have very low latency and high bandwidth as much as possible. In the following cases, we want the opposite:
Let’s take it one by one:
In those cases, the data centers used in DR scenarios need to be located in remote areas to reduce their chances of being affected by disaster situations. However, this makes communication difficult. In such cases, traffic is usually served with high latency and low bandwidth. (due to high prices) so we see real-time replication turn into an expensive solution.
If the RPO and RTO values (we will cover this topic in a different blog post) meet your expectations, instead of replicating the data in real-time, you can ship data between two different LINSTOR clusters at certain time intervals or to another LINSTOR node in a single cluster in a different data center.
LINSTOR does data replication between its nodes with DRBD. However, there may be situations where you want two different LINSTOR clusters to transfer data to each other without DRBD. In such cases, we cannot expect LINSTOR to share a single DRBD resource among different clusters. Therefore, we can use snapshot shipping to transfer data from one LINSTOR cluster to another LINSTOR cluster.
As we all know, replication is not equal to backup. Therefore, although replication is an essential issue for your HA needs, backup is also critical in business continuity plans.
To take a backup, you need to freeze the data at a specific time and transfer the data up to that moment to some storage.
Today, this storage is accepted as object storage due to its API structure and ease of use. Here, LINSTOR can transfer any DRBD resource to object storage (s3 compatible) at certain time intervals (hourly, daily, weekly, monthly, etc.)
Because snapshot shipping has become a requirement for these three scenarios, LINSTOR engineers have integrated snapshot shipping into the code.
In this way, you can operate from a single command line or GUI for disaster scenarios, cluster replication, and backup needs that an SDS needs to solve and that you need on the storage side.
First, you need to create a “remote” definition in LINSTOR. The remote definition can be either an s3 compatible storage or another LINSTOR cluster.
To create an S3 target:
# LINSTOR remote create s3 myRemote s3.us-west-2.amazonaws.com my-bucket us-west-2 admin password
Or to create a remote LINSTOR cluster:
# LINSTOR remote create LINSTOR myRemote 192.168.0.15
Then all you have to do is start the shipping process with the following command:
# linstor backup create myRemote myResourceFor more detailed version of the functions, please check out our LINSTOR SNAPSHOT SHIPPING UserGuide.
Snapshot shipping is a bit of a sloppy term. Actually, it means, at first, the complete volume is transferred (in all relevant cases, this is a thinly provisioned volume, and only the parts of the volume that have real data are transferred). After the initial shipment, only differences between snapshots are transferred.
Expressed in other words, with shipping of incremental snapshot differences for all overwrites between two snapshots, only the last write gets transferred once.
A second important aspect is that a flaky network connection can not influence IO performance on the active side. A snapshot-delta gets applied into a new snapshot on the target, so an interrupted transfer leaves the previous snapshot untouched in place.
If LINSTOR’s storage pool is managed with ZFS, this feature leverages on zfs’s send/recv utilities. For LVM-thin, it uses the less known thin-send-recv utility by LINBIT.
Share this post
LINBIT is committed to protecting and respecting your privacy, and we’ll only use your personal information to administer your account and to provide the products and services you requested from us. From time to time, we would like to contact you about our products and services, as well as other content that may be of interest to you. If you consent to us contacting you for this purpose, please tick above to say how you would like us to contact you.
By clicking submit below, you consent to allow LINBIT to store and process the personal information submitted above to provide you the content requested.
As the world’s leading provider of Software-Defined Storage, High Availability, and Disaster Recovery software, LINBIT adds server clustering capabilities to any containerized, virtualized, or bare metal environment.
Copyright © 2022 LINBIT
All rights reserved. The words LINSTOR®, DRBD®, LINBIT®, and the logo LINSTOR®, DRBD®, and LINBIT® are trademarks or registered trademarks of LINBIT HA-Solutions GmbH and LINBIT USA LLC.