Spotlight on LINSTOR’s design and technology
What we do to LINSTOR’s design and technology and how we do it to create a powerful, flexible and robust storage cluster management software.
LINSTOR® is an application that integrates with highly automated systems, such as software-defined storage systems or virtualization environments. Users often interact with the management interface of some other application that uses LINSTOR to manage the storage required for that application’s use case. This also means that users may not have direct access to the storage systems or to the LINSTOR user interface.
A single storage cluster can be the backend of multiple independent application systems. Therefore, the biggest challenge for software like LINSTOR is to remain responsive. Even if some actions or components of the cluster fail. Additionally, the software should be flexible enough to cover all use cases and enable future extensions or modifications. Also, despite all the resulting complexity, it should be easy to understand and easy to use for administrators. This is key when it comes to installing and maintaining the storage system.
It is quite clear to anyone who has worked on a bigger software project as a developer that many of those requirements work against each other. Customizability, flexibility, and an abundance of features cause complexity, but complexity is the natural enemy of usability, reliability and maintainability. When we started the development of LINSTOR, our challenge was to design and implement the software so that it would achieve our goals with regard to feature richness and flexibility while at the same time remaining reliable and easy to use.
Modularity in LINSTOR’s design and technology
One of the most important aspects of LINSTOR’s design is its modularity. We divided the system into two components – the Controller and the Satellite. The Controller component could remain as independent as possible from the Satellite component – and vice versa.
Even inside those two components, many parts of the software are exchangeable. The communication layer, the serialization protocol, the database layer, and all of its API calls. Even all of the debug commands that we use for internal development. And many other implementation details are exchangeable parts of the software. This provides not only maximum flexibility for future extensions, but it also acts as a sort of safety net. For example, if support for the database or the serialization protocol that we use currently were dropped by their maintainers, we could simply exchange those parts without having to modify every single source code file of the project, because implementation details are hidden behind generic interfaces that connect various parts of our software.
Another positive side effect is that many of those components, being modular, are naturally able to run multiple differently configured instances. For example, it is possible to configure multiple network connectors in LINSTOR, each bound to different network interfaces or ports.
A single communication protocol
As a cluster software, LINSTOR must of course have some mechanism to communicate with all of the nodes that are part of the cluster. Integration with other applications also requires some means of communication between those applications and the LINSTOR processes, and the same applies to any kind of user interface for LINSTOR’s design and technology.
There are lots of different technologies available, but many of them are only suitable for certain kinds of communication. Some clusters use distributed key/value stores like etcd for managing their configuration, but use D-Bus for command line utilities and a REST interface for connecting other applications.
Instead of using many different technologies, LINSTOR uses a single versatile network protocol for communication with all peers. The protocol used for communication between the Controller and the Satellites is the same as the one used for communication between the Controller and the command line interface or any other application. Since this protocol is implemented on top of standard TCP/IP connections, it also made all aspects of LINSTOR’s communication network transparent. An optional SSL layer can provide secure encrypted communication. Using a single mechanism for communication also means less complexity. The same code can be used for implementing different communication channels.
Transaction-safety
Even though LINSTOR keeps its configuration objects in memory, there is an obvious need for some kind of persistence. Ideally, the contents in memory should match what is persisted. This means that any change should be a transaction, both in memory and on persistent storage.
Most Unix/Linux applications have traditionally favored line-based text files for the configuration of the software and for persisting its state. LINSTOR, however, keeps its configuration in a database. Apart from the fact that a fully ACID-compliant database is an ideal foundation for building a transaction-safe application, using a database also has other advantages.
For example, if an upgrade of the software requires changes to the persistent data structures, the upgrade of the data can be performed as a single transaction. The result is either the old version or the new version of the data. Not some broken state in between. Database constraints also provide an additional safeguard that helps ensure the consistency of the data. Let’s assume that there was a bug in our software. And a failure to detect duplicate volume numbers being assigned to storage volumes. The database would abort the transaction for creating the volume due to constraint violations. Thereby preventing inconsistencies in the corresponding data structures.
LINSTOR uses its own integrated database by default. This helps users avoid setting up and maintaining a database server. It is an integral part of the Controller component. Optionally, the Controller can also access a centralized database by means of a JDBC driver.
Read more about LINSTOR’s design and technology in the second blog post!
Find LINSTOR in our Github repository. Contact the team if you want to discuss anything mentioned in this post. We’re always online and always available to talk.
