Storage systems are almost unquestionably the most critical component of larger IT systems. Without storage systems operating normally, websites could serve outdated information, customers and employees may not have access to the information they need, and in extreme cases, data loss could put a company out of business entirely. You could make similar statements for other components in IT systems, but it would be difficult to permanently destroy a network via misconfiguration – that’s easily done with data.
Selecting the incorrect storage system for a use case can have less dramatic consequences while operating normally. This could result in slow read and write performance, complex integrations with other systems, or difficulty managing and supporting the solution from an administrative perspective.
Any component as crucial as storage needs to be carefully considered. One of the first things you’ll need to think about when considering different storage solutions is the type of storage you need for your data management use case. In this article, we will go over three different types of storage systems and hopefully help you figure out which one is right for you.
Different Types of Storage
Block, file, and object storage are the three most common types of storage solutions you’ll find in modern IT systems. Each of these storage formats has characteristics that make it better or worse for various use cases. Block storage breaks up amounts of data into evenly sized blocks which are stored and retrieved from uniquely addressed segments of a physical disk; file storage organizes and represents unstructured data as a hierarchy of files in folders; and object storage is unstructured data stored in repositories (or buckets) that are flatly organized and generally highly scalable.
What is Block Storage?
Block storage stores data in uniquely addressed fixed-size blocks on physical hard drives. Hard drives, or collections of hard drives, can be logically partitioned into smaller sets of blocks in order to create individual volumes from the larger disk, or array of disks. Block storage can be directly attached to a system, or attached over a network using protocols like iSCSI or iSER, Fiber Channel, or more recently NVMe-oF in storage area network (SAN) environments.
Block storage is the “lowest level” of the storage types discussed in this post, meaning protocols and software accessing block storage have the least abstraction between them and the physical hard drives. It’s this low level access that makes it one of the most performant types of storage, making it a great fit for applications that demand low latencies and high throughputs, such as databases, message queues, and mission critical virtual machines.
Block storage’s low level of access does come with the restriction of only being able to attach a volume to a single system (unless the software accessing the block storage is cluster aware). This means an orchestrator, or cluster resource manager, is needed to make block storage highly available.
What is File Storage?
File storage is the hierarchical folder structure filled with files, which should be familiar to all computer users. File storage is implemented over block storage, providing a user-friendly interface for storing and retrieving otherwise unstructured and dissimilar sets of data. You’re likely using local file storage on your personal computer, but file storage can also be shared on a network within an organization. When file storage is shared on a network, it’s often referred to as Network Attached Storage, or NAS.
NAS solutions are intended to be accessed by multiple systems/users simultaneously, and therefore are often used within organizations to store and share important information from a centralized location The centralized nature of the NAS makes it easier to backup and recover data, as it can all be done from a single location. The shared nature also makes it ideal for storing virtual disk images in a hypervisor cluster, common files in a web farm, large archives or multimedia files like security camera footage from various sources.
The file locking that must happen whenever you’re accessing data from multiple systems at once does lend itself to more overhead, and therefore less performance than you’d get from a SAN volume. However, all-flash NAS systems and improvements in network technology have helped close the gap in performance between the slower ethernet attached NAS and faster Fiber Channel attached SANs.
What is Object Storage?
Object storage is a storage architecture that places data into discrete “buckets” or “objects” coupled with a variable amount of custom metadata describing the data. Each object is given a globally unique identifier, and is stored in flatly structured, often distributed, systems. Object storage can be implemented over a filesystem, but can also write directly to a block device for fewer layers of abstraction and therefore better performance. Object storage provides interfaces for developers to leverage in applications, making storage easier to implement than the lower level storage types.
Object storage’s custom metadata and relative ease of interfacing with applications makes it well suited for larger deployments involving huge amounts of unstructured data. Services like Facebook, Spotify, and Dropbox use object storage to store user’s photos, multimedia, and uploaded files respectively. Features like object replication ensure that objects are available in the event of hardware failure.
This higher level approach to storing data does have its drawbacks. Typically, write performance isn’t as good as block or file based storage, partially due to the storage system’s need to coordinate and update the location of an object in a distributed system, as well as ensuring access isn’t granted to more than one writer at a time. Write performance has improved greatly since object storage’s inception, but generally speaking, it still lags behind block and file based storage. Also, managing data of variable sizes makes it more difficult to tune a storage backend for larger or smaller writes.
Conclusion
As with many solutions in the IT infrastructure world, the best option for you and your organization largely depends on your unique use case. As we have touched on in this article, all three types of storage systems shine in their ideal use cases but have obvious downsides when the wrong storage type gets shoehorned into a project. We hope this sheds some light on the three pillars of storage and when to consider them as your storage solution. Don’t forget, LINBIT is here to help with all of your high availability, disaster recovery, and block storage needs!
