Different Types of Storage | Object vs File vs Block Storage

Objects, blocks and files are different storage architectures that store and present the data in different ways based on requirements. Each has its benefits and limitations. Given below is a brief on each type of storage and when the storage architecture is typically used.

What are storage devices?

Any device or medium which is either permanently attached or is movable/transportable, capable of storing information in an electronic form can be referred to as a storage device. Some examples are hard disk drive, CD ROM, flash media, DVD ROM, memory stick, etc. Devices such as iPod, PDA, mobile phones, etc. also contain storage devices as part of the hardware.

Classification of storage devices

Two types of storage devices are used with computers. The first type is referred to as primary storage and comprises of the RAM or the internal storage. The RAM stores data that is directly accessible by the computer’s processor (CPU). RAM is commonly referred to as ‘memory’. The primary memory is temporary and is used to store program instructions and intermediate results of procedures.

The secondary storage device may be internal, external or removable. An example of secondary storage is the hard disk drive. It is a device that is non-volatile. Though it may be located inside a computer, this type of storage is not considered primary because it cannot be directly accessed by the CPU. Data on a hard disk drive is organized in the form of files. This affords slower access and is cheaper.

Tertiary storage is typically not located inside a computer. These are usually high-capacity storage devices that are designed to hold large amounts of data. An optical disk is an example. This type of device includes a mechanism to locate specific data and transfer it to a drive when requested.

Different Types of Storage Devices

Any computer user ends up using many different types of secondary storage devices.

• The internal or external hard disk drive is usually connected to the computer for retrieval of stored data using an interface such as a USB cable.
• Computers of today have the disk drives as externally connected devices. These can be easily removed and stored elsewhere. These devices hold the stored data unless explicitly deleted or overwritten using instructions from the computer.
• Until a few years ago, floppy disks, magnetic tapes, and other magnetic media were used as secondary storage devices and were popular. These devices used the principle of a read/write head magnetizing material that was coated on the disk to store information.
• Once these became cumbersome to use, storage device manufacturers started looking at optical storage devices. Such a device is written onto and read from using a laser beam. The marked advantage that they have is the capability to store large amounts of data.
• The different types of optical memory are CD-ROM, DVD-ROM, CD-R/W, etc. With every new software or application that is being developed, the demand for storage is greater.
• Flash drives also referred to as thumb drives are the most popular portable storage devices of today. They are compact and connect with the help of a USB port.
• Memory cards are used in digital cameras or mobile phones are also secondary storage devices. These can be transferred to a computer using a reader that is connected through a USB port.
• Some computers have installed solid-state drives (SSD) in place of hard disks. An SSD is advantageous in that it has no movable parts like the hard disk. Their costs are now very competitive with the result that they are being used more in computers now.
• Cloud storage refers to the technology that uses data stored in a remote location. Cloud storage uses devices located elsewhere and are connected using the Internet or other types of network connection. The cloud storage provider manages the devices and provides data backup too. Many users appreciate the managed services provided by cloud operators.

Image Credit: trigent.com

Object Storage

Object storage or object-based storage refers to the data storage architecture that manages stored data in the form of objects and not as files. The units of data storage called objects consist of:

1. Data: The stored data is organized into objects. These may be complete files or parts of a file (sub-file). They may also simply be a collection of bits and bytes that are related but nor part of any file.
2. Metadata: The metadata is created by the one who creates the object storage. The metadata is made up of information that is contextual, i.e., what the data is all about, what it should be used for, any other information about how it is used to be used, and its confidentiality.
3. A unique identifier: This is a unique piece of information (usually 128 bits) that is tagged along to be able to find out this data over a distributed system. The physical location of the data becomes irrelevant.

Object Data Storage Implementation

Once comprehensive metadata is added to the file, it is placed in a flat address space referred to as a storage pool. The comprehensive metadata is what makes object storage successful as it provides information about both the use and function of the data lying in the storage pool.

Benefits of Object Storage

Some of the benefits of object storage are as follows:

• Data Analytics: As object storage is driven by metadata, and because metadata for every piece of data is available, the opportunity for analyzing data becomes much greater.

• Infinite Scalability: You can keep on adding any amount of data to the pool.

• Quicker Retrieval: As there is no folder hierarchy, data can be retrieved quickly.

• Less costly: It is a cheaper method to store large amounts of data.

• Resource Optimization: Resources can be used at an optimal level with object storage of data.

Object storage works best for cases such as web content, archives and data backup. The flat address space facilitates ease of use. Objects that are protected are built into the object architecture by using multiple copies (at least three) of the data over a distributed system. If any node fails, the data can still be made available to the user by using the copies. Corruption of data is also overcome in much the same way.

Object storage always is not the answer. You, as a user, have to decide as to which storage architecture suits your needs the best.

Object Storage Providers

Storing a large amount of data in a traditional environment or by a single user is expensive. There are many object storage providers that offer cloud storage services. We list some of them here.

AWS S3: They are one of the pioneers in object storage technology. You can upload any amount of data and they will allow you to upload/download using API, the browser. They promise 99.99% durability of data. They offer the Standard, Standard IA and Glacier types of storage. You can move your data from one type of storage to another. They allow for storage of data on the basis of region and this means you can have faster content distribution when you keep your data near your customer. Any client can start with free storage offer of 5 GB.

Google Cloud Storage suits players in the small size to enterprise level of business. They offer four types of storage: Multi-Regional, Regional, Nearline, and Coldline. You can choose the one that suits you depending on the frequency of access to data and from which part of the world it is accessed more frequently. They have multiple data centres in the world and you can store your data in any city that you want. You also get to enjoy free tier storage of 5 GB.

Spaces by DigitalOcean, offers a two-month trial period. In New York and Amsterdam, where they operate data centres from, the cost is $5 monthly for 250 GB storage with an additional $0.20 per GB. For fixed-rate monthly fares, they are among the cheapest service providers.

Other players in this segment are Cloud Files by Rackspace, IBM Cloud Object Storage, Alibaba OSS, Microsoft Azure storage, and Oracle Storage. Whereas Azure has multiple data centres across different places in India, Oracle allows for different ways to connect with the data. Alibaba vouches for 99.9% availability guarantee and can handle over 50000 requests per second.

Object Storage Vendors

Object storage vendors deploy storage for regulatory archiving in the financial, insurance, legal, and healthcare sectors. Others use it for large-scale web files high-performance video, computation and entertainment purposes.

Some of the leading Object Storage Vendors are IBM Cloud Object Storage, Tarmin GridBank Enterprise, Dell EMC ECS, Quantum Lattus, NooBaa, NetApp, StorageGrid, Hitachi Content Platform, Caringo, and Cloudian HyperStore among others.

Object storage is continuously evolving in a manner similar to the way in which file accessed storage continues to evolve.

Object Storage API

A user can create, get and modify objects and metadata using Object Storage API. This is implemented as a set of Representational State Transfer (REST) web services. The HTTPS (SSL) protocol can be used to interact with Object Storage. Standard HTTP calls can be used to perform API operations. For using APIs that are language-specific, you can use RESTful API.

To change data, the user is required to authenticate oneself with a token. The token can be obtained from an authentication service after presenting the user’s credentials. On clearance, the service returns a token and an account URL.

Object Storage API supports a non-serialized response format. The Object Storage system organizes data like accounts, containers and objects through the service provider. The service provider creates your account and you become the owner of the account. It is made up of all the containers. The container is the namespace for objects. You can make use of an access control list (ACL) to control access to the objects in a container. The object stores the data content such as images and documents.

Object Storage API allows a user to perform the following functions in addition to many others.

• Store as many objects as desired

• Upload and store objects of any size

• Compress files

• Manage object security

• Schedule deletion of objects

This list is not exhaustive.

Object Storage AWS

In the object storage arena, Amazon Web Services (AWS) has many product offerings. It is useful to have a basic idea to know which product to use for what purpose. S3, EBS, and EFS are three products that work differently.

Amazon S3 (Simple Storage Service) offers both high-level data scalability performance and security. Customers across different industries can use this service to store any amount of web content, mobile app, enterprise application, IoT device, big data, and archive data. The management features of S3 are simple and easy to use. This enables users to easily access stored data and tune according to their business needs. AWS S3 offers 99.99% data durability for millions of companies worldwide.

File Storage

Types of File Storage

Network storage devices are typified in the way they are interfaced on the client-side. Multiple clients are able to access a single shared folder in a traditional file-sharing system. The two popular protocols that enable this system are NFS and SMB/FICS.

The file system offers the simplest architecture for storage systems. When the amount of data grows larger, the resource demands grow and cannot be compensated by simply adding storage space.

The files are then organized into directories and sub-directories. Naming conventions make it easy for them to be organized. Most file storage systems allow for a centralized and easy retrievable system for the data. The cost is reasonable.

The file system works best when the amount of data is small and stored on personal computers and servers used in medium to large enterprises or workplaces. They can typically be used in conditions such as an office where you need to store and share files.

Locally archived files can use file types of storage systems with a NAS (Network Attached Storage) solution. The data centre in such a case is typically small. This type of storage architecture can also be used to protect data. This is supported by the use of standard protocols, different drive technologies, and native replication methods.

File Storage Containers

The data is stored in containers the form of files, given a name and tagged with metadata which consist of file creation date, modification date and the size of the file.

Data stored in files are retrieved using very little metadata. This metadata informs the computer where exactly the file is located. Every file is arranged following a specific hierarchy, by directories and sub-directories. This arrangement is well supported by NAS systems.

In cloud file storage, data is stored in the cloud. Other applications and servers can access this data through shared file systems. Users can create, edit, delete read and write files or even organize them in a logical fashion using directory trees. Cloud file-systems allowing shared access controls security with user and group permissions.

Block Storage

Block storage can be considered as the alternative to file storage. A block is a volume that is filled with files which are further split into blocks of equal size. Usually, such blocks can be well managed by a server-based operating system.

Each of these chunks of data can be managed as individual hard drives. There are many third-party applications used by organizations that help to manage data in block storage architectures.

Block storage is known to handle metadata very efficiently and the operating system allocates the storage for various applications and decides where the data is going to reside in the block. The control is efficient and accounts for the high performance that block storage architecture offers.

Block storage is used for a wide range of database applications that require high-performance levels, email servers that do not support file systems or network-based systems, virtual machines that use guest operating systems, etc.

Block-based architecture can be expanded as the volume of data grows; however, the integrity of such systems is likely to be tested when the volume grows into terabytes or petabytes of storage.

Block Storage Providers

Most of the major enterprise-level storage providers offer San products/block storage solutions. Top providers include Dell EMC, Hitachi Data Systems, IBM, HPE, NetApp, etc. In the cloud, AWS Elastic Block Store (AWS EBS) are providers of scalable block storage usable by EC2 (Elastic Cloud Compute) instances. Therefore, all those applications that run on SAN can place their databases, applications and workloads on Amazon’s cloud.

Block Storage Vendors

Block storage vendors include Huawei, DataDirect Networks, Nutanix, Oracle, etc., in addition to the largest storage providers. These vendors have several block storage platforms. Some of them provide unified arrays for both block and file storage.

Block Storage in AWS

Amazon EBS (Elastic Block Store) is the block storage offered by Amazon Web Services. This is used to store persistent data. Highly available block storage volumes are provided for EC2 instances making it suitable for the same. The three types of volume offered are EBS General Purpose (SSD) suitable for small and medium workloads, Provisioned IOPS (SSD) which suits I/O intensive transactional workloads, and magnetic for infrequent access of data. The cost, performance and characteristics vary for the three types.

The benefits of using Amazon EBS include fine access control with encryption, reliable and secure storage, usage of SSD technology for higher performance, and easy data backup.

With the advancement of technology, the amount of data has grown exponentially and every major enterprise is looking for the best ways to store its data. Despite the fact that the IT applications of today are trending towards object-based storage, file and block systems are still used extensively. The point to start at is to identify individual needs appropriately and choosing the best-fit storage architecture.

ApacheBooster is a must-have plugin for those who need to increase their server’s speed in a short span of time. It is cheaper and optimum in functionality compared to other plugins in the market.