Securing AI Data Growth with Scalable Object Storage

In this post, I will talk about securing AI data growth with scalable object storage.

Data volume continues to grow at warp speed and with it the pressure to securely store vast numbers of large data sets. An estimated 200 zettabytes of data storage exist now and arguably a majority of that data needs protection. By 2030 estimates are that the volume of data will jump to close to 660 zettabytes. 

AI and GenAI’s processing of unstructured data is largely fueling this growth, giving the new generation of threat actors a fresh target opportunity – large language models (LLMs) rich with data. Businesses are seeing that securely storing these large data sets as well as growing volumes of other sensitive data can’t be done with traditional methods.

They’re deploying object storage with multidimensional scaling to provide the coverage and scale they need to defend against attacks. It’s a gathering storm as threat actors are now using AI to execute threats, turning AI against itself. Fighting these actors will take a storage method tailored to support large datasets and to reduce risk across all dimensions through which data travels.

Why is Object Storage Relevant?

Businesses have turned to object storage as the preferred method for protecting historical levels of data for on-premises data, as has already happened in the public cloud with services such as AWS S3.

As opposed to legacy methods like block or file storage, object storage’s architecture treats data as distinct objects composed of the data itself plus descriptive attributes, or metadata. Each object’s rich metadata can include hundreds of attributes — security tags, compliance rules, even AI dataset labels — making it ideal for diverse, large-scale datasets.

The objects are stored in logical containers called buckets and access occurs through APIs, which makes it easy to integrate data lakes or AI and analytics workloads. As opposed to traditional block storage, for example, which enables direct file changes, object storage’s APIs sets up barriers to make it more difficult for a threat actor to succeed. To access the data would require overwriting of an object or writing a modified object.

Another key aspect is object storage’s AWS S3 foundation. Amazon Simple Storage Service (S3) is the widely adopted API industry standard for storing, scaling, and efficiently retrieving data from the cloud and on-premises object storage. AWS3 is credited with helping establish object storage as a favored solution for managing and retrieving unstructured data.

Fighting Back with Multidimensional Scaling

Multidimensional scaling is a capability in leading object storage systems that provide new levels of adaptability for future growth. MDS works on the premise that if you can’t effectively scale to keep up with high data flows, manage and monitor large data workflows, and authenticate access, you can’t secure the data. MDS solves this by scaling to support increasing numbers of users, apps, storage capacity, metadata, performance, and security operations.

The ways in which this dimension of MDS can enhance data security are:

Scaling Security Operations per Second. S3 access requires both user authentication checks and security policy requests on every API interaction. These security ops quickly become a major resource and computational drain on storage systems, as most systems do not offer a way to scale these services independently. Cloud users can generate millions of requests per second on the storage infrastructure, each API request requiring user authentication and checking and evaluating complex access policies to guard against data privacy violations. However, in both private and public cloud environments, enforcing these security protocols is critical to cyber security defense. A modern solution implemented with multidimensional scaling can scale a disaggregated security service independent from other storage operations. It can scale to needed volume, meeting user demand without sacrificing performance.

Scaling Management and Performance. Monitoring storage security and performance related to the continual flow of unstructured data and the need to manage S3 buckets for security and lifecycle management present key operational challenges for security and IT staff. To successfully manage this data onslaught staff can efficiently scale functions like performance monitoring and activity logging. By automating tasks, staff saves time and stays ahead of issues, including events that might signal a cyber threat.

When S3 buckets can scale into the millions in use cases like backup-as-a-service, IT is ready for a better approach to managing bucket-specific policies like security and lifecycle. IT wants to avoid hitting hard limits on the number of buckets and taxing a storage system’s performance. A newer approach is to use distributed architecture and flash storage to enable scaling up to millions of buckets, maintaining low-latency, and ensuring high-performance.

Conquering the Future with Scalable Security

The growing use of AI to execute costly cyber-attacks and the increasing volume of AI, GenAI and unstructured data – all prompt an examination of better ways to manage and secure data.

Object storage and the scaling, organizational and access control attributes of MDS offer a means of strengthening data security while volume continues to grow. It is an approach tailored to a data centric, present, and future.

INTERESTING POSTS

About the Author:

Paul Speciale

Chief Technology Evangelist and CMO at Scality |  + postsBio

Paul Speciale is a data storage and cloud industry veteran with over 20 years of experience with small and large companies. Paul is currently the Chief Technology Evangelist and CMO for Scality, leading the team across activities ranging from building awareness to content development and lead generation, as well as being a spokesperson for the company.