If a petabyte won’t qualify as big data, what is going to? Cleversafe’s founder explains.

At what point does data take the plunge to special data? There is not any internationally sanctioned demarcation, without a doubt , and it should appear that the rapid growth of unstructured data, particularly video, has the imaginary boundary at the move.

According to Chris Gladwin, founder and vp of the item storage vendor Cleversafe, storage requirements have soared dramatically over the last decade, but that’s only the start. Emerging technologies comparable to 4K video and cloud services will make today’s big data storage needs seem laughably small by 2024.

Today, a petabyte of information — 1,024 terabytes, to be exact — probably meets many people’s definition of “big data.” The storage manufacturer Aberdeen, for instance, sells a one-petabyte storage rack, aptly named the Petarack, for a groovy $375,000.

Fast-forward 10 years, however, and a petabyte not will qualify as big — no less than not inside the enterprise. “You suspect of it as a pretty large system today, but in 10 years, you will not also be ready to buy systems that small,” Gladwin told InformationWeek. “Today, a petabyte is half a cupboard, but you exit 10 years, and that is like a part of a server. It won’t even qualify as an enterprise-scale system.”

[UK startup tackles the challenge of enabling computers to know the spoken word. Read Natural Language Processing: Big Data’s Role.]

Since Gladwin founded Cleversafe in 2004, the Chicago company has always thinking about storage systems with very large capacity. A decade ago, only 60-70 companies on earth acquired a petabyte or more of latest storage every year. “There has been maybe one organization at the moment that was at a 100-petabyte scale, and that was it. Now, 10 years later, the variety of companies on the planet which are deploying a petabyte or more of storage each year is around 7,000.”

A decade from now, “for those who study the capacity-optimized segment of enterprise storage — that’s the large enterprise storage systems — we’re projecting that zero percent of the market should be systems which might be a petabyte or less.”

So if a petabyte won’t qualify as big data, what’s going to?

Perhaps an exabyte, or 1,024 petabytes. “Today’s petabytes are tomorrow’s exabytes. There are 5-10 [Cleversafe] customers this year with the intention to deploy at exabyte scale.”

Currently, just a small group of businesses need that much storage. “It is the Internet-deployed services which are storage-intensive. It might probably include social networking and corporations that supply storage-as-a-service to others. The really big ones are at that scale.”

The growing demand from cloud storage, particularly user-generated content, is driving this trend, to boot. “You are taking a host of video files — you get one hundred million customers making them — and begin storing them. The subsequent thing you recognize, you’re at exabyte-scale pretty quick.” Government installations are reaching that level, too.

The arrival of 4K video — and finally 8K — may also hasten the arriving of exabyte-level storage systems. “We’re beginning the transition to 4K, and you may see the transition to 8K after that. Regardless of compression, each is actually a 4x jump” in storage. “In the event you actually need to get to large scale, you are not going to get there with numbers, text, and documents. You’re really talking about big objects, and many them — like videos or genomes, such things as that.”

Google is already at that scale, Gladwin said. “There is a billion people making videos, and that they all land on YouTube.”

Engage with Oracle president Mark Hurd, NFL CIO Michelle McKenna-Doyle, General Motors CIO Randy Mott, Box founder Aaron Levie, UPMC CIO Dan Drawbaugh, GE Power CIO Jim Fowler, and other leaders of the Digital Business movement on the InformationWeek Conference and Elite 100 Awards Ceremony, to be held at the side of Interop in Las Vegas, March 31 to April 1, 2014. See the entire agenda here.

Jeff Bertolucci is a technology journalist in La who writes mostly for Kiplinger’s Personal Finance, The Saturday Evening Post, and InformationWeek. View Full Bio

More Insights