The Coming Data Deluge

The top vendors in the overall storage software market are led by EMC with 24% share in 2008, followed by Symantec at 18%, IBM at 13%, and NetApp at 8%.  Interestingly, although these players command substantial presence smaller companies continue to chip away at market share.  I would not be surprised if, in ten years, the list of top ten vendors shuffles around quite a bit.

According to IDC, the total data created by businesses and consumers is roughly doubling every 18 months.  At this current rate, the universe of content created across the globe will grow five-fold by the end of 2012.  And, there’s no clear end in sight. It doesn’t take much to understand this when one starts to think about their own personal storage growth over just a few months.  Among others, the obvious following factors are playing a key role in this data deluge:
  • Global growth in Internet users, broadband penetration, and Internet-connected devices (both in terms of multiple computers per user and, now, powerful mobile devices).
  • Adoption of social networking applications and the exponential growth in media (photos, video, & music) that gets duplicated & distributed across the network.
  • Increasing bandwidth availability (which enables richer and bigger applications, especially video).
  • Migration of content stored on old media (paper, film, etc.) into digital format (movies, music, legal files, letters, books, magazines, etc) and the coming digitization of healthcare and other records.
This increasing deluge of digital data, I believe, is bolstering a continuously growing demand for focused storage software (and services), starting with traditional backup & recovery (with improved functionality across platforms & media), replication, archiving, and storage management.

Another important data point to consider is that today corporate organizations spend, on average, 28% of IT capital expenditures on data storage hardware, software, and services.  That’s a massive percent.  Given that a large majority of that is hardware today, how will that percent evolve over the next ten years as data continues to grow exponentially and historic data continues to accumulate?

Put into perspective from a consumer angle; think about what your files would look like if you had kept every single photo you ever took, every letter/paper you ever wrote, every card you ever received, every drawing you ever drew, and every document you ever saw or read.  Had you kept all of that material/ media for your lifetime, how would you manage it today?  How would you go about finding something?  How would you decide where to file something?

In this digital age, where we are individually clearly keeping more personal media than we ever have before (just think about the number of digital photos you have on your hard drive versus three years ago),  we are individually spending much less, as a percent, than corporations do.  Our spending on media & communications has dramatically increased over the past twenty years, but our personal spending on filing & storage of that media has not increased in a corresponding manner.

This material increase in data clearly presents a huge opportunity for continued innovation in capacity optimization technologies and more efficient system architectures to address this massive piece of our IT ecosystem (both in corporate environments and in the consumers’ living room).

Storage is a key component of this wave of digital media innovation we are living in.  So, if you ask me, the innovative companies in storage software are going to appear & grow mainly because of:
  • As mentioned above, absolute growth in bits & bytes, highlighting increased for storage management and protection processes, both for ease of access and for cost reduction.
  • Cloud computing, which is shifting the physical location of data, and demanding new tools & solutions to deliver storage-as-a-service, archiving & de-duplication, and multi-tenant management.
  • Increasing platform & application complexity, where we are moving from a singular platform (Microsoft) and a handful of application standards (text, images, email) to multiple platforms and applications.
This last point is critical.  A few years ago, we lived a world where we basically backed up Microsoft and, mainly, MS Office applications (along with some basic 2D & 3D media files).  Today, storage software has to deal with heterogeneous environments (for instance, various kinds of mobile data devices, networked computers, netbooks, and Apple, gaining ground in the desktop world), and an increasing number of file types & media.

There is clearly a lot of innovation yet to come in an industry that many view as mature.
blog comments powered by Disqus