I just got back from a series of trips round the U.S. and Europe. One of the most surprising things I learned over the course of the last month, is that more and more developers, IT decision-makers, “data guys” and just plain business people, are beginning to get religion around evolving their approaches to storage.
Now, I am not saying everyone is sold on Riak or even NoSQL, far from it. But what I am saying is that individuals inside some large, small and soon-to-be-large organizations realize that they a) can increase their storage capacity and broaden their data storage bandwidth (ie store more types of objects) while b) not significantly increasing their overall IT budgets.
Right in the middle of my travels, a Forrester report was published which included results of a survey of more than 1500 IT professionals. They were asked about which areas of storage were going to increase in the next year. The results:
In my opinion, the fact that “general file storage” and “archiving” are just as important as “database systems” proves that we have seen nothing yet, as it were, it terms of how big the market can and will be for object and document oriented data storage solutions. I think if you take the market opportunity for custom solutions that look and feel like Amazon S3 but live inside private or hybrid clouds (or even on non-EC2 public clouds) – we’re talking big business. And if you see how well a solution like Riak is fitted for, say a global session store for a massively utilized web property – then it mounts up to even more potential.
The most important aspect of this report, in terms of how well it bodes for distributed databases, is that the authors point to the fact that new storage solutions will be played out in the cloud. The report survey also revealed that storage accounts, or should I say will account for about 17% of total costs in 2012. To keep costs down, and to scale properly as these data volumes increase – the cloud has to come in to play.
So, as I’ve said again and again – there is a huge opportunity for data solutions that focus on being optimized for distributed systems.
The report, sadly, does not go on to mention how there are multiple open source and highly scalable solutions to the kinds of data issues outlined in the piece. Instead, it takes a “traditional storage married to Hadoop” approach to offering up a solution. While this accounts for solving a small portion of the types of data and use cases that are pressing companies of all sizes, it certainly does not solve all of them.
And again, as the light turns green for NoSQL and Big Data across many storage segments – there is room for a whole lot of great solutions.