The future of storage is a really wide, and open ended topic. I’ve been in the data center/systems administration/storage business for 30 years, so I can look back a long way. The one thing that’s been a constant during all that time is that things change, and they change in ways we never expected at the time.

So looking very far into the future is going to be rife with speculation and in most cases just plain wrong. But what I think we can do is look at current technology which is just being introduced, and make some intelligent guesses about how it might be used in the very near future, say within the next couple of years.

I think that there are a few sea change kinds of technologies which are just starting to become mature enough that they are being deployed into production right now. Things like storage virtualization, deduplication, thin provisioning, etc. are really starting to catch on and I think that they are going to get combined in some interesting ways. For example, I think that server virtualization when combined with thin provisioning, network attached storage, and deduplication can have a very profound impact on some of the real issues that we are fighting in the data center today.

Specifically, if you combine those technologies, you will be able to deploy systems much more quickly, and manage them much more efficiently than we ever have before and you will be able to do it using less disk space than you might be today.

That’s a good thing since systems are proliferating like mad. Server virtualization has created more “servers” than ever and created more demand for storage than ever. Being able to grow a virtual server’s available space on the fly without disruption to the users of that virtual server will soon become a requirement in this 24×7x365 world a lot of companies are finding themselves living in. Not only will you have to be able to grow the storage on a virtual server at the drop of a hat, but you will need to be able to manage all of the storage behind all of those virtual servers much more efficiently than you ever have before. That means dealing with the issues of end of lease, outages, backup and recovery, and DR. Some of those new technologies are going to be very helpful in dealing with all of those issues, and combining them will create some synergies as well.

Soon we will be able to grow the storage on a virtual system in the fly, maybe even automate the growth from a pool of storage just for that purpose. The OS and applications stored on those virtual servers will be stored using deduplication technology reducing the on disk footprint by a significant margin which will help to slow the growth of storage a little. We will back up these virtual servers to disk, again using deduplication technology so that the backups don’t take up huge amounts of disk space and we will use CDP to replicate that data to our backup data center, again using network deduplication technology to reduce the quantity of data flowing over our network link. All of the above we can do by putting together existing, shipping technology.

Some of the issues that we are beginning to run into, however, still don’t have a cost effective solution yet. For example how we are going to address the performance issue of current disk drive technology? Back in the day when we had to throw a lot of drives at a big database in order to get enough space bandwidth to disk wasn’t an issue. But drives have been getting bigger and bigger for a long time but the rate that I can pump data to/from a drive has remained relatively flat in comparison to the quantity of data stored on a drive. To the point now where we end up having to run drives 1/2 empty in order to spread the database across enough spindles to get the performance we need.

One interesting technology that is just starting to ship from array vendors is SSD. Sure, it’s expensive as heck. Some say 300x what a regular disk drive costs. On the other hand, if you’re running your database on 1/2 empty drives, that isn’t exactly cost effective either. Besides, like with any technology I’m sure that as more and more of it gets shipped the price per GB will come down. So I think that SSD is something we are going to see a lot more of over the next 3 or 4 years. Of course it will start with the people who have the deep pockets and the applications that really need the speed. ERP, for example is one application that springs to mind on the business side that can take advantage of SSD. There are certainly a lot of other application on the scientific side that will also be able to take advantage of SSD, and I’m sure that certain 3 letter government agencies already have it deployed (that’s a guess, please don’t send the guys with the mirrored sunglasses to my house to find out everything I know about this, I don’t know anything, really).

The bottom line is that the business side of IT is going to be looking to reduce costs, just like always, while they continue to grow their dependence on IT. The seemingly never ending thirst for more storage is not going to be ending any time soon. If anything new technologies will just continue to accelerate this growth, and our challenge is going to be to figure out how to manage these massive amounts of storage as efficiently as possible. Because if we don’t, upper management is going to find someone who can to replace us.

My advice to everyone who’s managing storage is, don’t be afraid of new technology. Find ways to combine it in ways that will help you to be more efficient and effective in managing the storage. Remember, you have been entrusted with something that is rapidly becoming the crown jewels of your company, the company’s information and it’s up to you to manage and protect that information. That’s an awesome responsibility, but don’t let it scare you, it’s also a heck of an opportunity!