Techdirt Insigit Community Share your feedback on the rapidly evolving
Storage Area Network (SAN) market.
Powered by the Techdirt Insight Community.

SAN and Deduplication

 

Joseph Hunkins Submitted by Joseph Hunkins on May 2nd, 2008

My initial thinking, which I now realize was flawed, was that the deduplication aspect of the SAN Market would become insignificant thanks to the twin factors of plummetting storage costs, increased processing speeds, and robust cloud computing storage environments like Amazon S3. However I now see that despite these major advances several factors make deduplication a key part of any robust SAN implementation:

1) Without effective deduplication complications can arise that will corrupt or mismanage data.

2) Total data storage needs are still outstripping the reduction in storage costs. As broadband content becomes ubiquitous video files will take up staggering amounts of storage space.

3) In addition to lowering the total storage needs of the enterprise, energy and processing needs are significantly reduced by innovative use of deduplication.

Jon Mellon of COPAN summarizes the need for deduplication nicely:

With de-duplication, companies reduce not only the number of copies of data kept, but also the amount of data that is replicated, moved, and otherwise retained for long-term storage. An organization’s backups typically accumulate a lot of redundant data over time – some of it duplicated 10 to 30 times – but a well-architected system that employs de-duplication can reduce overall storage by a factor of 10:1 or 20:1, or even more. Altogether, that kind of efficiency has an enormous impact on every aspect of storage costs.

NetApp, a leading deduplication software vendor offers online deduplication cost calculators:
Backup Calculator | Primary and Archive Data Calculator

Thus is appears that deduplication solutions are not only “ready for prime time” but are an increasingly important concern for your SAN implementations. When combined with full SAN implementations vendors like NetApps offer many customers excellent storage upgrade paths without massive overhauls of existing architectures. NetApps appears to be a key innovator in this space given their market share and their innovative solutions that connect old and new components using inexpensive SAN innovations like iSCSI. Rather than overhaul the entire enterprise architecture it is often most cost effective and expedient to use a hybridized SAN environment where you keep the expensive legacy networking environments that are using Fiber Channels and connect them to newer components that use the far less expensive iSCSI..

Sources and relevant linkage:

Deduplication summary - a great post from Jon Mellon:
http://compliancegoals.com/pastissue/article.asp?art=272481&issue=235

Deduplication post by Heidi at Computer world:
http://blogs.computerworld.com/node/1676

Deduplication Article (dated but good):
http://www.enterprisestorageforum.com/technology/features/article.php/3617296

NetAPP’s huge success and market leadership in SAN market:
http://news.moneycentral.msn.com/ticker/article.aspx?Feed=BW&Date=20071210&ID=7920595&Symbol=US:NTAP

Insight from TIC member Stephen Foskett:
http://thefutureofstorage.com/archives/15

Another TIC Insight on SAN Market from Mike Kramer:
http://thefutureofstorage.com/archives/36

SAN Market relinquished by big players. Estimated value of $13 Billion:
http://masshightech.bizjournals.com/masshightech/stories/2004/06/28/story5.html

IBM Whitepaper - Dated but good detail on several some Storage Area Network issues:
http://www.redbooks.ibm.com/redbooks/pdfs/sg245470.pdf

 

One Response to “SAN and Deduplication”

  1. Stephen Foskett Says:

    Joseph,

    You hit on a key point: Deduplication is absolutely critical to control growth in certain applications like backup and archiving where duplication is a fact of life. It may never yield massive benefits in the mainstream primary storage world, but some situations can really see huge savings with the tech. And it’s here - mainstream all the way with every vendor now offering some dedupe technology.

    But, as with the case of all tech, caveat emptor. Make sure you see your solution in action with realistic data loads before you buy. Rumor has it that many current dedupe solutions don’t scale well…

    Stephen

Leave a Reply