While these concepts and decisions should be fairly basic, I thought I would throw them out there and see if I either get some feedback from others who have gone through some of the same decision making processes, or possibly help out someone who is going through it right now.

As I worked on the design of my upcoming server virtualization infrastructure, obviously storage was one of my primary decisions. I already had my primary storage platform in place with various SAN arrays sitting behind a storage virtualization appliance, and I pretty much knew that I would just be expanding on that to support the new virtual machines. Plus, that fit in perfectly with the HA design that was already decided for the VM host machines.  I rolled out two quad proc quad core hosts with 64 GB of RAM each, and then 1 smaller 2 proc quad core machine with some DASD and a decent amount of RAM for a staging box for my P to V migrations.

Next I had to decide on the arrays. I pretty much knew that any physical box that was currently RAID 10, would be RAID 10 as a VM. What I had to decide was if I would just go RAID 10 across the board. Many physical machines were RAID 5, but virtualization adds its own bit of overhead, and there’s never anything wrong with boosting performance on anything. It really wasn’t cost prohibitive in buying the physical disks, but it did present a problem in maxing the physical disk count per controller a little quicker than I liked. Also, my IOPS per $$$ looked better with going all 10, but again, it was going to lead to bringing in additional controllers to support the number of physical disks that were going to be required. So in the end, I did a server by server worksheet, and ended up about half and half. I simply have TIER 1 and TIER 2 disk groups in my storage arrays to support the 2 groups of servers I decided on. Everything is very manageable and scalable, and I felt like I got the best bang for my buck.

read more …

Virtualization is a big buzzword right now, and for good reason.  The more we virtualize, the more flexibility we build into our infrastructure.  What we are really talking about is providing a layer of abstraction between our resources.  The entire IT industry is moving in this direction and it’s fantastic.  Swapping server hardware has never been easier.  Network traffic can be grouped and segmented without extra hardware.  Storage space can move moved, increased, and even migrated to higher performance arrays without downtime.  These added abstractions need to continue and I see every aspect of the data center benefitting from virtualization.

You virtualized what?

Anything is possible.  We can abstract the power from the hardware and circuits, which might allow us to redistribute power as needed to various parts of the data center and ease use of products with heterogeneous power requirements, including the power hungry 10 gigE coming our way.  We would only trip virtual circuits.  Virtual power, virtual cooling, virtual phones, virtually anything is possible.  Virtual IT staff?  Well, that’s called outsourcing the staff and is as useful as virtual money, right?  In the world of IT storage, providing a layer of abstraction between the file system and the operating system should be focused on.  We also need to improve the underlying resilience technology.  With SSDs being fragmented by nature and most of the time being slower on writes than spinning disk, a RAID 1+0 looks pretty good, especially since it’s cheaper per gigabyte too.  A RAID 1+0 could lose half of its disks and still run without issue.

read more …

Many storage challenges focus on correlating the higher-level use of data with the nuts and bolts of storage. These discussions often revolve around the conflict between data management, which demands an ever-smaller unit of management, and storage management, which benefits most from consolidation. Developing storage management capability that is both granular and scalable is one key to the future of storage.

Storage Management: Scaling Up

As I discussed in my last piece, Turning the Page on RAID, the data storage industry has traditionally focused on reducing granularity. Disk capacity has expanded, and RAID technology has multiplied this by combining multiple physical drive mechanisms into a single virtual one. Storage virtualization technologies, from the SAN to the server, have also often been touted primarily as a mechanism to reduce heterogeneity. From a technical perspective, therefore, granularity has been an obstacle to overcome.

The core organizational best practice for storage management is reducing complexity and increasing standardization. Consolidation of storage arrays and file servers is a common goal, as IT seeks to benefit from economies of scale. The goal of both initiatives is the creation of a storage utility or managed storage service.

read more …

Storage area networking is a rapidly evolving and complex topic, and unless your company has staff with a fair level of experience and expertise in storage area networking applications you may want to consider using a consultant that is well versed in issues relating to SAN deployments.

When budgeting for your new storage system or changes to an existing SAN, you’ll want to factor in not only hardware costs but also the likely consulting costs to fully deploy the new systems.   Even for a fully in-house solution there will likely be costs outside of just the new hardware, and these will vary depending on vendors, solutions, and sales negotiating ability.

read more …

[NETAPP] NetApp has demonstrated FCoE Target for a while.

[EMC] The recent CLARiiON CX4 overhaul clearly paved the way to support FCoE target controller sometime in 2009.

[HP] The data storage blog (http://www.communities.hp.com/online/blogs/datastorage/archive/2008/03/04/HPPost5865.aspx) certainly concludes they embrace FCoE.

[IBM] One of the T11 committee members driving hard behind FCoE standard. I don’t see why they wouldn’t jump on and work out native FCoE target.

[HDS] “Moving to FCoE for storage arrays would only require the replacement of the front end port modules….it is expected to be much easier for HDS to convert to FCoE.” from Hu Yoshida’s latest blog (http://blogs.hds.com/hu/2008/08/storage_network_unification.html).

A year ago, I had doubts whether or not these storage vendors would take native FCoE seriously. Now I have no doubt that they, at least some of them, will start to support FCoE within the next two years.

It seems very clear they decide to move down FCoE path.

Continuing from my previous article which introduced Converged Enhanced Ethernet, lets look at how we can converge our storage and data networks and have a single fabric in the data centre.

The value of HBAs

In this plan, your network adapters and drivers (HBAs) must respond to these flow control messages to stop or slow data transmission so as to avoid congestion in the network and subsequent packet loss. Thus HBAs will becomes critical to the success of iSCSI or FCoE because they will need to have large and smart buffering schemes to be able to handle the data pumping. I have talked in previous articles about the value of HBAs in your servers and this is another example of their value.

read more …

A key issue around the development Converged Enhanced Ethernet(CEE) (also known as Data Centre Ethernet in Cisco marketing because of their proprietary extensions), is that CEE will enable iSCSI to reach it’s full potential. For nearly everyone, iSCSI will become the default the technology once the CEE standards are finalised, and products come to market. What ? You haven’t heard this before ? Lets have a quick intro to CEE.

What are we trying to solve ?

Existing applications are tolerant to packet loss in the network, and QoS is managed (but not solved) by classifying traffic and managing queues in the network (where queues exit). However, Storage is very sensitive to loss, both FC and iSCSI can retransmit lost data, but this causes significant bottlenecks and a performance hit to the application.

In a modern data centre, attempting to QoS storage AND voice AND other high value traffic is a practical impossibility. The configuration and maintenance of a converged backbone with competing requirements would be unsuccessful in enough cases that the market would reject any such attempt.

The obvious thing is to stop packet loss. The only practical way to achieve this is to create some signalling that notifies the sender to stop sending data before the loss or congestion occurs. Congestion is always possible in ANY NETWORK no matter how much bandwidth you provision.

read more …

Thanks to the overwhelming benefits most IT departments have already deployed virtualization across some of their infrastructure with plans to increase the level of virtualization in the near future. According to the EMA study cited below the adoption of virtualization is growing at about 25% per year with over 95% of all enterprises using some form of virtualization already.

The cost benefits of virtualization can be exceptional but the process also provides for superior availability, disaster recovery, and load balancing.  It also facilitates faster software deployment and development as those processes are disconnected from physical hardware limitations.   Virtualization also reduces downtime and the complications associated with hardware failure.

read more …