I've been noticing recently that storage is becoming a problem. By that, I don't mean that storage itself is a problem, or acquiring storage is a problem. Rather, I mean that storage management is a problem.

Before we get into why it's a problem, let's backup just a bit an  look at the evolution of storage over the past few years to see how suddenly, we find ourselves knee deep in volumes and switches and management applications still not having enough storage space.

In years past (and yes, this will be a gross over-simplification), you generally had a server with what you thought would be enough internal storage, 3 or 4 disks in a RAID set of some type. Quickly, after a few years, we realized that we needed a LOT more storage that would comfortably fit into the rack mount chassis of our x86 based server. Enter external SCSI storage as the solution. As a solution, it was pretty decent. Throw a bunch of SCSI disks in a cabinet, give the cabinet its own power supply and hook up some servers. Create some LUNs and away you go.

Unfortunately, this solution was (and is) neither very flexible nor very scalable. Enter SANs and fibre (yes, that's the correct way to spell "fibre" when referring to storage connectivity) storage. SAN solutions allow the use of switched to route traffic so that many machines (servers) can see individual LUNs without worrying about which drives are mine, yours, etc. SANs give you the capability to  connect dozens of servers to the same switch and setup zones defining which servers could see which disks, a major leap forward in technology that SCSI did not have. With technology moving forward, we also see in SANs the ability to create a raid set and carve multiple LUNs out of that, or to create disks using stripes of all disks, not just defined raid sets. Yes, SANs definitely were a leap forward in the technology area.

Now for the bad news. SANs are a total PITA to manage. I cannot tell you how many issues I've seen with zoning, presenting the wrong disks to the wrong server, presenting the same disk down multiple paths to the same server (or multiple servers) simultaneously, incompatible firmware between the HBA, the switch and storage processor….the list goes on and on. All of the flexibility and tuning ability came at the expense of management overhead, where one mistake (incorrect firmware on a switch) can toast all of your servers attached to the SAN.

Now, enter the beauty and the joy that is iSCSI. This is a major advancement in storage technology. iSCSI basically allows you to use the SCSI protocol over IP. On the surface, that doesn’t really sound like much, except a new transport media for the SCSI protocol (yes, SCSI is a protocol, defining disk communication as well as a hardware spec), but, it's much much much more than that.

In order to connect to SCSI storage, you no longer need switches, expensive HBAs (fibre HBA - $1200 for the latest model. Gigabit adapter, $110, both major brands). You now have the ability to freely move servers around without having to re-zone or worry about cabling. You can have the same server connected to multiple SCSI targets at the same time without multiple HBAs, switches or zoning (for instance, the XP machine I am writing this on is connected to a NetApp Filer and a Windows 2003 machine using the StringBean WinTarget (http://www.stringbeansoftware.com) software to share disks out as SCSI devices. Do I need to HBAs? Why no I don't. All I need is one network card and my Windows XP SP2 workstation is simultaneously connected to two separate SCSI targets. The NetApp filer is being used for a cluster tested for several of our clusters here. I have Virtual Server running on my desktop machine, loading the VMs I have stored on the 2003 server I am connected to as an iSCSI target, and the VMs are using iSCSI to connect to their cluster drives on the NetApp box (which, BTW, is a pretty darn cool little box. I haven't found anything I can't easily do. From start to finish, never having seen one EVER, it took me 7 hours to set it up and get it running the way we want). Basically, each machine gets a unique iSCSI initiator ID (part of that is a hash of the machine name) and you zone by creating a LUN and in a text window, simply popping in then initiator ID's that have access to that LUNs. It's even web based so you can cut-n-paste and manage it from anywhere on your network.

Speed? Is iSCSI fast? Check out these numbers from Equalogic, http://www.equallogic.com/docs/EqualLogic_ESG_Lab_Report_04.pdf. They are reporting numbers like "25 array Peer Storage system using Microsoft iSCSI Software initiator produced 1.2 million IOPs out of cache". Basically, it's fast, cheap, flexible and good. Especially if you used gigabit and MPIO.

So far, I haven't found ANYTHING that iSCSI can't do to meet my storage need. iSCSI is the stuff,  I say. Please, download the iSCSI initiator, whitepapers, case studies, Kool-Aid, etc, at http://www.microsoft.com/windowsserversystem/storage/technologies/iscsi/default.mspx, because it's the stuff.