Failover Clustering and Network Load Balancing Team Blog
Hi cluster fans,
By now you have probably heard of the new architectural innovation being brought to Failover Clustering in Windows Server 2008 R2 known as Cluster Shared Volumes, or CSV. CSV functions as a distributed-access file system optimized for Hyper-V. A comparison would be to a clustered file system, however, unlike other clustered file systems, CSV does not use any proprietary technology – it uses standard NTFS, so there is nothing special you need to purchase or support – it just works! If your storage is suitable as a standard clustered disk, it can be used as a Cluster Shared Volume. In the past, only one node could host a virtual machine (VM) and access the VHD on the shared storage, so if another node needed to host the VM or access the disk, it would need to failover and it would affect every resource on that shared disk. With CSV in R2, any node can host the VM and any node can access the VHD on shared storage, so VM and disk ownership can move freely across cluster nodes without impacting any other resources on that shared disk.
CSV will provide many benefits, including easier storage management, greater resiliency to failures, the ability to store many VMs on a single LUN and have them fail over individually, and most notably, CSV provides the infrastructure to support and enhance live migration of Hyper-V virtual machines. This blog post will cover configuring and deploying Hyper-V VMs using CSV in a Windows Server 2008 R2 Failover Cluster. We will discuss other benefits and features of CSV and live migration in future posts. All information is based on the Beta 7000 build of Windows Server 2008 R2 and is subject to change for the RTM version of this product.
Please report any bugs through the Microsoft Connect site or send us feedback by clicking the ‘Email’ link in the upper-left corner of this site.
Thanks,Symon PerrimanProgram ManagerClustering & High-Availability
To configure CSV we must first build the cluster. There is nothing different which needs to be done for CSV. R2 clustering will still support iSCSI, Fibre Channel and Serial Attached SCSI (SAS) for storage. CSV will work with any of these, so long as the disk is using NTFS as the file system.
[Update 3/31/10] For best practices regarding Hyper-V networking and NICs based on RTM behavior, please refer to this documentation: http://technet.microsoft.com/en-us/library/ff428137(WS.10).aspx
For the networks, it is still recommended to have a public network for client connections. Additionally, we recommend a dedicated network for CSV, internal communication and 'heartbeat' traffic, and another NIC for live migration traffic which should be at least 1 GB. This is to ensure that the heartbeat network does not get flooded by live migration traffic and miss health-checks which would cause a failover. If you are using iSCSI you will need another NIC for that iSCSI network.
Once you have verified that the hardware you will use in the cluster supports Hyper-V, you install the Failover Clustering feature and the Hyper-V role from Server Manager on every node in the cluster. Due to the way which CSV disks are accessed, it is recommended that you use the same drive letter for your OS on every cluster node (such as C:\).
Once the hardware is connected and the correct roles and features are installed, you should ensure that the entire clustering configuration is supported. This is done by running the built-in Validate a Configuration tool (Guide: Validating Hardware for a Failover Cluster). So long as every component in the solution has a logo for Windows Server 2008 R2 and it passes ‘Validate’ then it is supported for CSV. Alternatively, you can purchase a pre-Validated solution from one of our partners in the Failover Clustering Configuration Program.
After the solution has passed ‘Validate,’ create a cluster (Guide: Creating a Failover Cluster)
To enable Cluster Shared Volumes (CSV) on a cluster, click on that cluster name in the navigation pane in the Failover Cluster Manager MMC snap-in. In the center pane, click the “Enable Cluster Shared Volumes…” link. A notification appears reminding you that CSV is only for use with Hyper-V. The cluster now supports CSV.
A new node appears in the navigation pane for Cluster Shared Volumes:
Now that CSV is enabled, create some CSV disks.
A. Select the Cluster Shared Volumes node in the navigation pane in the Failover Cluster Manager.
B. In the right Actions pane, select “Add storage”. This will bring up a window that shows all the disks in the Available Storage group. Check the disks you want to add, then select “OK”
C. The new disk(s) now appear in the list for Cluster Shared Volumes:
CSV allows every cluster node to access the disk concurrently. This is accomplished by creating a common namespace under %SystemDrive%\ClusterStorage. For this reason, it is necessary to have the OS on the same drive letter on every node in the cluster (such as C:\, which will be used in this blog). You will see the same directory from every node in the cluster and this is the way to access CSV disks.
Each CSV disk has its own volume in the directory and is assigned the default name, VolumeX, for each disk. In this example we have 3 CSV disks so we see 3 folders. The C:\ClusterStorage directory must keep the same name, however the volumes within this directory can be renamed.
You will need to copy your VHDs onto these directories to create highly-available VMs. CSV will support dynamically-expanding, fixed-sized, and differencing VHDs. CSV will not support pass-through disks.
After enabling CSV and placing the VHDs on the CSV disks, we are ready to make those virtual machines highly-available.
A. Open Hyper-V Manager
B. From the Action pane, select “New” and then “Virtual Machine”. This will open the New Virtual Machine Wizard.
C. Provide a name for the Virtual Machine and check-mark the box for “Store the virtual machine in a different location”, then specify the path under “C:\ClusterStorage\” for the volume that you want the VM to use.
D. Specify the memory for the VM, then “Next >”
E. Specify the network for the VM, then “Next >”
F. In the “Connect Virtual Hard Disk” page of the wizard, if you specify either “Create a virtual hard disk” or “Use and existing virtual hard disk”, specify a path to a Cluster Shared Volume under C:\ClusterStorage\
G. Select “Next >”
H. Choose the desired options in the “Installation Options” wizard page. Select “Next >”
I. Select “Finish” at the Summary Page of the Wizard
Now that we have created our VMs on our CSV disks, we can make them highly-available so they can be managed by the cluster and can fail over.
A. Open Failover Cluster Manager
B. In the left pane, select “Services and Application”
C. In the Action pane, select “Configure a Service or Application”. This will open the “High Availability Wizard”
D. Select “Virtual Machine”, then select “Next >”
E. Check the Virtual Machine that you want to add to the Failover Cluster. (Note, the VM must be shutdown/turned off to be added) Select “Next >”
Failover Clustering will determine if this VM is using CSV or standard cluster disks based on the path of the VHD (it will look for the %SystemDrive%\ClusterStorage path).
F. Review the Confirmation page in the wizard and select “Next >”
G. Review the Summary page in the wizard. If the status is not “Success”, review the report by selecting the “View Report...” button and investigate the information for possible causes. Otherwise, select “Finish”
The Virtual Machines will be shown in the left pane of Failover Cluster Manager. They are automatically given the Name “Virtual Machine” with a number to differentiate them. These names can be changed by right clicking on them and selecting “rename”.
Start your VMs when you are ready.
You now have your CSV cluster up and running with no special hardware or additional considerations. You can manage the Virtual Machine resource like any standard resource, such as changing properties, creating dependencies, and performing failovers. You will notice that for VMs you can now perform a live migration which allows you to move a running VM from one cluster node to another cluster node without any client disruption. This keeps a VM highly-available even when the VM moves between different physical machines.
To perform a live migration, select the ‘Live migrate this virtual machine’ button and pick a target in the right Actions pane in the Failover Cluster Manager snap-in. This will perform the live migration and its status will be indicated in the center information pane.
You can have CSV without live migration and you can perform a live migration without CSV, however, these are complementary technologies which enhance each other. We will look at the functions of each in more detail in later blog posts.
· Whitepaper: Windows Server 2008 R2 & Microsoft Hyper-V Server 2008 R2 - Hyper-V Live Migration Overview & Architecture: http://www.microsoft.com/downloads/details.aspx?FamilyID=fdd083c6-3fc7-470b-8569-7e6a19fb0fdf&displaylang=en
· Step-by-Step Guide for Hyper-V: Using Live Migration in Windows Server 2008 R2: http://technet.microsoft.com/en-us/library/dd446679.aspx
Please check out our recent blog posts on copying files to CSV which may explain the behavior you are seeing: http://blogs.msdn.com/clustering/archive/2009/12/09/9934381.aspx
If you are still having trouble, please post your question to the 2008 R2 Failover Clustering forum: http://social.technet.microsoft.com/Forums/en-US/windowsserver2008r2highavailability/threads/
It's unclear how you set up the recommendation to "Additionally, we recommend a dedicated network for CSV and live migration traffic which should be at least 1 GB."
The CSV network uses the lowest metric and LiveMigration the second lowest metric. How do you separate out the heartbeat network from the CSV network? I don't see any way of doing this. How do you tell the cluster "for heartbeats use this network". I don't see how and reading this blog was a reason we bought another NICs to separate out heartbeat, CSV and Livemigration on their own separate NICs.
Here is the official documenation regarding NICs in a cluster. The multiple NICs are useful for separating: Internal cluster traffic, live migration traffic, hyper-v management traffic, public traffic and iSCSI traffic (if applicable).
I'll edit the blog to add this link for RTM behavior.
Thanks. I just was unclear how clustering in 2008 determined the "heartbeat" network. But it appears it sends the heartbeat on ALL NICs that the cluster is allowed to send network traffic on. Therefore there is redundancy if you have at least 2 cluster networks that the cluster can send on.
Autostart disabled VMs are not visible in Hyperv Manager after restart of the Host
-Having a cluster of two nodes
-Having some VMs clustered(meaning Highly Available) and others are in the in the individual nodes
-Changing the Autostart property to disabled for a couple of clustered VMs
-Reboot both the nodes
-Now the VMs which have Autostart property disabled are not visible in the Hyperv Manager wizard. But they still exist in the cluster.
Wondering why this is happening. Is it a bug on Hyperv Clustering?
Could you explain what is really happening in this scenario mentioned above?
Help would be highly appreciated.
Hyper-V Manager does not have cluster awareness, so you need to manage your highly-available VMs through Failover Cluster Manager. This is expected behavior.
Thanks for the reply, msclustm! Yeah, It seems to be normal behavior.
I have the exact same problem running Windows 2008 R2.
"The copy process takes forever and it says "5 seconds remaining". Even Explorer became unresponsive, and is shown as "Not Reponding" on task manager. I then restarted all of the nodes and the storage, and everything comes up ok again. If I try to copy any VHD file to the Cluster Storage, the problem is presented again, and everything is slow or unresponsive, until I restart all of the Cluster Nodes."
Any ideas folks ?
This issue can be resolved by copying the data from the Coordinator Node: blogs.msdn.com/.../9934381.aspx
at step 3 (Create CSV Disks) I get an error saying no disks are available. Which is funny, because under the Storage node I see my Cluster Disk 1. CSV enabled just fine.. why cant I add my disk 1? I understand the disk must be clean.. my Disk 1 contains only the Cluster folder which appeared after the cluster was succesfully created. Do I need a 2nd LUN to use CSV?
Is there a similar solution to Shared file systems for non Hyper V - stand alone servers that need to share files in an Active / Active construct attached to SAN?
Yes - in Windows Server 2012, AKA "Windows Server 8", we have improved CSV, and it supports Continuously Available Scale-Out File Server as well as Hyper-V scenarios... These are the two biggest scenarios for CSV in Windows Server 2012, but please note that CSV is a great match for ANY scenario that uses shared storage, because it can provide additional resiliency in the face of node, network, or storage interface failure.
My colleague Jose Barreto has a great blog for Scale-Out File Services scenarios here: blogs.technet.com/.../josebda
What if we simply need to have shared volumes containing files only in a clustered type environment. This is a basic 2 server, 1 DAS env. Currently we only have the LUNs mapped to one server and if that server fails we have to change the mapping to the other host manually. Both servers are direct attached to the storage unit with SAS. Is this possible using the built in clustering software?