Clustering and High-Availability

Free Private Cloud Training: Are you ready for the Microsoft Private Cloud?

Rob-MSFT — Thu, 19 Jan 2012 20:24:49 GMT

On February 21 & 22 Microsoft Learning is running a 2-day virtual training event to help the world learn about the upcoming enhancements with the Creating & Managing a Private Cloud with System Center 2012 Jump Start.

It is 100% free and open to the public, so register now!

Event Overview

Adopting this exciting new computing paradigm provides a whole new landscape of technology and career direction for IT professionals. Microsoft Learning and the Microsoft System Center 2012 team have partnered to bring you an exciting opportunity to learn what you need to know to deploy, manage and maintain Microsoft’s private cloud solution. Leveraging the popular Jump Start virtual classroom approach, the industry’s most gifted cloud experts will show attendees why this new private cloud solution, based on System Center 2012 and Windows Server, has garnered so much attention. Presenters include Symon Perriman, Sean Christensen, Adam Hall, Kenon Owens, Prabu Rambadran & Chris Van Wesep and there will be a live Q&A during the event.

Event Agenda

Day 1: Deployment & Configuration (Feb. 21)

Part 1: Understanding the Microsoft Private Cloud
Part 2: Deploying the Infrastructure Components
Part 3: Deploying the Private Cloud Infrastructure
Part 4: Deploying the Service Layer
Part 5: Deploying the Applications & VMs

Day 2: Management & Operations (Feb. 22)

Part 6: Managing the Infrastructure Components
Part 7: Managing the Private Cloud Infrastructure
Part 8: Managing the Service Layer
Part 9: Managing the Applications & VMs

Jump Start Overview

This accelerated Jump Start sponsored by Microsoft Learning is tailored for IT professionals familiar with Windows Server technologies, Hyper-V virtualization, and the System Center management solutions. The course is designed to provide a fast-paced and technical understanding of how and why Microsoft’s approach to the private cloud delivers scalability, security, flexibility and control. Here are few unique benefits of this course:

Students have the opportunity to learn from and interact with the industry’s best cloud technologists!
This high-energy, demo-rich learning experience will help IT Professionals understand why Microsoft private cloud solutions are making a splash in the industry.
Students will see with their own eyes how Windows Server 2008 R2 and System Center 2012 work together to provide the best combination of security and scale.
Information-packed agenda! Day one of this two-day online course will focus on designing and deploying the right solutions for your organization, while day two will provide an in-depth look at the tools available to help monitor, secure and control the operational aspects of a private cloud.

Thanks!
Symon Perriman

Technical Evangelist, Microsoft

TechNet Edge Blog & Videos

Twitter @SymonPerriman

Free Online Training: Understanding Microsoft's High-Availability Solutions

Rob-MSFT — Thu, 15 Dec 2011 19:34:49 GMT

Hi Cluster Fans,

We recently launched the free & public Microsoft Virtual Academy (www.MicrosoftVirtualAcademy.com). This training portal offers quick and easy access to presentations, webcasts, whitepapers and other technical content about Microsoft’s technologies.

This week we just released a brand new track on Understanding Microsoft’s High-Availability Solutions.

This course covers the best ways to provide continual availability to all applications, services, servers and VMs in your datacenter. The modules will cover all the planning, deployment and management considerations for bringing high-availability to every components of your datacenter or Private Cloud. By the end of this course you will have an understanding of the basics of each of Microsoft's HA solutions and when each of them should be used. You will understand all the different methods you can use to keep your services up and running. Technologies that will be covered in this course include: Hyper-V, Failover Clustering, Network Load Balancing, SQL Server, Exchange Server, IIS, DFS-R, DNS, and every System Center product (including SC 2012) with a focus on Virtual Machine Manager (VMM)

The course includes:

Understanding Microsoft’s High-Availability Solutions
- Infrastructure HA with Failover Clustering & Hyper-V
- Application HA with SQL, Exchange & Other Servers
- Management HA with System Center
Failover Clustering Overview
- Introduction to Failover Clustering
- Server High-Availability
High-Availability & Clustering
- Overview, Software & Hardware
- Deployment & Management
- Hyper-V HA
Disaster Recovery

Introduction to Disaster Recovery
Storage Considerations

Thanks!
Symon Perriman

Technical Evangelist for IT Professionals

Free Clustering & HA Training for Hyper-V & Private Cloud

Elden Christensen — Wed, 30 Nov 2011 19:06:00 GMT

Next Tuesday, December 6^th from 8am-11am PST, we are offering a free training: Failover Clustering with Hyper-V: Designing a Highly-Available Infrastructure for the Private Cloud

Join Symon Perriman (Technical Evangelist for Private Cloud) and Elden Christensen (Principal Program Manager Lead for Clustering) to understand how Windows Server 2008 R2 Failover Clustering with Hyper-V provides critical infrastructure for any datacenter with VM high-availability and mobility. This event will provide an introduction to clustering, then focus on the design, deployment and management considerations for your virtualized datacenter or Private Cloud. Topics include hardware, validation, deployment, host clustering, guest clustering, virtualization, live migration, multi-site clustering, System Center Virtual Machine Manager 2008 & 2012, and offers a wealth of best practices. No prior clustering knowledge is needed.

Agenda:

· Introduction to Clustering – Learn the basics of Windows Server 2008 R2 Failover Clustering
· Hyper-V High-Availability – Dive into best practices, tips and tricks to provide VM high-availability and mobility
· Private Cloud High-Availability – Enhance the datacenter with integration from SCVMM (2008 R2 and 2012) and multi-site clustering

Thanks!
Symon Perriman

Technical Evangelist for IT Professionals

Private Cloud | Server | Virtualization | System Center

TechNet Edge Blog & Videos Twitter @SymonPerriman

Virtualization & Clustering Exam Training

Elden Christensen — Wed, 30 Nov 2011 19:04:03 GMT

On Thursday December 1^st and Wednesday January 11^th, Technical Evangelist Symon Perriman and Technical Instructor Philip Helsel will host an online 8-hour deep dive training event for the Microsoft 70-659 Technical Specialist exam, Windows Server 2008 R2, Server Virtualization. This virtual event runs for 8 hours at different times and will include presentations, demos and live Q&A with the attendees. It costs $99 to attend, but includes a free exam voucher worth $150! It is public so sign up for some great training to help improve your career here: http://mctreadiness.com/MicrosoftCareerConferenceRegistration.aspx?pid=274. Register early to make the most of a weekly virtualization coaching newsletter!

· Thursday, December 1, 2011 from 10pm – 6am PST (perfect for students in Asia and Europe)
· Wednesday, January 11, 2011 from 12pm – 8pm PST
· More times coming soon…

Clustering will be featured in Modules 1 and 4, where we will talk about cluster infrastructure planning, networking, storage, Cluster Shared Volumes, live migration, cluster integration with System Center Virtual Machine Manager and more!

Module 1 – Installing and Configuring Host and Parent Settings
Module 2 – Configuring Child Settings
Module 3 – Managing and Monitoring Virtual Environments
Module 4 – Ensuring High Availability and Recoverability
Module 5 – Performing Migration
Module 6 – Configuring Remote Desktop (RD) Role Services Infrastructure

Technologies that we will cover include: Windows Server 2008 R2, Hyper-V, System Center Virtual Machine Manager (VMM), System Center Operations Manager (OpsMgr), System Center Data Protection Manager (DPM), Windows Server Backup, Failover Clustering, Remote Desktop Services, Active Directory, Microsoft Assessment & Planning Toolkit (MAP), Virtual Machine Servicing Tool (VMST), Windows Server Update Services (WSUS) & more!

Learn More & Register: http://mctreadiness.com/MicrosoftCareerConferenceRegistration.aspx?pid=274

Exam Information: http://www.microsoft.com/learning/en/us/Exam.aspx?ID=70-659

Thanks!
Symon Perriman
Technical Evangelist
Private Cloud Technologies
Microsoft

See videos on TechNet

Follow announcements @SymonPerriman

Virtualization Career Training Opportunities

Rob-MSFT — Tue, 27 Sep 2011 16:40:00 GMT

Virtualization Career Training

On Tuesday, October 4^th Technical Evangelists Symon Perriman and Rick Claus are hosting an online conference on Virtualization Career Training with Microsoft Learning. This half day virtual event (8am – 11am PST) will offer a Level 100 to 200 introduction for anyone who wants to learn more about Microsoft Virtualization and how it can help their career. It is free and public so sign up for this warm-up for the Jump Start event on October 6^th.

Module 1 – Technology: Learn about Microsoft’s virtualization technologies, how they work, and the future roadmap to the Cloud!
Module 2 – Career: Understand the importance of virtualization and Private Cloud, and how it can make or break an IT Professional’s career!
Module 3 – Certification: Get prepared for your next steps towards a virtualization career by understanding and preparing for the Microsoft 70-659 Technical Specialist exam, Windows Server 2008 R2, Server Virtualization.

Learn More: http://mctreadiness.com/MicrosoftCareerConferenceRegistration.aspx?pid=270
Register: http://mctreadiness.com/MicrosoftCareerConferenceRegistration.aspx?pid=288

Instructor Bios: http://mctreadiness.com/MicrosoftCareerConferenceRegistration.aspx?pid=287

Virtualization Exam 70-659 Training

On Thursday October 6^th Technical Evangelist Symon Perriman and Technical Instructor Philip Helsel will host an online 8-hour deep dive training event for the Microsoft 70-659 Technical Specialist exam, Windows Server 2008 R2, Server Virtualization. This virtual event runs from 8am to 5pm PST and will include presentations, demos and live Q&A with the attendees. It costs $99 to attend, but includes a free exam voucher worth $150! It is public so sign up for some great training to help improve your career here: http://mctreadiness.com/MicrosoftCareerConferenceRegistration.aspx?pid=272. Register early to make the most of a weekly virtualization coaching newsletter!

Module 1 – Installing and Configuring Host and Parent Settings
Module 2 – Configuring Child Settings
Module 3 – Managing and Monitoring Virtual Environments
Module 4 – Ensuring High Availability and Recoverability
Module 5 – Performing Migration
Module 6 – Configuring Remote Desktop (RD) Role Services Infrastructure

Learn More: http://mctreadiness.com/MicrosoftCareerConferenceRegistration.aspx?pid=274

Exam Information: http://www.microsoft.com/learning/en/us/Exam.aspx?ID=70-659

Configuring IP Addresses and Dependencies for Multi-Subnet Clusters - Part III

Rob-MSFT — Thu, 01 Sep 2011 06:50:00 GMT

Hi Cluster Fans,

This blog post will show the configuration of IP Addresses on multi-subnet clusters for the cluster’s network name, or “Client Access Point” (CAP). This is different from the IP Addresses configuration for other clustered workloads as there is no way to add resource to the CAP via the GUI, it must be done using PowerShell (or Cluster.exe).

First, check out Part 1 of the series which showed how to configure IP Addresses for clustered workloads using the GUI. Part 2 covers some more advanced settings such as Possible Owners. To see a video walkthrough of the multi-site deployment for this cluster, visit: http://technet.microsoft.com/en-us/edge/disaster-recovery-cluster-deployment-demo-multi-site-failover-clustering.

In this scenario I have a 2-node cluster, DRCluster, and I have just added an additional two nodes at my second site. If all four nodes were available when the cluster was created, the IP Addresses on both subnets for the CAP would have automatically been created. However in this scenario the nodes in the second subnet were added later, so the new IP Address for the CAP must be manually created.

Overview

First let’s take a look at the GUI for the ‘Core Cluster Resources’ (CCR) which contains the CAP. This can be done by selecting the cluster in the left navigation pane and expanding the ‘Core Cluster Resources’ section.

Adding a new IP Address to the Group

To add a new IP Address to the CCR we must use PowerShell (or Cluster.exe). We will launch PowerShell as an administrator and import the clustering module:

PS > Import-Module FailoverClusters

Next we will find the correct name of the CCR by running the cmdlet:

PS > Get-ClusterGroup

We see this is simply referred to as ‘Cluster Group’:

Next we want to add a new IP Address which is a type of cluster resource. Do not forget that PowerShell has great built-in help with examples, so even if you are not sure what the correct syntax is, you can use Get-Help <cmdlet name> -full to see all the details. For just the examples, use Get-Help <cmdlet name> -examples. This is very helpful in this case, giving the exact example we want to use:

To add an IP Address to the CCR we use the Add-ClusterResource cmdlet:

PS > Add-ClusterResource –Name NewIP –ResourceType “IP Address” –Group “Cluster Group”

We see that this was successful and the resource is in the Offline state:

This new IP Address is now visible in the GUI:

Configuring the New IP Address

To continue configuring this using PowerShell, you would use the Set-ClusterParameter cmdlet and Set-ClusterResourceDependency. Once again you can use Get-Help <cmdlet name> to get additional guidance. However that is not necessary as it is now possible for us to continue configuring the IP Address and set the dependencies from the GUI.

Right-click on the new IP Address and select ‘Properties’. Under the ‘General’ tab you are able to select the subnet that it is on (10.* for this scenario), as well as whether it uses DHCP or static IP Addresses:

After clicking ‘Apply’ and exiting you will see that the IP Address remains offline. If a static IP Address has been assigned it should be displayed. If DHCP has been selected, then no IP Address will show.

To immediately request a DHCP IP Address, right-click on the IP Address and select ‘Bring this Resource Online’. It will fail and an error message will be displayed. This is expected since the CCR is not on that subnet, however it will grab the IP Address that it will want to use at this time. Alternatively you can just wait until the first failover to the other subnet to get this IP Address. In our case we have been given 10.121.27.199.

Configuring Dependencies

Finally we will make the CAP dependent on either of the two IP Addresses. This will be a ‘OR’ dependency because we want the CAP to be available so long as an IP address on either network is available.

To do this, right-click on the CAP and select ‘Properties’, then select the ‘Dependencies’ tab.

Select the ‘Click here to add a dependency’ text.

From the dropdown in the ‘AND/OR’ column select ‘OR’ since this will be an ‘OR’ dependency type, then from the dropdown in the ‘Resource’ column select the new IP Address:

Finally click Apply, and when you return to the Failover Cluster Manager GUI you can see that it has been added to the CAP group:

It is in an offline state which is expected since the CAP is still on the 157.* subnet.

Testing Failover

We want to make sure that the new IP Address for the CAP will come online on the new cluster. However in the GUI there is no way to move this resource group to another node (other than stopping the cluster service of the node it is currently on).

To do this in PowerShell we use the Move-ClusterGroup cmdlet and specify a node on the other subnet.

PS > Move-ClusterGroup “Cluster Group” –node DRCluster-2a

In the GUI you will now see that the CAP has changed subnets. It is now online on the 10.* subnet and offline on the 157.* subnet:

This should help you understand how to successfully figure IP Addresses for the CAP in multi-subnet clusters. Syntax for these operations using Cluster.exe is available here: http://technet.microsoft.com/en-us/library/cc736767(WS.10).aspx

Thanks!

Symon Perriman
Technical Evangelist

Private Cloud Technologies

Microsoft
@SymonPerriman

Identifying Stale Cluster Computer Objects

Elden Christensen — Thu, 18 Aug 2011 01:38:00 GMT

On a Windows Server Failover Cluster for each Network Name resource there is a logical name which has a corresponding computer object (CO) created. The computer object associated with the Cluster Name this is commonly referred to as the Cluster Name Object (CNO) and for all other Network Name resources these are commonly referred to as Virtual Computer Objects (VCO). The cluster service creates and manages these CNO and VCO computer objects.

Because CNO and VCO’s are managed by the cluster, they can at times have slightly different behavior than normal computer objects. Understanding these differences can be important when trying to identify when computer objects are stale and no longer being used, so that it is safe to delete them.

If you are running a Windows Server 2008 R2 functional level Active Directory forest, I recommend turning on the Active Directory Recycle Bin feature (which is disabled by default). Here is the step-by-step guide. Active Directory Recycle Bin helps minimize directory service downtime by enhancing your ability to preserve and restore accidentally deleted Active Directory objects without restoring Active Directory data from backups, restarting Active Directory Domain Services (AD DS), or rebooting domain controllers. This is critical for being able to quickly and seamlessly recover from accidental computer object deletion.

There are a couple different attributes commonly used by domain administrators as hints to identify computer objects that are no longer in use and are safe to be deleted:

pwdLastSet – This is the last time the password was changed.
- Windows Server 2003 – The computer object password is not periodically rotated on Windows Server 2003 and below clusters. This attribute should not be used to identify stale objects for deletion on Windows 2000 Server or Windows Server 2003 clusters which have the Enable Kerberos Authentication option turned on. Note that computer objects were not created by default for cluster Network Name resources in these legacy releases.
- Windows Server 2008 – A feature was added in Windows Server 2008 that failover clustering will rotate the CNO and VCO passwords exactly the same as a normal machine computer object. The default domain policy for MaximumPasswordAge will be honored, which is every 30 days by default. Failover Clustering will also honor if DisablePasswordChange is set, to disable password rotation. Password rotation for CNO and VCO's is initiated when the age reaches 75% of expiration to ensure passwords are rotated within the policy, so with the default settings cluster will rotate the password on the 23rd day.
LastLogonTimeStamp – This is the last time a logon occurred. This is a new attribute introduced with a Windows Server 2003 functional level domain.
- When the cluster Network Name resource comes online, it will logon and the LastLogonTimeStamp attributed will be updated. The credentials are cached by clustering and used for the lifespan of the system. If a failure or failover occurs, during the next Network Name resource online a logon will re-occur. When the password for a CNO or VCO is changed per the MaximumPasswordAge policy described above, the credentials are updated and a logon occurs.
  - Using the LastLogonTimeStamp attribute can be an inaccurate hint to identify if a cluster computer object is stale. This is because if a production cluster is up and stable no logon’s will occur and the LastLogonTimeStamp attribute will not be updated until 75% expiration of MaximumPasswordAge. Using LastLogonTimeStamp as a hint can result in accidental CNO and VCO deletion. It is recommended that LastLogonTimeStamp be greater than 30 days (at an absolute minimum) to determine if an object is stale.

Given that CNO and VCO computer objects are associated with mission critical high availability systems, your domain administrators may simply choose to avoid automated / scripted deletion of cluster computer objects. This can be accomplished by querying and excluding computer objects that contain a service principal name (SPN) of “MSClusterVirtualServer” from deletion. Another technique is to move all CNO and VCO's to a custom organizational unit (OU) and exclude that OU from running CO deletion scripts.

As an extra safety mechanism you can protect CNO and VCO's from accidental deletion by simply checking the "Protect object from accidental deletion" box on the objects in the Active Directory Users and Computers snap-in. See this blog for more details

Another hint which can be used to identify stale cluster computer objects is if the computer object is in a Disabled state. By default, when a cluster Network Name resource is deleted or if a cluster is destroyed, the CNO and VCO’s are placed in a disabled state. Any cluster computer object which is in a Disabled state are no longer being used by the cluster.

When destroying a cluster, you may wish to delete the computer objects instead of leaving them in a disabled state. This can be accomplished by passing the –CleanupAD switch to the Remove-Cluster PowerShell cmdlet when destroying the cluster.

If you are reading this after you have already run a script that accidently deleted a CNO or VCO computer object, here is a blog which has a step-by-step guide on how to recover the computer objects:

If you do not have the AD Recycle Bin feature, follow this blog:
http://blogs.technet.com/b/askcore/archive/2009/04/27/recovering-a-deleted-cluster-name-object-cno-in-a-windows-server-2008-failover-cluster.aspx

If you do have the AD Recycle Bin feature, follow this blog:
http://blogs.technet.com/b/askcore/archive/2011/05/18/recovering-a-deleted-cluster-name-object-cno-in-a-windows-server-2008-failover-cluster-part-2.aspx

Thanks!
Elden Christensen
Principal Program Manager Lead
Clustering & High-Availability
Microsoft

Announcement: Upcoming Cluster & Hyper-V Training for User Groups

Rob-MSFT — Thu, 07 Jul 2011 21:10:45 GMT

Hi Cluster Fans,

On Thursday, July 14^th we will be giving online training to user groups about Failover Clustering & Hyper-V: Planning your Highly-Available Virtualization Environment. This event is run in collaboration with Microsoft’s User Group Support Services (UGSS) and will be presented by Technical Evangelist Symon Perriman.

This workshop will actually teach you or your user group leads how to redeliver this content, so that you can train others on this topic. It will cover key discussion points, frequently asked questions, common misconceptions, how to create demo environments, and how to deliver several eye-catching demos. There will be a live Q&A to address any questions you have.

The training will cover a range of beginner (Level 200) to intermediate (Level 300) themes related to Windows Server 2008 R2 Failover Clustering, Hyper-V and System Center Virtual Machine Manager 2008 R2 & 2012. Topics include:

Clustering Basics
Guest Clustering & Host Clustering
Deployment & Validation
Optimizing for Hyper-V
Datacenter Management with SCVMM

This will be presented live at two times so anyone around the world can join during their day. The event will run for 90 minutes from 8am – 9:30am PST and again at 6pm – 7:30pm PST on Thursday, July 14^th.

To sign up, simply visit http://www.zoomerang.com/Survey/WEB22CJ93QZY26/, complete the survey, and pick the time that you wish to attend.

See you there!

Symon Perriman

Technical Evangelist

Private Cloud Technologies

Microsoft

@SymonPerriman

Validating a Cluster with Zero Downtime

Rob-MSFT — Tue, 28 Jun 2011 07:22:00 GMT

Hi Cluster Fans,

As you may already know, the requirements for having a supported cluster have been significantly simplified since the release of Windows Server 2008. There is no longer a need to have every component in the configuration listed in the Windows Server catalog as any commodity hardware should now support clustering. However it is required that every component receives a Windows Server 2008 (R2) logo and the entire solution passes the built-in “Validate a Configuration” (“Validate”) suite of tests.

Validate (Step by Step Guide) can be run at any time once the Failover Clustering feature has been installed, including before the cluster has been deployed, during cluster creation and while the cluster is running. In fact, additional tests are executed once the cluster is in use which check that best practices are being followed for the highly-available workloads. However, according to the Microsoft Support Policy for Failover Clusters, the “Validate test should also be run whenever a major component of the cluster is changed or updated. The following are examples…Adding a node to the cluster; Upgrading or replacing the storage hardware; Upgrading the firmware or the driver for host bus adapters (HBAs); Updating the multipathing software or the DSM; Changing or updating any network adapter…” So how can you minimize the impact of Validating a cluster while it is in production?

When the “Validate a Configuration” wizard is launched it offers the choice to run all tests or a subset of tests. With this granularity it is possible to select all the tests which do not impact the cluster and skip those which can impact high-availability for a group. In fact almost all of the tests can be run while the cluster is online, without impacting anything running on the cluster, other than a slight performance hit due to processing the tests themselves. There are five categories of tests:

Cluster Configuration – These tests are only executed on clusters that have been deployed to ensure that best practices are being followed. They provide a simple way to review cluster settings and determine whether they are properly configured.
Inventory – These tests will inventory the hardware, software, and settings (such as network settings) on the servers, and information about the storage.
Network – These tests will ensure that your networks are set up correctly for clustering.
Storage – These tests will analyze the shared cluster storage to check it is behaving correctly and supports the required functions of the cluster.
System Configuration – These tests will check the system software and configuration settings across servers for compatibility.

Across these dozens of tests, only a few of them will impact running cluster workloads and these are all within the storage category, so skipping this entire category is an easy way to avoid disruptive tests. Listing All Disks and Potential Cluster Disks will not impact anything. Validating Disk Access Latency, File System, Microsoft MPIO-based disks, and SCSI device Vital Product Data (VPD) can impact the disk’s performance as tests are performed against the disk which is in use, however they will cause no downtime unless the disk latency becomes so slow that it triggers an alert. Several tests will actually trigger failovers and move the disks and groups to different cluster nodes which will cause downtime, and these include Validating Disk Arbitration, Disk Failover, Multiple Arbitration, SCSI-3 Persistent Reservation, and Simultaneous Failover. So if you want to test a majority of the functionality of your cluster without impacting availability, exclude these tests.

Failover Clustering does come with a built-in safeguard to prevent accidental downtime when running the storage tests in Validate. If the cluster has any online groups when Validate is initiated, and the storage tests remain selected, it will prompt the user for confirmation whether they want to run all the tests (and cause downtime), or to skip testing the disks of any online groups to avoid downtime. If the entire storage category was excluded from being tested, then this prompt is not displayed. This will enable cluster validation with no downtime, but of course it is not complete as some of the tests have been skipped, yet according to the Microsoft Support Policy “the proposed solution must pass the full Validate test.” So what happens if you need to test your storage, yet all of your disks are being used by running workloads? Or how do you test that your Windows Server 2003 storage will work after a migration without actually impacting your production 2003 cluster.

This can be done by simply creating a new cluster disk from the same storage array, exposing it to all nodes and running all tests against just that disk. This gives you the benefits of running Validate against that type of disk to ensure that it will work while not risking any downtime to production workloads. This can be done by running Validate, selecting all the tests, but keeping any running services or applications online.

Now you can feel confident that your disks should work on your new cluster! It is still important to note that the support policy technically requires the entire configuration to be tested (including all disks), so if you have a storage-related issue and have not run Validate against all disks you may be asked to do so. Nevertheless this is still a great approach to Validating your cluster after a configuration change or migration with minimal impact.

Thanks,
Symon Perriman
Technical Evangelist
Private Cloud Technologies
Microsoft

Understanding Software Options for your Datacenter Infrastructure

Rob-MSFT — Fri, 17 Jun 2011 23:59:00 GMT

Hi Cluster Fans,

This blog post will help you understand the different software SKU options you have when planning for a virtualized datacenter, private cloud or any other deployment model which uses Failover Clustering as a critical infrastructure component to enable virtual machine (VM) mobility. Failover Clustering is a high-availability solution which connects servers together to monitor the health of each other, restart services or application when a host crashes or is in maintenance, and allows running VMs to move between hosts with no downtime.

Important: This post will only review the software which is running on physical servers at the host-level. This does not evaluate the software which is running inside virtual machines at the guest-level, which is almost unrestricted.

Overview

Understanding the different options available in each server SKU related to Failover Clustering is important in the decision-making process. Factors for consideration include the pricing model, architecture, number of nodes, virtualization support and licensing, server core support and which other Windows Server Roles & Features can be deployed on the same server.

Windows Server Failover Clustering is considered an “enhanced” feature, so it is generally available in the higher-end SKUs. As a result it is not available in Windows Server Standard, Web, Foundation or HPC Server. So we recommend that you do not use Windows Server Standard, Web, Foundation or HPC Server for your physical infrastructure. More information about comparing the SKUs is here.

Failover Clustering is available in the following SKUs:

Here’s a summary of the SKU selection considerations:

	Hyper-V Server	Enterprise	Datacenter	Itanium	Storage Server
Pricing Model	Free	Servers	Processors	Processors	OEM Only
Number of Nodes	16	16	16	8	16
Virtualization Support	Yes	Yes	Yes	3^rd Party	NA
VM Guest License	0	4	Unlimited	Unlimited	NA
Server Core	Only Core	Yes	Yes	No	No
Roles & Features	Limited	All	All	Limited	Limited

Current licensing information available at: http://www.microsoft.com/windowsserver2008/en/us/licensing-r2.aspx
Current pricing information available at: http://www.microsoft.com/windowsserver2008/en/us/pricing.aspx

Understanding the Editions

Microsoft Hyper-V Server

Hyper-V Server 2008 R2 is a free SKU based on the Server Core installation type. Yes, you heard me, it is completely free and available for download here. Although the host OS is free, it does not come with any licenses for VMs, so these must be purchases separately, whereas other SKUs may come with licenses for the guest OSs. Hyper-V Server is also designed purely as a virtualization platform for the datacenter, meaning that it supports Hyper-V virtualization and Failover Clustering, but does not come with most of the other server infrastructure roles and services, such as Active Directory, File Server, IIS, etc. However the Failover Clustering feature set is complete, including Cluster Shared Volumes (CSV), Live Migration and supports up to 16 nodes. Since it is only based on the Core Installation it also has a smaller attack surface and fewer servicing requirements, which should lead to higher availability. Although some people stay away from using Core due to the lack of GUI, most do not realize that Core installations can be managed with a full GUI using the free Remote Server Administration Tools (RSAT) from a Client machine, or through using System Center Virtual Machine Manager (SCVMM). If Hyper-V Server is deployed in the cluster, all node must run Hyper-V Server or the “Validate a Configuration Wizard” will fail. This is because not every features and role is supported on Server Core, so some resources would not be able to failover to Core nodes.

More information: http://www.microsoft.com/hyper-v-server/en/us/default.aspx

Windows Server Enterprise and Datacenter Edition

A majority of Windows Server 2008 R2 Failover Clustering deployments use either the Enterprise or Datacenter Editions. Enterprise is licensed per-server, while Datacenter is licenses per-processor. Both are identical in clustering support, Core support, other Features and Roles, and both support Hyper-V virtualization. However Datacenter comes with an unlimited number of licenses to run Datacenter inside VM guests, while Enterprise comes with 4 licenses to run inside VM guests for 5 total licenses (1 physical license plus 4 guest licenses). This can be an important consideration as you evaluate how many guest OS licenses you need based on your current and projected future virtualization usage. Datacenter also comes with enhanced hardware support, including: Hot Add Processors, Hot Replace Memory, Hot Replace Processors, and 64 sockets.
More information: http://www.microsoft.com/windowsserver2008/en/us/r2-compare-specs.aspx

Windows Server Itanium Edition & Architecture

Windows Server 2008 R2 Failover Clustering supports 64-bit (x64) and Itanium (ia64) architecture. The previous release, Windows Server 2008 Failover Clustering, also supported 32-bit (x86) architecture however this in no longer supported in Windows Server 2008 R2. Generally, 64-bit deployments are recommended as they support more 3^rd party components, Windows Server features and roles, Core installation, as well as 16 nodes. Itanium is generally used for SQL-specific deployments and Itanium should only be used if there is a specific reason for doing so as it will not be supported in later version of Windows Server. Additionally Itanium has no in-box Hyper-V support since it does not contain a hypervisor which is needed for hardware virtualization, however 3^rd party hypervisors can be used. The host OS license does provide an unlimited number of guest OS licenses for this SKU.
More information: http://www.microsoft.com/windowsserver2008/en/us/2008-IA.aspx

Windows Storage Server

Windows Storage Server 2008 R2 is built upon an optimized release of the reliable and flexible file services of Windows Server 2008 R2 for better file serving performance. It has an integrated deployment experience with Failover Clustering using iSCSI storage technology. However, as it is specifically designed for File Services, it generally does not have virtualization support, so it would not be recommended for a virtualized datacenter or a Private Cloud deployment, and it does not include any VM guest OS licenses. This also means that it would not utilize Cluster Shared Volumes (CSV) as this cluster storage type only supports Hyper-V VMs. While Storage Server does support 16 nodes and most clustering features, there is no Server Core support and most other Server Roles & Features have limited or no functionality. This SKU is available only through OEMs and is targeted for File Server scenarios.
More information: http://www.microsoft.com/windowsserver2008/en/us/wss08.aspx

Pricing Model

The pricing structure varies based on the SKU licensing, and in some cases is free. Remember to consider the total cost of licensing the entire datacenter, and not just the OSs running on the physical infrastructure.

Here is a summary of the costs:

Hyper-V Server 2008 R2 is FREE! However it comes with no licenses for the guest OS, so consider these costs when you plan your deployment.
Windows Server 2008 R2 Enterprise is licensed per server.
Windows Server 2008 R2 Datacenter is licensed per processor.
Windows Server 2008 R2 Itanium is licensed per processor, and is only available through OEMs or Volume Licensing.
Windows Storage Server 2008 R2 is licensed by OEMs only, so the price is determined by the hardware and quantity.

More information: http://www.microsoft.com/windowsserver2008/en/us/pricing.aspx

Number of Nodes

All Windows Server SKUs which support Failover Clustering also support up to 16 nodes in a cluster, except Itanium, which supports up to 8 nodes.
More information: http://www.microsoft.com/windowsserver2008/en/us/r2-compare-specs.aspx

Virtualization Support

Virtualization is becoming considered a critical piece of the datacenter, especially when moving towards a Private Cloud model.

Here is a summary of the virtualization support:

Hyper-V Server 2008 R2 and Windows Server 2008 R2 Enterprise & Datacenter have full virtualization support with Hyper-V.
Windows Server 2008 R2 Itanium requires a 3^rd party hypervisor to support virtualization.
Windows Storage Server 2008 R2 is used for File Services, so it does not have virtualization support.

Virtual Machine Licenses

If the cluster will host Hyper-V Virtual Machines (VMs), licensing for the OS which is running inside the VM is something which should be considered. Do not just think about how many VM licenses you need today, but also consider how many you will need in a few years from now. You can also cluster VMs to make the virtualized services or apps running inside the VMs highly-available, but remember that the guest OS must support Failover Clustering, so you would not be able to use Standard, Web, Foundation or HPC Server editions inside the guest if you wished to cluster it.

Here is a summary of the VM licenses:

Hyper-V Server 2008 R2 comes with no guest OS licenses. Since this is a free host SKU, the guest licenses must be purchased separately.
Windows Server 2008 R2 Enterprise comes with 4 guest OS licenses.
Windows Server 2008 R2 Datacenter comes with unlimited guest OS licenses.
Windows Server 2008 R2 Itanium comes with unlimited guest OS licenses, however it still requires a 3^rd party hypervisor to support virtualization.
Windows Storage Server 2008 R2 is not used for virtualization so it does not come with any guest OS licenses.

More information: http://www.microsoft.com/windowsserver2008/en/us/r2-compare-specs.aspx

Server Core

Server Core is an installation option which provides a minimal environment for running specific server roles and features that reduces the maintenance and management requirements and the attack surface for those server roles and includes no UI. Since there are fewer components, there are fewer required servicing updates, which are frequent with such technologies as Internet Explorer. With fewer reboots and a smaller attack surface, it is possible to maintain higher service availability than the traditional full installation option, so it is recommended for use with clustering. Although many view management using command-line as a limitation, it is actually possible to manage these servers using a UI by connecting to the core servers through Failover Cluster Manager on a full installation version, using Remote Server Administration Tools (RSAT) or System Center Virtual Machine Manager (SCVMM). However not every feature and role is supported on Server Core, so it is essential to check that the planned workload will be supported. SQL Server, for example, is not supported on Server Core. See the link below for a list of supported roles. If Server Core is deployed in the cluster, all node must run Server Core installation or the “Validate a Configuration Wizard” will fail. This is because not every features and role is supported on Server Core, so some resources would not be able to failover to Core nodes in a mixed cluster.

Here is a summary of Server Core support:

Hyper-V Server 2008 R2 only runs on Server Core. There is no other installation option.
Windows Server 2008 R2 Enterprise & Datacenter have Server Core installation options during deployment.
Windows Server 2008 R2 Itanium & Windows Storage Server 2008 R2 do not have a Server Core installation option.

More information: http://www.microsoft.com/windowsserver2008/en/us/r2-compare-core-installation.aspx

Role & Feature Support

Not all of the SKUs support all the Windows Server Roles and Features which are required to deploy a complete datacenter infrastructure. When planning your software purchases, a single SKU may not meet all of your needs, even if it meets the needs of your highly-available virtualization environment. Server Core, for example, does not support all roles and features, which means that Hyper-V Server could not be used across your entire infrastructure.

Here is a summary of the Windows Server Roles & Features which are supported:

Hyper-V Server 2008 R2 only supports the Hyper-V role and Failover Clustering feature as it is designed as a highly-available virtualization host SKU. It does not support most other Windows Server Roles and Features.
Windows Server 2008 R2 Enterprise & Datacenter supports all roles and features.
Windows Server 2008 R2 Itanium support the Application Server & IIS role and a majority of the features, but it is not comprehensive.
Windows Storage Server 2008 R2 primarily support file-related roles & features, such as File Server (SMB), NFS, DFS, FSRM and FCI.

More information: http://www.microsoft.com/windowsserver2008/en/us/r2-compare-roles.aspx & http://www.microsoft.com/windowsserver2008/en/us/r2-differentiated-features.aspx

Using this information, we hope that you can make a better informed decision when selecting the appropriate software SKU for your datacenter infrastructure.

Thanks!
Symon Perriman
Technical Evangelist
Private Cloud Technologies
Microsoft

Configuring Network Prioritization on a Failover Cluster

Elden Christensen — Fri, 17 Jun 2011 16:53:00 GMT

Hi Cluster Fans,

This blog will describe the Network Prioritization feature and how to configure it. Network Prioritization is used to designate which type of traffic should be directed through which network in a Failover Cluster running Windows Server 2008 R2. When designing a highly-available infrastructure, it is important to avoid any single points of failure, so there should be redundancy in all the hardware components, including the servers, storage and the networking. For this reason we recommend and assume that there are multiple networks in your cluster. If a network is unavailable Windows Server Failover Clustering will automatically direct traffic through another network to maintain service availability for applications or VMs. However some networks may be preferred for certain types of traffic based on the network’s speed, security or function, so it can be important to designate which network is used for which purpose.

Types of Cluster Network Traffic

Understanding the different categories of networking traffic can help you plan the best use for your networks.

Cluster & CSV Traffic

We define cluster traffic as the information needed for the nodes to communicate with each other to ensure that it functions correctly. This can include communication such as heartbeats for health-checking, updates to the cluster database, join requests from partitioned nodes, and much more. Additionally Cluster Shared Volumes (CSV) traffic will use this network for metadata updates, or when it is in redirected mode, meaning that a node hosting a VM cannot directly assess its disk, so it redirects the traffic via another node to that disk. This can be configured by right-clicking on the network in Failover Cluster Manager, selecting Properties, and selecting the radio button for “Allow cluster network communication on this network”.

Live Migration Traffic

If a cluster is taking advantage of the live migration feature to move running Virtual Machines (VMs) between cluster nodes, then live migration traffic is an important consideration. During a live migration, large chunks of memory are copied from one server to another as quickly as possible. This burts the network with heavy traffic, and can block other types of network communication from getting through. For this reason it is recommended to have a dedicated network for live migration traffic so that it does not interfere with other important network traffic. Live migration is commonly used when a physical host needs planned maintenance (such as patching and a reboot), so running VMs are live migrated from that host to avoid any downtime for the VM guests. The faster that this network is, the more traffic can pass through it, thus the host can be evacuated quicker, so it is very common to see the fastest network being dedicated towards live migration. This can be configured by right-clicking on the network in Failover Cluster Manager, selecting Properties, selecting the radio button for “Allow cluster network communication on this network”.

Public Traffic

Whether the cluster is providing high-availability for VMs, SQL databases, File Servers or anything else, a “client” needs to access that application, service or VM. A “client” is defined loosely as a user or application which needs to communicate with that workload running on the cluster. For the client to access that data, they need to make requests and have data sent back to them through those networks. These networks are generally less secure as they are more exposed and accessible, and they could be subject to network flooding, which could impact performance or throughput of traffic. For this reason it is recommended that you do not use this network for cluster traffic, live migration or any other use and to explicitly open it up to clients. This can be configured by right-clicking on the network in Failover Cluster Manager, selecting Properties, selecting the radio button for “Allow cluster network communication on this network” and selecting the checkbox “Allow clients to connect through this network”.

Storage Traffic

If your cluster uses iSCSI or Fibre Channel over Ethernet (FCoE) for the cluster’s shared storage, this traffic goes through an Ethernet network which the cluster will identify as a cluster network. To avoid storage I/O performance being affected with iSCSI or FCoE, it is recommended that you provide a dedicated network for storage traffic so that other network traffic does not interfere with this data. For this reason it is recommended that you do not use this network for cluster traffic, live migration or any other use. This can be configured by right-clicking on the network in Failover Cluster Manager, selecting Properties, and selecting the radio button for “Do not allow cluster network communication on this network”.

Network Prioritization

Since it is a best practice to use multiple networks in your cluster, there is likely the need to specify the function for the various cluster networks. This can be done on Windows Server 2008 R2 through the Network Prioritization (NP) feature. NP will list the order of cluster networks and give the ability for the networks to be “ranked”, where different ranks indicate different network roles. To rank a network, it is given a unique integer from 1 to 268,000,000+, which is called a “metric”.

To view the networks, their metric values, and if they were automatically or manually configured, run the clustering PowerShell cmdlet:
PS > Get-ClusterNetwork | ft Name, Metric, AutoMetric

By default, all internal cluster network will have a metric value starting at 1000 and incrementing by 100. The first internal network which the cluster sees when it first comes online has a metric of 1000, the second has a metric of 1100, etc. We assume that a network is ‘internal’ if it does not have access to a default gateway. The initial list of internal networks is determined by the order which the network adapters were seen by the cluster when it was created.

By default all external cluster network will have a metric value starting at 10000 and incrementing by 100. The first external network which the cluster sees when it first comes online has a metric of 10000, the seconds has a metric of 10100, etc. We assume that a network is ‘external’ if it has access to a default gateway. The initial list of external networks is determined by the order which the network adapters were seen by the cluster when it was created.

The cluster will then use the order of the metrics as the order of networks. The lowest network will be used for “Cluster & CSV Traffic”. The second lowest network will be used for “Live Migration Traffic”. Additional networks with a metric below 10000 will be used as backup networks if the “Cluster & CSV Traffic” or “Live Migration Traffic” networks fail. The lowest network with a value of at least 10000 will be used for “Public Traffic”, and any additional networks with a metric above 10000 will be used as backup networks for “Public Traffic”. Give the highest possible values to any networks which you do not want any cluster or public traffic to go through, such as for “Storage Traffic”, so that they are never used, or only used when no other networks at all are available, depending on your settings.

So let’s say you get the following output from running
PS > Get-ClusterNetwork | ft Name, Metric, AutoMetric

     Name                       Metric     AutoMetric
     ----                       ------     ----------
     Cluster Network 1          1000       True
     Cluster Network 2          1100       True
     Cluster Network 3          1200       True
     Cluster Network 4          10000      True
     Cluster Network 5          10100      True

In this scenario, these networks would carry the following types of traffic:

Cluster Network 1 (1000) – Cluster & CSV Traffic
Cluster Network 2 (1100) – Live Migration Traffic
Cluster Network 3 (1200) – Backup network for Cluster & CSV Traffic and Live Migration Traffic
Cluster Network 4 (10000) – Public Traffic
Cluster Network 5 (10100) – Backup network for Public Traffic

Configuring Network Prioritization

It is possible to customize NP if the cluster does not automatically assign networks to use the traffic pattern that you want, which will change the ranked order, and hence the function. For example, you may want Cluster Network 3 to be used for “Live Migration Traffic” as it is the fastest, so you would change its Metric to a value between 1000 and 1100, such as 1050, so that it is ranked second on the list. Once Cluster Network 3 has the second-lowest metric it will be used for Live Migration Traffic.

To change the value of a network metric, run:
PS > $n = Get-ClusterNetwork “Cluster Network 3”
PS > $n.Metric = 1050

This will change the metric of Cluster Network 3 to 1050.

Now you get the following output from running
PS > Get-ClusterNetwork | ft Name, Metric, AutoMetric

     Name                       Metric     AutoMetric
     ----                       ------     ----------
     Cluster Network 1          1000       True
     Cluster Network 3          1050       False
     Cluster Network 2          1100       True
     Cluster Network 4          10000      True
     Cluster Network 5          10100      True

You may have noticed that is a property associated with each network called AutoMetric. This indicates whether the Metric was set using the default values (True) or if it had been later adjusted by an admin (False). This gives insight into whether NP has been configured on the cluster. Using this flag, it is actually possible to change the value of a network back to its original and automatically assigned value, by running the cmdlet:
PS > $n = Get-ClusterNetwork “Cluster Network 3”
PS > $n.AutoMetric = $true

Overriding Network Prioritization Behavior

There are two ways to override the default behavior of NP. The first is by changing the network’s properties by right-clicking on the network in Failover Cluster Manager, selecting Properties, and changing the radio buttons or checkboxes. For example, if you select “Do not allow cluster network communication on this network”, then it will not be possible to send any “Cluster & CSV Traffic” or “Live Migration Traffic” through this network, even if the network has the lowest metric values. The cluster will honor this override and find the network with the next lowest value to send this type of traffic.

The second override is exclusively for “Live Migration Traffic”. The networks for live migration can be configured more granularly by right-clicking on any Virtual Machine resource, selecting Properties and clicking the Network for live migration tab. Here you have the ability to specify which networks can and cannot be used for “Live Migration Traffic” and in which order they should be used. Even though it appears that this setting may be unique to that specific VM, it is actually a global setting for live migration. This means that it will override the “Live Migration Traffic” network configured through NP and all VMs will perform a live migration through the network(s) specified here. If this setting is change multiple locations, the last change will be honored.

With this information we hope you are better able to understand how to deploy, configure and use your cluster networks to get the optimal performance and function from each.

Thanks,
Symon Perriman
Technical Evangelist
Private Cloud Technologies
Microsoft

Managing Hardware Snapshots on Failover Clusters

Elden Christensen — Tue, 31 May 2011 03:53:34 GMT

In this blog I will discuss how clusters identify disks and considerations of presenting a hardware snapshot of a LUN back to the same cluster.

Windows Server Failover Clustering has a kernel mode driver called ClusDisk.sys that listens for PnP arrivals of new disks to the system. When a new disk arrives ClusDisk will look at the disk signature of the disk, if the signature matches that of a known clustered disk, then ClusDisk will call into Partition Manager and register that disk as being owned by clustering. This will leave the disk in an Offline state with the volume dismounted, and clustering will be registered so that it is the only mechanism that can tell the disk to come online (and on which node). This enables clustering to orchestrate which server and when disks should be mounted, and ensure volumes do not become corrupted. This ownership is reflected in Disk Management (DiskMgmt.msc) by the disk being listed as “Reserved”.

Note: A disk being listed as “Reserved” in DiskMgmt.msc is related to PartMgr ownership and has nothing to do with Persistent Reservations or SCSI commands to the LUN (this is a common point of confusion).

When leveraging hardware snapshots you will have a block-by-block replica of a disk. All data on that replica LUN will be identical to the source LUN, including the disk signature in sector 0 of the master boot record. If you attempt to present that replica LUN back to the same system, you will now have two separate LUN’s with identical disk signatures… Windows does not allow you to mount two disks with the same signature.

To address this, there is logic to handle disk signature collisions. When a disk arrives that has a signature which matches that of a disk which is already mounted, when the second disk is mounted the signature is changed to a new unique value. As a result, the disk will be identified as a new disk.

On a Failover Cluster you cannot rely on this signature collision handling logic when a volume is mounted because the ClusDisk driver will see the arrival of the replica disk, it will then recognize it has the signature that matches that of a known clustered disk and will register with Partition Manager to leave that disk in an offline state. As a result, the mount collision logic will never be able to take effect.

If you want to present a hardware snapshot of a LUN back to the same cluster, which already has the source LUN mounted you will need to change the disk signature of the disk (and cannot rely on the mount collision logic). There are a few options to achieve this (these comments are primarily directed at storage vendors):

Use the BreakSnapshotSet VSS API when breaking the snapshot, this will allow generating a new disk signature on the snapshot when it is created.
If the storage array does not work within the VSS framework, then the storage array can internally handle generating a new random (non-matching to any already clustered disk) disk signature when breaking the snapshot (just as the VSS API would do).
On a system outside of that cluster, you can use DiskPart with the ‘uniqueid disk’ command to manually change the disk signature. Then present the LUN to the cluster.

Thanks!
Elden Christensen
Principal Program Manager Lead
Clustering & High-Availability
Microsoft

Follow the Cluster Blog on your Windows 7 phone

Elden Christensen — Tue, 31 May 2011 02:11:03 GMT

Now you can now follow this blog on your Windows 7 phone! An RSS reader for cluster-related feeds is now available, including this cluster team blog, the cluster Technet forums, and even blogs from Cluster MVP’s.
Keep up to date on the latest Windows Server Failover Clustering news, right on your Windows 7 phone for FREE!

Features:

Ad-free, fast, no unnecessary graphics, effects, and colors (it uses system colors you set instead of deciding for you), letting you focus on the content
Caches content for offline reading for when you don’t have reception or network connection
Highlights new entries that you haven’t yet read, or entries that have been updated
Allows you to save your favorite entries/posts for quick access, and so they don’t drop off after time
Supports display orientation lock and white text on black background for easier night time reading (and to conserve power on OLEDs)
E-mail an entry/post to someone, or open it in IE for additional content

On your Windows 7 phone open the ‘Marketplace’ and search for “Cluster Feeds”, where you will find it under news & weather.

Understanding Quorum in a Failover Cluster

Rob-MSFT — Sat, 28 May 2011 05:17:00 GMT

Hi Cluster Fans,

This blog post will clarify planning considerations around quorum in a Failover Cluster and answer some of the most common questions we hear.

The quorum configuration in a failover cluster determines the number of failures that the cluster can sustain while still remaining online. If an additional failure occurs beyond this threshold, the cluster will stop running. A common perception is that the reason why the cluster will stop running if too many failures occur is to prevent the remaining nodes from taking on too many workloads and having the hosts be overcommitted. In fact, the cluster does not know your capacity limitations or whether you would be willing to take a performance hit in order to keep it online. Rather quorum is design to handle the scenario when there is a problem with communication between sets of cluster nodes, so that two servers do not try to simultaneously host a resource group and write to the same disk at the same time. This is known as a “split brain” and we want to prevent this to avoid any potential corruption to a disk my having two simultaneous group owners. By having this concept of quorum, the cluster will force the cluster service to stop in one of the subsets of nodes to ensure that there is only one true owner of a particular resource group. Once nodes which have been stopped can once again communicate with the main group of nodes, they will automatically rejoin the cluster and start their cluster service.

For more information about quorum in a cluster, visit: http://technet.microsoft.com/en-us/library/cc731739.aspx.

Voting Towards Quorum

Having ‘quorum’, or a majority of voters, is based on voting algorithm where more than half of the voters must be online and able to communicate with each other. Because a given cluster has a specific set of nodes and a specific quorum configuration, the cluster will know how many "votes" constitutes a majority of votes, or quorum. If the number of voters drop below the majority, the cluster service will stop on the nodes in that group. These nodes will still listen for the presence of other nodes, in case another node appears again on the network, but the nodes will not begin to function as a cluster until the quorum exists again.

It is important to realize that the cluster requires more than half of the total votes to achieve quorum. This is to avoid having a ‘tie’ in the number of votes in a partition, since majority will always mean that the other partition has less than half the votes. In a 5-node cluster, 3 voters must be online; yet in a 4-node cluster, 3 voters must also be online to have majority. Because of this logic, it is recommended to always have an odd number of total voters in the cluster. This does not necessarily mean an odd number of nodes is needed since both a disk or a file share can contribute a vote, depending on the quorum model.

A voter can be:

A node
- 1 Vote
- Every node in the cluster has 1 vote
A “Disk Witness” or “File Share Witness”
- 1 Vote
- Either 1 Disk Witness or 1 File Share Witness may have a vote in the cluster, but not multiple disks, multiple file shares nor any combination of the two

Quorum Types

There are four quorum types. This information is also available here: http://technet.microsoft.com/en-us/library/cc731739.aspx#BKMK_choices.

Node Majority

This is the easiest quorum type to understand and is recommended for clusters with an odd number of nodes (3-nodes, 5-nodes, etc.). In this configuration, every node has 1 vote, so there is an odd number of total votes in the cluster. If there is a partition between two subsets of nodes, the subset with more than half the nodes will maintain quorum. For example, if a 5-node cluster partitions into a 3-node subset and a 2-node subset, the 3-node subset will stay online and the 2-node subset will offline until it can reconnect with the other 3 nodes.

Node & Disk Majority

This quorum configuration is most commonly used since it works well with 2-node and 4-node clusters which are the most common deployments. This configuration is used when there is an even number of nodes in the cluster. In this configuration, every node gets 1 vote, and additionally 1 disk gets 1 vote, so there is generally an odd number of total votes.

This disk is called the Disk Witness (sometimes referred to as the ‘quorum disk’) and is simply a small clustered disk which is in the Cluster Available Storage group. This disk is highly-available and can failover between nodes. It is considered part of the Cluster Core Resources group, however it is generally hidden from view in Failover Cluster Manager since it does not need to be interacted with.

Since there are an even number of nodes and 1 addition Disk Witness vote, in total there will be an odd number of votes. If there is a partition between two subsets of nodes, the subset with more than half the votes will maintain quorum. For example, if a 4-node cluster with a Disk Witness partitions into a 2-node subset and another 2-node subset, one of those subsets will also own the Disk Witness, so it will have 3 total votes and will stay online. The 2-node subset will offline until it can reconnect with the other 3 voters. This means that the cluster can lose communication with any two voters, whether they are 2 nodes, or 1 node and the Witness Disk.

Node & File Share Majority

This quorum configuration is usually used in multi-site clusters. This configuration is used when there is an even number of nodes in the cluster, so it can be used interchangeably with the Node and Disk Majority quorum mode. In this configuration every node gets 1 vote, and additionally 1 remote file share gets 1 vote.

This file share is called the File Share Witness (FSW) and is simply a file share on any server in the same AD Forest which all the cluster nodes have access to. One node in the cluster will place a lock on the file share to consider it the ‘owner’ of that file share, and another node will grab the lock if the original owning node fails. On a standalone server, the file share by itself is not highly-available, however the file share can also put on a clustered file share on an independent cluster, making the FSW clustered and giving it the ability to fail over between nodes. It is important that you do not put this vote on a node in the same cluster, nor within a VM on the same cluster, because losing that node would cause you to lose the FSW vote, causing two votes to be lost on a single failure. A single file server can host multiple FSWs for multiple clusters.

Generally multi-site clusters have two sites with an equal number of nodes at each site, giving an even number of nodes. By adding this additional vote at a 3^rd site, there is an odd number of votes in the cluster, at very little expense compared to deploying a 3^rd site with an active cluster node and writable DC. This means that either site or the FSW can be lost and the cluster can still maintain quorum. For example, in a multi-site cluster with 2 nodes at Site1, 2 nodes at Site2 and a FSW at Site3, there are 5 total votes. If there is a partition between the sites, one of the nodes at a site will own the lock to the FSW, so that site will have 3 total votes and will stay online. The 2-node site will offline until it can reconnect with the other 3 voters.

Legacy: Disk Only

Important: This quorum type is not recommended as it has a single point of failure.

The Disk Only quorum type was available in Windows Server 2003 and has been maintained for compatibility reasons, however it is strongly recommended to never use this mode unless directed by a storage vender. In this mode, only the Disk Witness contains a vote and there are no other voters in the cluster. This means that if the disk becomes unavailable, the entire cluster will offline, so this is considered a single point of failure. However some customers choose to deploy this configuration to get a “last man standing” configuration where the cluster remain online, so long as any one node is still operational and can access the cluster disk. However, with this deployment objective, it is important to consider whether that last remaining node can even handle the capacity of all the workloads that have moved to it from other nodes.

Default Quorum Selection

When the cluster is created using Failover Cluster Manager, Cluster.exe or PowerShell, the cluster will automatically select the best quorum type for you to simplify the deployment. This choice is based on the number of nodes and available storage. The logic is as follows:

Odd Number of Nodes – use Node Majority
- Even Number of Nodes
  - Available Cluster Disks – use Node & Disk Majority
  - No Available Cluster Disk – use Node Majority

The cluster will never select Node and File Share Majority or Legacy: Disk Only. The quorum type is still fully configurable by the admin if the default selections are not preferred.

Changing Quorum Types

Changing the quorum type is easy through Failover Cluster Manager. Right-click on the name of the cluster, select More Actions…, then select Configure Cluster Quorum Settings… to launch the Configure Cluster Quorum Wizard. From the wizard it is possible to configure all 4 quorum types, change the Disk Witness or File Share Witness. The wizard will even tell you the number of failures that can be sustained based on your configuration.

For a step-by-step guide of configuring quorum, visit: http://technet.microsoft.com/en-us/library/cc733130.aspx.

Thanks!
Symon Perriman
Technical Evangelist
Private Cloud Technologies
Microsoft

Recommended Service Pack 1 Failover Cluster Hotfix

Elden Christensen — Thu, 28 Apr 2011 17:40:43 GMT

If you are upgrading a Windows Server 2008 R2 Failover Cluster to Service Pack 1, it is strongly recommended that you apply the hotfix KB 2531907. Here is the link:
http://support.microsoft.com/kb/2531907

Announcing Microsoft Virtualization Jump Start Training Videos!

Elden Christensen — Thu, 28 Apr 2011 16:54:48 GMT

Microsoft Learning hosted another new and exclusive Jump Start virtual training event – Microsoft Virtualization for VMware Professionals. We are thrilled to announce availability of the HD-quality video recordings – FREE – on TechNet Edge and soon coming to MS Showcase, the Zune Marketplace, iTunes! Additionally, every module will be made available (with assessment questions) on the Microsoft Virtual Academy (MVA) in the next few weeks.

What’s the high-level overview?

· The 15-hour course is broken into three sections covering Microsoft Virtualization: Platform, Management, and VDI

· “Team-teaching” approach led by Microsoft Technical Evangelist, Symon Perriman and Microsoft partner and virtualization architect, Corey Hynes

· Every module is an engaging discussion, packed with best practices and real-world demonstrations

Who is the target audience for this training?

· IT Professionals, Virtualization Engineers, Data Center Managers, IT Decision Makers, Network Administrators, Storage/Infrastructure Administrators & Architects

· Microsoft customers and partners currently using VMware

· Companies using VMware searching for real-world answers as they consider whether or not Microsoft Virtualization truly fits their needs

Where do I go for this great training?

The HD-quality video recordings of this course are on TechNet Edge. Here are direct links to each module:

· Entire course on TechNet Edge: Microsoft Virtualization for VMware Professionals

o Platform

· 1: Virtualization Overview

· 2: Differentiating Microsoft & VMware

· 3a: Hyper-V Deployment Options & Architecture | Part 1

· 3b: Hyper-V Deployment Options & Architecture | Part 2

· 4: High-Availability & Clustering

o Management

· 5: System Center Suite Overview with focus on DPM

· 6: Automation with Opalis, Service Manager & PowerShell

· 7: System Center Virtual Machine Manager 2012

· 8: Private Cloud Solutions, Architecture & VMM Self-Service Portal 2.0

o VDI

· 9: Virtual Desktop Infrastructure (VDI) Architecture | Part 1

· 10: Virtual Desktop Infrastructure (VDI) Architecture | Part 2

· 11: v-Alliance Solution Overview

· 12: Application Delivery for VDI

· Links to course materials on Born to Learn

Where can people learn more?

Virtualization experts worldwide are taking a new look at Microsoft Virtualization. After this Jump Start, we hope you will understand why. Check out the Microsoft Learning “Virtualization Training Portal” to better understand virtualization-specific certifications and where to find instructor-led classes to help you and your team. Additionally there is great training content on the Microsoft Virtual Academy (MVA) where you can learn at your own pace, connect with other IT Pros and earn prizes & rewards.

Thanks,

Symon Perriman

Technical Evangelist

Microsoft Virtualization for VMWare Professionals – Free Online Classes - March 29-31

Elden Christensen — Thu, 24 Mar 2011 23:38:04 GMT

Just one week after Microsoft Management Summit 2011 (MMS), Microsoft Learning will be hosting an exclusive three-day Jump Start class specially tailored for VMWare and Microsoft virtualization technology pros. Registration for “Microsoft Virtualization for VMWare Professionals” is open now and will be delivered as a FREE online class on March 29-31, 2010 from 10:00am-4:00pm PDT.

On March 29^th we will spend 2 hours focusing on Microsoft’s high-availability solutions for Virtualization using Windows Server 2008 R2 Failover Clustering. Learn about cluster planning, deployment, management, System Center integration and disaster recovery using multi-site clustering.

What’s the high-level overview?

· This cutting edge course will feature expert instruction and real-world demonstrations of Hyper-V and brand new releases from System Center Virtual Machine Manager 2012 Beta (many of which will be announced just one week earlier at MMS).

· Day 1 will focus on “Platform” (Hyper-V, virtualization architecture, high availability & clustering)

· Day 2 will focus on “Management” (System Center Suite, SCVMM 2012 Beta, Opalis, Private Cloud solutions)

· Day 3 will focus on “VDI” (VDI Infrastructure/architecture, v-Alliance, application delivery via VDI)

· A full outline is available on the Registration page

· Every section will be team-taught by two of the most respected authorities on virtualization technologies: Microsoft Technical Evangelist Symon Perriman and leading Hyper-V, VMWare, and XEN infrastructure consultant, Corey Hynes

Who is the target audience for this training?

Suggested prerequisite skills include real-world experience with Windows Server 2008 R2, virtualization and datacenter management. The course is tailored to these types of roles:

· IT Professional

· IT Decision Maker

· Network Administrators & Architects

· Storage/Infrastructure Administrators & Architects

How do I to register and learn more about this great training opportunity?

· Register: Visit the Registration page and sign up for all three sessions

· Blog: Learn more from the Microsoft Learning Blog

· Twitter: Here are a few posts you can retweet for us:

o Mar. 29-31 "Microsoft #Virtualization for VMWare Pros" @SymonPerriman Corey Hynes http://bit.ly/JS-Hyper-V @MSLearning #Hyper-V

o @SysCtrOpalis Mar. 29-31 "Microsoft #Virtualization for VMWare Pros" @SymonPerriman Corey Hynes http://bit.ly/JS-Hyper-V #Hyper-V

o Learn all the cool new features in Hyper-V & System Center 2012! SCVMM, Self-Service Portal 2.0, http://bit.ly/JS-Hyper-V #Hyper-V #Opalis

What is a “Jump Start” course?
A “Jump Start” course is “team-taught” by two expert instructors in an engaging radio talk show style format. The idea is to deliver readiness training on strategic and emerging technologies that drive awareness at scale before Microsoft Learning develops mainstream Microsoft Official Courses (MOC) that map to certifications. All sessions are professionally recorded and distributed through MS Showcase, Channel 9, Zune Marketplace and iTunes for broader reach.

Please join us for this fantastic event!

Configuring IP Addresses and Dependencies for Multi-Subnet Clusters - Part II

Symon Perriman MSFT — Wed, 19 Jan 2011 03:37:08 GMT

Hi Cluster Fans,

A few weeks ago I posted a blog about configuring IP Addresses with multiple subnets and got some feedback about some additional scenarios to consider. This blog post will discuss the best practice of setting possible owners for the IP Addresses.

Setting Possible Owners

The ‘Possible Owners’ setting is a resource property which allows an admin to designate whether a resource should come online on a specific node. By default, all nodes in the cluster will be the possible owner of a resource. However, you may want a workload to never come online on a specific node, so you will want to adjust these settings to prevent the cluster from trying to start the resource on that node. The resource will be able to move to a non-possible owner, however it will never automatically come online on that node.

An IP Address resource in a multi-subnet cluster is a perfect example of why you may want to configure this property. In our example from the previous blog post we deployed a File Server that used two subnets, a 172.24.13.0/24 and a 172.24.14.0/24. We will refer to these as our “.13 Subnet” and “.14 Subnet”, respectively. We configured two IPv4 Addresses, 172.24.13.1 on my .13 Subnet and 172.24.14.1 on my .14 Subnet. Since the .13 IP Address will only come online on nodes which are on the .13 Subnet, only those nodes should be possible owners. Since the .14 IP Address will only come online on nodes which are on the .14 Subnet, only those nodes should be possible owners.

Although configure Possible Owners is not required for multi-subnet clustering, it is a best practice since it reduces the amount of work which the cluster has to do after a failover since it will never try to bring those IP Addresses online on nodes which will never support it.

To configure Possible Owners, right-click on each IP Address resource and select Properties, then the Advanced Policies tab. Toggle the Possible Owners to the correct setting then click Apply and exit. Make sure you do this for both IP Address resources.

In our example, symonp-N1 is on our .14 Subnet, and symonp-N2 is on our .13 Subnet. This means that we want to make symonp-n1 the Possible Owner for the .14 IP Address and symonp-n2 the Possible Owner for the .13 IP Address.

Now your IP Address resources will only come online on the appropriate cluster nodes.

This KB article contains a great summary of the semantics of the various cluster ownership policies: http://support.microsoft.com/kb/299631.

Thanks,

Symon Perriman
Program Manager II
Clustering & High-Availability
Microsoft

Configuring IP Addresses and Dependencies for Multi-Subnet Clusters

Symon Perriman MSFT — Wed, 05 Jan 2011 15:41:00 GMT

Hi Cluster Fans,

At TechEd Berlin, several customers asked about the complexity of configuring multiple IP Addresses for clusters which have nodes in different subnets, so this short guide will provide a walk-through showing how easy it is to configure using Windows Server 2008 R2 Failover Clustering.

In Windows Server 2003, we had a requirement that all nodes must reside on the same subnet, so if nodes had to be in different physical locations to provide a disaster recovery solution, the administrator would have to create a virtual LAN (VLAN) across both sites. While this solution works great, a VLAN is more complex to configure and maintain. Starting in Windows Server 2008, a feature called an “OR dependency” was added to Failover Clustering which gave admins the ability to have nodes with a Network Name that could use one IP Address OR another IP Address.

Using this concept of an OR dependency, we can ensure that a clustered group can stay online on two or more subnets (or sites), by saying that if the application has an IP Address from the first site OR an IP Address from the second site (OR an IP Address from the third site, etc.) then the group can remain online and can serve client connections. Best of all – and unlike most competitive solutions – this failover to the other sites is fully automated! Let’s be honest, if a real flood hits the datacenter, the cluster administrator isn’t going to sit in a pool of water trying to bring applications online on a remote site, he’s swimming home to take caer of his family. If your solution does not have automatic failover, your solution may not survive a real disaster. But you do not have to worry with Windows Server Failover Clustering since the disaster recovery solution is not only automatic, but easy to configure with multiple subnets.

In this example I will create a highly-available File Server (ClusterFileSrv) which uses two subnets, a 172.24.13.0/24 and a 172.24.14.0/24. We will refer to these as our “.13 Subnet” and “.14 Subnet”, respectively.

First I will create a highly-available File Server and configure a static IP Address of 172.24.13.1 on my .13 Subnet.

Next I will add a new IP(v4) Address from right-clicking on the File Server’s name. I could also do this by selecting Add a Resource from the Actions pane:

You will see the IPv4 Address resource appear in the same group, but now it needs to be configured. You will notice that the IP Address is offline since it isn’t configured, and this changes the overall state of the group from ‘Online’ to ‘Partial Online’ since not every resource in the group is online. To configure this IP Address, right-click this new resource and select Properties:

Now provide the IP Address for the .14 Subnet. Make sure you select the correct network. Enter a Static IP address or select DHCP if you are using this technology. In the example I am using a Static IP Address of 172.24.14.1 on my .14 Subnet.

After I click Apply and exit this properties page I can now see the new IP Address in Failover Cluster Manager. You will notice that this is still offline, which is expected since the group is currently on a node which is using the .13 Subnet.

Now I will configure the dependencies of the Network Name, ClusterFileSrv, so that it can use either the .13 Subnet OR the .14 Subnet. This means that the File Server can come online if the IP Address on either subnet is available. To configure these dependencies, right-click the higher-level object (the Network Name), select Properties, and then select the Dependencies tab.

To group the new IP Address within this dependency expression, click on the location of the on-screen dialog, “Click here to add a dependency”. From the dropdown, select the .14 Subnet IP Address which was just created. Change the AND/OR option to OR, since we want this to be an OR relationship.

After applying the changes, this IP Address will now be grouped with the Network Name in Failover Cluster Manager. The .14 Subnet IP Address will still be offline. Now let’s test our configuration by moving this group to the .14 Subnet on the other node.

After failover, we can see that the .13 Subnet IP Address goes offline, but the .14 Subnet IP Address comes online. Since the Network Name uses an OR dependency, ClusterFileSrv is also able to come online after the move because the IP Address is available on the .14 Subnet to serve client connections.

I hope this makes adding IP Addresses and creating dependencies even easier for you.

[Update 1/18/11] Check out Part II of this blog series: http://blogs.msdn.com/b/clustering/archive/2011/01/19/10117423.aspx.

Thanks,
Symon Perriman
Program Manager II
Clustering & High-Availability
Microsoft

Understanding Hyper-V Virtual Machine (VM) Failover Policies

Symon Perriman MSFT — Tue, 14 Dec 2010 03:15:00 GMT

In this blog I will discuss the behavior of the failure policies for a highly available Hyper-V virtual machine running on a Windows Server 2008 R2 Failover Cluster. For the most part, cluster failover policies are generic and apply to all workloads on a cluster, whether it is SQL, a File Server, etc. There is a wealth of documentation out there, but I thought I would take a focused view at understanding what influences where VMs go when a node crashes.

Default Failover Policies:

When there is a failure of a node, VMs are spread across the remaining cluster nodes and distributed to the nodes hosting the fewest number of VMs. For example, let’s say that NodeA crashes and it was hosting 10 VMs. The Cluster Service will take one of the VMs, it will then look across the surviving nodes and find the node currently hosting the fewest number of VMs (technically it looks at cluster Group ownership and selects the node with the lowest count). The VM is then started on that node. Then the next VM is selected, then again looks across the cluster again for the node now currently hosting the fewest VMs is selected and the VM is started there. This happens for all VMs until they are all placed. VMs will be distributed across the cluster to different nodes based on who is currently hosting the fewest VMs.

To prevent "boot storms" where simultaneously starting a large number of VMs could severely impact the server’s performance or the underlying storage, there is throttling of VM starts for any individual node. As a safety precaution during failover or on node boot, VM start is throttled to 32 VMs concurrently in the process of being started at any given time on each node (technically the VM has Online called to the cluster resource). The rest will be queued up to start on that node. Once a VM completes starting, by getting past POST (technically the cluster resource transitions from Online Pending to Online), another VM is then started to slightly stagger all the VM starts.

For the average person, this is probably all you need to know or care about. However, for the geeks I’ll now continue to some more advanced topics.

Advanced Failover Policies:

In general the default settings are the right choice for most people, and I recommend you stick with those. However, Clustering does offer a wealth of granular options so you can fine tune the default behavior.

· Possible Owners – For a given VM (technically any cluster Resource) you can configure the nodes which the VM has the possibility of failing over to. By default it’s all nodes, but if you have a specific node you never want this VM to failover to you can remove it from being a possible owner and prevent it.

· Preferred Owners – For a given VM (technically any cluster Group) you can configure the preference for node order on failover. So let’s say that this VM normally runs on NodeA and you always want it next to go to NodeC if it is available, then preferred owners is a way for you to define a preference of first go to this node, then next go to this other node, then go to this next node. It’s a priority list, and clustering will walk that list in where to place the VM. This will override the default behavior of selecting the node currently hosting the least VMs I described above, and gives you explicit control of where VMs go. More information about Preferred and Possible Owners: http://support.microsoft.com/kb/299631.

· Anti-Affinity – For a given VM (technically any cluster Group) there is a cluster group property called AntiAffinityClassNames that allows you to configure the preference to attempt to keep that VM off the same node as other similar VMs. Let’s say for example you have two domain controllers running in VMs. It would probably be best to keep those running on different nodes if possible. When determining failover, the cluster service will deprioritize any node which is hosting a similar VM. If there is no other option (in the goal of making VMs available) it will place them on the same host. More information: http://msdn.microsoft.com/en-us/library/aa369651(VS.85).aspx.

Other Influencers of Failover:

There are a few other settings worth discussing which can influence VM placement and behavior on failover. Let’s discuss some of those:

· Pause Node – At the server level, you can Pause a node. When a node is paused, it means that no VMs (technically no cluster Group) can failover to that node. If a node is paused, it will be removed from the possibility to be a failover target. Pausing a node is handy when doing maintenance tasks like applying a patch, this prevents VMs from failing over to the node when you are doing something to it.

· Disable Failover – For a given VM (technically any cluster Group) you can configure the “Auto Start” setting. If auto start is disabled for a VM, it means that a VM will not be started on failover. This could be useful if you have a low priority VM that you don’t necessarily want to failover, but you still want it to be clustered so that you can, for example, perform live migrations.

Startup Placement Policies

· Persistent Mode – When a cluster as a whole is shut down and then restarted clustering will attempt to start VMs back on the last node they were hosted on. This is controlled by the “Persistent Mode” setting, and is enabled by default. The default amount of time the cluster service will wait for the original node to rejoin the cluster is 30 seconds; this is configurable via the cluster common property ClusterGroupWaitDelay. You may choose to disable Persistent Mode for high priority VMs, where you do not want to wait for the original node to come back… just start the VM as soon as possible. See this blog for additional information.

Thanks!
Elden Christensen
Principal Program Manager Lead
Clustering & High-Availability
Microsoft

Clustered File Servers and the Remote Volume Management Firewall Setting

Symon Perriman MSFT — Tue, 07 Dec 2010 03:17:40 GMT

Hi Cluster Fans,

A common mistake when creating clustered file is not opening the appropriate firewall setting, Remote Volume Management. This can cause the file server to give errors for various operations. Remote Volume Management provides remote software and hardware disk volume management using RPC. More information about using Remote Volume Management with File Servers is available here: http://technet.microsoft.com/en-us/library/cc725945.aspx.

If you try to ‘Manage shares and storage’ or ‘Add a shared folder’ you may see an error message if this firewall port is not opened.

The common error messages are:

· ‘ServiceName: The cluster cannot be accessed. The RPC server is unavailable’

· ‘ServiceName: Connection to the Virtual Disk Service failed. A VDS (Virtual Disk Service) error occurred while performing the requested operation. The connection to the server might be blocked by Windows Firewall. Make sure that the firewall on the server (or on all nodes in a failover cluster) is configured correctly. See the Managing Disks Remotely topic in Disk Management Help for the correct firewall settings.’

To allow these features to function correctly, enable the Remote Volume Management Firewall settings on every node in the cluster through Windows Firewall.

Now you will be able to manage disks, files and shares on your Failover Cluster.

The official Microsoft Knowledge Base article describing these steps is here: http://support.microsoft.com/kb/947047.

Thanks,

Symon Perriman
Program Manager II
Clustering & High-Availability
Microsoft

Changing Clustered IP Addresses & Configuring a DHCP or WINS Cluster

Symon Perriman MSFT — Tue, 30 Nov 2010 04:46:00 GMT

Hi Cluster Fans,

In Windows Server 2008 and Windows Server 2008 R2 we have simplified how easy it is to create and use a Failover Cluster. For example, if we detect you are using DHCP in your datacenter, we will automatically assign the cluster as well as clustered applications and services a DHCP IP Address. This just makes configuration even easier since environment defaults are used, so less information is needed!

But what happens if you want to deploy just a single DHCP (or WINS) Server on your cluster? The recommendation is to use a static IP Address, so that the DHCP Server can always be reached, without requiring it to request a DHCP IP Address from a DHCP Server. If you use DHCP, you should always have multiple DHCP Servers in your environment for high-availability and redundancy, but even if you experience multiple simultaneous failures, you will still want them to come back online as quickly as possible by giving them statically assigned IP Addresses. If you only had only a single DHCP server which was using a DHCP address, this DHCP server may never come up.

The following steps will show how to configure a DHCP server with a static IP Address when DHCP is detected in the environment. First open the Windows Server 2008 R2 Failover Cluster Manager. Then launch the High-Availability Wizard by selecting the action, Configure a Service or Application for High-Availability. This wizard will ask just a few questions before giving you a clustered DHCP (or WINS) Server. First it asks for the application, feature, role or service to make highly-available. A DHCP Server is selected.

Second it asks for a friendly name for the DHCP server, this is the Network Name. If DHCP was not being used, a prompt for an IP Address would be show.

Finally shared cluster storage is selected. After confirming the information, a highly-available DHCP Server is created in very few simple steps.

Since a DHCP IP Address was used for the DHCP Server, we want to give it a static IP Address. First right-click the IPv4 Address and select Properties. Currently a DHCP IP Address is selected for use.

This is easily changed to use a statically assigned IP Address, 10.121.36.100.

Now after applying the changes and exiting, the DHCP is using the statically assigned IP Address.

You can use this method to change any type of IP Address this quick, including IPv6 or other statically assigned IPv4 and IPv6 Addresses.

Thanks,
Symon Perriman
Program Manager II
Clustering & High-Availability
Microsoft

Trouble Connecting to Cluster Nodes? Check WMI!

Symon Perriman MSFT — Tue, 23 Nov 2010 20:38:09 GMT

Hi Cluster Fans,

One of the more frequent cluster network connection issues we see happens when the cluster cannot use WMI. WMI is Windows Management Instrumentation, which is an interface through which Windows components can provide information and notifications to each other, often between remote computers (more info about WMI). Failover Clustering and System Center Virtual Machine Manager (SCVMM) often use WMI to communicate between cluster nodes, so if there is an issue contacting a cluster node, WMI may be the culprit. We use WMI in most of our wizards, such as ‘Create Cluster Wizard’, ‘Validate a Configuration Wizard’, and ‘Add Node Wizard’, so any of the following messages and warnings we list could be due to WMI issues:

· "RPC Server Unavailable" error.

· Access is Denied.

· The computer ‘Node1’ could not be reached.

· Failed to retrieve the maximum number of nodes for ‘{0}’.

· The computer ‘Node1.contoso.com’ does not have the Failover Clustering feature installed. Use Server Manager to install the feature on this computer.

o Note: first confirm you have installed the Failover Clustering feature on this node

Troubleshooting Steps

Follow these series of troubleshooting steps to allow you to continue connecting your cluster.

1) Ensure it is not a DNS Issue

It is possible that the reason you cannot contact the other servers is due to a DNS issue. Before troubleshooting WMI, try connecting to that cluster, node or server using these methods when prompted by the cluster:

a) Network Name for the cluster or node

a. Example: MyNode

b) FQDN for the cluster or node

a. Example: MyNode.contoso.com

c) IP Address for the cluster or node

a. Example: 10.10.10.123

d) Some wizard pages have a ‘browse’ button which allows you to find other clusters in the domain through Active Directory

2) Check your that WMI is Running on the Node

Windows Server 2008 R2 Failover Clustering supports PowerShell and earlier version also come with a lightweight WMI client (WBEMTest). Using either PowerShell or Wbemtest you can confirm that WMI is up and running. Although you can use WMI remotely, it is better to test this directly on the server to ensure there are no other networking or firewall issue affecting the connection.

WMI Service

First check that the ‘Windows Management Instrumentation’ Service has started on each node by opening the Services console on that node. Also check that its Startup Type is set to Automatic.

Next we will check that Failover Clustering WMI (MSCluster) is running. These tests would be applicable after the cluster has already been created since we are checking for cluster-specific WMI functionality.

WBEMTest or directly on the server (2008 and 2008 R2)

· Launch CMD

· CMD > WBEMTest

· The Windows Management Instrumentation Tester will launch

· Select Connect

· Namespace: Root\MSCluster

· Select Connect

o If you see more options available, it means you are connected and WMI is working

§ Feel free to try a query to confirm, such as selecting ‘Query’ and enter: SELECT * from MSCluster_Resource

o If you see an error, there is a WMI issue

PowerShell or remotely from another node within the same cluster (2008 R2 only)

· Launch Elevated PowerShell

· PS > get-wmiobject mscluster_resourcegroup -computer MyNode -namespace "ROOT\MSCluster“

o If you see a lot of information displayed, WMI is running

o If you see an error, there is a WMI or firewall issue

3) Check your Firewall Settings

When a cluster is created, we automatically open up all the firewall settings you need. However enterprise security policies can make changes over time, so it is worth checking that the firewall on each server is allowing cluster communication. WMI request a DCOM connection to be made between the nodes, so you need to ensure that the ‘Remote Administration’ setting is enabled on every cluster node. This can be done through the Windows Firewall GUI or running the elevated command: CMD > netsh firewall set service RemoteAdmin enable. You will see a variety of errors or warnings if your firewall is not property configured. For more information about how WMI uses the firewall and troubleshooting firewall issues, visit: http://msdn.microsoft.com/en-us/library/aa389286(VS.85).aspx.

4) Reboot the Node

This can often fix intermittent issues. Follow best practices when rebooting the server, such as live migrating VMs and gracefully failing over other services and applications to reduce downtime. Only do this if the other troubleshooting attempts described above have failed.

5) Rebuild a Corrupt WMI Repository

If you continue to see errors after checking that WMI is running, the firewall is properly configured and rebooting, it is possible that your WMI repository has become corrupt so the cluster can no longer read from it. The following steps will enable you to rebuild your repository so that the other nodes can read from it again.

· In the Services console, manually stop the WMI service to ensure that dependent services are stopped

· Start WMI service again

· Launch and elevated CMD or PowerShell

· CMD/PS > winmgmt /salvagerepository

6) Patch WMI for Performance Improvements

You initial connection problems should now be fixed. If you continue to experience intermittent connection issues caused by WMI, it could be due to the performance of your servers. We have released a hotfix for 2008 R2 which improves the speeds at which we return WMI queries, and this is optimized for the most common WMI calls which SCVMM makes. Get it here: http://support.microsoft.com/kb/974930.

Good luck in resolving your cluster connection issues with WMI!

Thanks,

Symon Perriman
Program Manager II
Clustering & High-Availability

Microsoft

Clustering & High-Availability at TechEd Europe

Symon Perriman MSFT — Thu, 14 Oct 2010 19:19:00 GMT

Hi Cluster Fans,

In a few weeks the cluster team will be presenting at TechEd Europe from November 8-12 in Berlin, Germany. Additionally, several partner teams and Cluster MVPs will also be delivering high-availability sessions. Throughout the week, the team will be staffing the cluster booth, so please stop by to ask us whatever you want to know about Failover Clustering, Network Load Balancing and other High-Availability technologies.

Thanks!
Symon Perriman

Program Manager II
Clustering & High-Availability

Microsoft

Cluster Team Events

VIR309 - Disaster Recovery by Stretching Hyper-V Clusters Across Sites

Tuesday, November 9 | 6:00 PM - 7:00 PM

Session Type: Breakout Session

Track: Virtualization

Speaker(s): Symon Perriman, Edwin van Mierlo

As servers are consolidated into VMs, the availability of those VMs is becoming increasingly important in providing disaster tolerance and business continuance. This session covers considerations of multi-site clustering, where virtual machines can be configured on a Hyper-V Failover Cluster that reaches across sites. This session also covers conducting live migrations across datacenters, and considerations of Cluster Shared Volumes with replication software.

WSV203-IS (R) - Failover Clustering 101: Everything you Wanted to Know but were Afraid to Ask

Thursday, November 11 | 2:30 PM - 3:30 PM
Friday, November 12 | 12:30 PM - 1:30 PM
Session Type: Interactive Session

Track: Windows Server

Speaker(s): Symon Perriman

Want to learn about Windows Server 2008 R2 Failover Clustering for the first time? We will show you how to build a highly-available File Server and Hyper-V cluster in just a few simple steps, while answering any and all of your questions. We will cover planning, licensing, virtualization, support, validation, hardware, networking considerations, building a cluster, deploying workloads on a cluster, and deploying virtual machines on a cluster. No prior clustering knowledge is needed!

WSV306-IS - Failover Clustering with Hyper-V Unleashed in Windows Server 2008 R2

Wednesday, November 10 | 10:30 AM - 11:30 AM

Session Type: Interactive Session

Track: Windows Server

Speaker(s): Symon Perriman & Cluster MVPs

This is a free-form session that will demo, whiteboard and discuss what’s on YOUR MIND! Share the pain you have using Failover Clustering with Hyper-V, ask difficult questions or let us know why some customers avoid clustering even when there is a need to implement a high availability solution. Get the answers you need, directly from the world’s leading clustering experts.

WSV313 - Failover Clustering in 2008 R2: What’s New in the #1 High-Availability Solution

Friday, November 12 | 9:00 AM - 10:00 AM

Session Type: Breakout Session

Track: Windows Server

Speaker(s): Symon Perriman

Learn about what makes Windows Server 2008 R2 Failover Clustering the top Server High-Availability solution! Clustering has changed in every area since 2003, so in this technical session you will learn about Hyper-V, Live Migration, validation, flexible hardware requirements, easy deployment, migration, Cluster Shared Volumes multi-site clustering and easier management. A live demo will deploy a 2008 R2 Failover Cluster from scratch. Clustering knowledge recommended.

WSV403 - Failover Clustering & Hyper-V: Planning your Highly-Available Virtualization Environment

Wednesday, November 10 | 5:30 PM - 6:30 PM

Session Type: Breakout Session

Track: Windows Server

Speaker(s): Symon Perriman, Joachim Nasslander

This technical session will discuss Hyper-V and Failover Clustering live migration, deployment considerations, licensing, upgrades, host clustering, guest clustering, disaster recovery, multi-site clustering, System Center Virtual Machine Manager, hardware and validation. What are the pros and cons of each virtualization solution? What's right for my customers and their scenarios? What about combining physical and virtual machines in the same cluster? This session will include a live demo of a Hyper-V Cluster deployment and live migration.

WSV01-HOL - Implementing High Availability

Session Type: Hands-on Lab

Track: Windows Server

This top-rated lab introduces the process for creating highly available services using Windows Server 2008 R2 cluster technologies, as well as basic administration tasks with new clustering PowerShell Cmdlets.

Partner Team Events

DAT210 - Mission Critical: High Availability Overview

Tuesday, November 9 | 10:30 AM - 11:30 AM

Session Type: Breakout Session

Track: Database Platform

More information coming soon.

DAT211 - Mission Critical: Comparing High-Availability Solutions

Tuesday, November 9 | 4:30 PM - 5:30 PM

Session Type: Breakout Session

Track: Database Platform

More information coming soon

DAT212 - Mission Critical: Improving High-Availability

Wednesday, November 10 | 9:00 AM - 10:00 AM

Session Type: Breakout Session

Track: Database Platform

More information coming soon.

DAT303-IS - Performance Scalability and Reliability on Mission Critical Applications with Microsoft SQL Server 2008 R2 and Windows Server 2008 R2

Wednesday, November 10 | 4:00 PM - 5:00 PM

Session Type: Interactive Session

Track: Database Platform

Speaker(s): Lonnye Bower, Nitasha Chopra

Join us on a walk-through of the steps we took to improve performance of a SQL Server 2008 R2 Tier 1 Mission Critical Core Banking application by more than 30% . Learn the processes we followed that resulted in:

· A world record number of transactions per second (tps), compared to other database platforms, for an ISV Partner application (specifics about the ISV and the application will be shared in the session as the details are pending public release)

· Improving performance to push SQL Server to execute over 120,000 batches per second (YES! That's 120K batches per second!)

· Overcoming the issues we encountered during the exercise

You will garner tips and tricks used by Microsoft’s SQL Performance Group and SQL Server Customer Advisory Team (SQLCAT) to optimize mission critical customer applications. You will also be able to review the different hardware architectures (based on the Intel® Xeon® processor 7500 series) involved in the exercise and the limits that were pushed by each. At the end of this session, you will receive several simple takeaways to help you obtain better levels of performance out of your enterprise implementations.

MGT201-LNC - Managing Your Virtual Environments

Thursday, November 11 | 1:20 PM - 2:05 PM

Session Type: Lunchtime Session

Track: Management

Speaker(s): Kenon Owens, Michael Michael

Grab your lunch and bring your questions. Join Management and Virtualization Experts for a 'No Holds Barred' Question and Answer Session on how Microsoft can help you manage your virtualization environment in the datacenter. This session will highlight some of the new capabilities of System Center and Windows Server 2008 as we answer your questions candidly on topics including: Disaster Recovery and Availability; Virtual Machine Sprawl; how Virtualization can assist in my Service Oriented World; Server Application Virtualization; and Transitioning to Private and Public Clouds.

PRC05 - Microsoft Exchange Server 2010 High Availability and Disaster Recovery - Be Prepared

Monday, November 8 | 9:00 AM - 3:30 PM

Session Type: Pre-Conference Seminar

Track: Unified Communications

Speaker(s): Vladimir Meloski

Microsoft Exchange Server is a business-critical system for many organizations. This workshop will provide you with knowledge on how to respond to the challenges of managing Microsoft Exchange Server 2010 high availability, and how to address any disaster recovery scenario. You will also learn about deploying healthy and performance optimized Exchange Organization.

The topics will include:

· Planning for high available network and site environment

· Planning for hardware and storage technologies

· Planning Active Directory for Exchange Organization

· Planning and deploying high availability for mailbox server roles, using Database Availability Groups

· Planning and deploying high availability for non-mailbox server roles, using software or hardware Network Load Balancing and Client Access Arrays for Client Access Server role, as well as strategies for Hub Transport, Edge and Unified Messaging server roles

· Planning for site resilience, and the steps needed to switchover successfully to an alternative disaster recovery site

You will find out how to implement efficient backup and restore strategies, for both mailbox and non-mailbox server roles, using Windows Server Backup and Microsoft System Center Data Protection Manager 2010, but also discover how you might choose to run backup-less Exchange Organization and use raid-less and JBOD storage technologies for mailbox server roles. You will learn about the techniques on how to test and verify your high availability and disaster recovery strategies, including scenarios from disk to site failure. After completing this workshop you will know best practices for implementing high availability, and for providing healthy, reliable and optimized Exchange Organization.

UNC303 - Microsoft Lync Server 2010 : Voice Architecture and Planning for High Availability

Thursday, November 11 | 2:30 PM - 3:30 PM

Session Type: Breakout Session

Track: Unified Communications

Speaker(s): Jamie Stark, Sundereshwaran Raman

The Lync Server team has made a significant investment in high availability with the upcoming release. In this session we review the new Survivable Branch Appliance offering for branch office resiliency, the new features for data center-based multi-site failover, and intra-site architecture for high availability.

UNC321-IS - Microsoft Exchange 2010 SP1 High Availability: Ask The Experts

Thursday, November 11 | 6:00 PM - 7:00 PM

Session Type: Interactive Session

Track: Unified Communications

Speaker(s): Ross Smith IV, Scott Schnoll

Do you have questions about high availability or site resilience in Exchange Server 2010 SP1? Ross Smith IV, Principal Program Manager, and Scott Schnoll, Principal Technical Writer, will be on hand to answer your questions.

Curious about design considerations? Wondering about networking requirements? Thinking about deployment options? This session is intended to be highly interactive and completely Q&A driven, so bring your questions and join in the fun.

UNC401 - Microsoft Exchange Server 2010: High Availability Deep Dive (including changes introduced by SP1)

Thursday, November 11 | 10:30 AM - 11:30 AM

Session Type: Breakout Session

Track: Unified Communications

Speaker(s): Scott Schnoll

Go beyond the basics of Exchange high availability and gain a solid understanding of what's going on under the hood in an Exchange 2010 database availability group. This session dives deep into the inner workings of continuous replication, Active Manager, best copy selection, quorum and witness, lagged copies, DAG networks, DAC mode and built-in monitoring and management scripts.

VIR302-IS - Understanding How Microsoft Virtualization Compares to VMware

Thursday, November 11 | 6:00 PM - 7:00 PM

Session Type: Interactive Session

Track: Virtualization

Speaker(s): David Greschler, Edwin Yuen

This is a must-see session for anyone who is comparing VMware with Microsoft's virtualization offerings. We will review Microsoft's technology compared to VMware, focusing on what differentiates the two technologies. The session will also cover effective strategies for integrating Microsoft virtualization into datacenter environments, and review one company's experience with implementing Microsoft virtualization. If you need to know how the Microsoft virtualization stack compares to VMware's, this is the session for you!

VIR321 - Advanced Storage Infrastructure Best Practices to Enable Ultimate Hyper-V Scalability

Tuesday, November 9 | 9:00 AM - 10:00 AM

Session Type: Breakout Session

Track: Sponsor Sessions

Speaker(s): Txomin Barturen

This session will address how Microsoft and EMC technologies work together to deliver scalable and reliable enterprise deployments of Microsoft virtualization. Working together, Microsoft and EMC tested the scalability of up to 16 node Hyper-V clusters on the EMC Symmetrix storage platform. Topics to be covered include optimization and characterization of Microsoft Hyper-V for large scale highly-available deployments, as well as performance and optimization of the EMC Symmetrix. Technical details and lessons learned from the tested solution will be covered.

Evaluating High-Availability (HA) vs. Fault Tolerant (FT) Solutions

Symon Perriman MSFT — Wed, 06 Oct 2010 03:09:46 GMT

When evaluating how to increase availability and reduce downtime for your deployments, solutions can commonly be categorized as either a 'High Availability' solution or a 'Fault Tolerant' solution. In this blog I thought I would take a moment to discuss pros and cons of each.

High Availability Solutions

High availability solutions traditionally consist of a set of loosely coupled servers which have failover capabilities. Each system is independent and self-contained, yet the servers are health monitoring each other and in the event of a failure, applications will be restarted on a different server in the pool of the cluster. Windows Server Failover Clustering is an example of a HA solution. HA solutions provide health monitoring and fault recovery to increase the availability of applications. A good way to think of it is that if a system crashes (like the power cord was pulled), the application very quickly restarts on another system. HA systems can recover in the magnitude of seconds, and can achieve five 9's of uptime (99.999%)... but they realistically can't deliver zero downtime. They also are flexible in that they enable recovery of any application running on any server in the cluster.

Fault Tolerant Solutions

Fault tolerant solutions traditionally consist of a pair of tightly coupled systems which provide redundancy. Generally speaking this involves running a single copy of the operating system and the application within, running consistently on two physical servers. The two systems are in lock step, so when any instruction is executed on one system, it is also executed on the secondary system. A good way to think of it is that you have two separate machines that are mirrored. In the event that the main system has a hardware failure, the secondary system takes over and there is zero downtime.

HA vs. FT

So which solution is right for you? Well, the initial and obvious conclusion most instantly come to is that 'no' downtime is better than 'some' downtime, so FT must be preferred over HA! Zero downtime is also the ultimate IT utopia which we all strive to achieve, which is goodness. Also FT is pretty cool from a technology perspective, so that tends to get the geek in all of us excited and interested.

However, it is important to understand they protect against different types of scenarios... and the key aspect to understand is what are the most important to you and your business requirements. It is true that FT solutions provide great resilience to hardware faults, such as if you walk up and yank the power cord out of the back of the server... the secondary mirror will take over with zero client downtime. However, remember that FT solutions are running a common operating system across those systems. In the event that there is a software fault (such as a hang or crash), both machines are affected and the entire solution goes down. There is no protection from software fault scenarios and at the same time you are doubling your hardware and maintenance costs. At the end of the day while a FT solution may promise zero downtime, it is in reality only to a small set of failure conditions. With a loosely coupled HA solution such as Failover Clustering, in the event of a hang or blue screen from a buggy driver or leaky application. Then the application will failover and recover on another system.

Another mitigating factor to remember is that most hardware components can be configured in a resilient fashion... such as dual NIC's with NIC Teaming, or dual HBA's with multi-path software, or systems inherit redundancy... such as redundant power supplies. So the fundamental question to ask yourself is how often are you walking through the datacenter and accidently trip over a power cord? Or how often do you have a non-redundant piece of hardware fail... such as a motherboard?

My perspective is that if you are having motherboards fail on a regular basis, it's probably time to find a new hardware vendor anyway. FT also comes at a cost... primarily performance degradation associated with keeping two systems synchronized, which can be significant. It is also increases the cost, in that you need to have redundant systems both actively consuming resources. That equates to a 100% extra resource requirement. Windows Server Failover Clustering allows for as little as 1 node of capacity reserved for failure/recovery, with up to 16 nodes in the cluster, which equates to 6.25% extra resource requirement.

So when you break it down and really start to think about the failure scenarios, if you have a critical app where performance doesn't matter and you are only worried about massive non-redundant hardware failures... then FT probably the better solution for you.

While a loosely coupled system such as Failover Clustering cannot deliver zero downtime for hardware failures, it does protect against a wider range of failures up and down the stack including hardware, OS, and even provides application health monitoring. HA solutions can also reduce downtime for other scenarios, such as patching. With Failover Clustering, you can move the application to another server when it comes time to patch the OS or application. Disclaimer: An application running in a VM guest OS on top of a virtualization HA platform does not provide the application health monitoring and application failover capabilities I've discussed here, but that's a topic for another blog. My core point is to be careful not to assume all "HA" branded solutions are equal, when comparing virtualization HA to OS clustering.

	Windows Server Failover Clustering	Fault Tolerant Solution
Hardware Failure	P	P
OS Level Failure	P
Application Failure	P

At the end of the day, there is no right or wrong answer. It boils down to you evaluating what your individual most common sources of downtime are, then deploying a solutions which helps mitigate them. Going back and analyzing the root cause of your sources of downtime over the last year is a good place to start, then you can come up with a strategy on what solution best mitigates them. Additionally, the business requirements vary for each deployment, so the service level agreement (SLA) you need to achieve, and what the acceptable levels of downtime for the failure conditions you need to protect against are ultimately up to you.

Thanks!
Elden Christensen
Senior Program Manager Lead
Clustering & High-Availability
Microsoft