This topic provides guidance on the key physical and virtual performance counters you need to watch for each of the resources. I must be honest, unless you are nut for infrastructure performance and optimization, the information below is not exactly as riveting as watching an episode of CSI (New York is my favourite), or Heroes/ 24! :) Why then have I included it in this series?

At some point you, or a member of your SharePoint Infrastructure team, will need to measure and optimize the performance of your physical or virtualized SharePoint farm. This is when this blog post will be very handy! It will help you understand WHICH performance counters to look at, WHAT they are really measuring and reporting, and lastly HOW to optimize your virtualized environment based on what is reported.

Finally this topic provides a brief overview of the supporting management technologies that can improve the governance and management of your virtualized SharePoint environment.

This topic discusses:

• Measure performance of your CPU, memory, disk and network of your physical and virtual SharePoint environment
• Recommendations for managing your virtual SharePoint environment

Measure performance of your CPU, memory, disk and network of your physical and virtual SharePoint environment.

Measure Processor Performance

Guest operating system processors do not have a set affinity to physical processors/cores

What does this fancy statement from TechNet really mean to a SharePoint infrastructure administrator? Hell, I had to read that statement a couple times! It means that the hypervisor of Hyper V (which sits between host machine and SharePoint guest machines) determines how physical CPU resources are allocated across your guest virtual machines. The hypervisor does this by scheduling guest processor time in the form of threads on the physical processor. How much time? This is determined using a combination of your virtual machine logical CPU allocation settings, and secondly reserves, caps and weights to balance CPU time between your hosts guest machines.

• Reserves allow you to “borrow” time from an idle guest machine.
• Caps allow you to limit a virtual machine to certain % time.
• Weights allow more critical virtual machine to get more CPU time than less critical virtual machine.

Measure host and guest operating system processor utilization

• Warning: “\Processor(*)\% Processor Time” monitor counter measures the processor utilization of the host operating system only. It is NOT an accurate counter for evaluating processor utilization of a guest operating system.
• To measure total physical processor utilization of the host operating system and all guest operating systems, use the “\Hyper-V Hypervisor Logical Processor(_Total)\% Total Run Time” performance monitor counter. Use the following thresholds to evaluate guest operating system processor utilization :
  • Less than 60% consumed = Healthy
  • 60% - 89% consumed = Monitor or Caution
  • 90% - 100% consumed = Critical, performance will be adversely affected
• To accurately measure the processor utilization of a guest operating system, use a combination of “\Hyper-V Hypervisor Logical Processor(_Total)\% Total Run Time” (LPTR) and “\Hyper-V Hypervisor Virtual Processor(_Total)\% Total Run Time” (VPTR) performance monitor counters on the Hyper-V host operating system. Its important to balance LPTR AND VPTR. See the “Optmizing Processor Performance” section and the measuring CPU performance section on how to interpret and optimize these values.

Measure Memory Performance

Measure available memory on the PHYSICAL host operating system

The amount of physical memory available to the Hyper-V host operating system can be determined by

• “\Memory\Available MBytes” - Available MBytes measures the amount of physical memory available to processes running on the computer, as a percentage of physical memory installed on the computer. The following guidelines apply when measuring the value of this performance monitor counter
  • 50% of free memory available or more = Healthy
  • 25% of free memory available = Monitor
  • 10% of free memory available = Warning
  • Less than 5% of free memory available = Critical, performance will be adversely affected
• “\Memory\Pages/sec” – This performance monitor counter measures the rate at which pages are read from or written to disk to resolve hard page faults. To resolve hard page faults, the operating system must swap the contents of memory to disk, which negatively impacts performance. A high number of pages per second in correlation with low available physical memory may indicate a lack of physical memory. The following guidelines apply when measuring the value of this performance monitor counter:
  • Less than 500 = Healthy
  • 500 - 1000 = Monitor or Caution
  • Greater than 1000 = Critical, performance will be adversely affected

Measure available memory on the GUEST operating system

Memory that is available to the guest operating systems can be measured with the same performance monitor counters used to measure memory available to the Hyper-V host operating system.

Measure Disk Performance

Measure disk latency on a PHYSICAL host operating system

For best initial Read and Write disk performance indicator, use “\Logical Disk(*)\Avg. sec/Read” and “\Logical Disk(*)\Avg. sec/Write”. These performance monitor counters measure the amount of time that read and write operations take to respond to the operating system.

Use logical disk versus physical disk performance monitor counters is recommended because Windows applications and services utilize logical drives represented as drive letters wherein the physical disk (LUN) presented to the operating system can be comprised of multiple physical disk drives in a disk array.

Measuring Disk Latency: Use “\Logical Disk(*)\Avg. Disk sec/Read” or “\Logical Disk(*)\Avg. Disk sec/Write” performance monitor counters when measuring disk latency on the Hyper-V host operating system:

• 1ms to 15ms = Healthy
• 15ms to 25ms = Warning or Monitor
• 26ms or greater = Critical, performance will be adversely affected

Measure disk latency on GUEST operating systems 

Response times of the disks used by the guest operating systems can be measured using the same performance monitor counters used to measure response times of the disks used by the Hyper-V host operating system.

For more information about disk performance analysis, see the following resources:Ruling Out Disk-Bound Problems at http://go.microsoft.com/fwlink/?LinkId=120947 .

Measure Network Performance

Hyper-V allows guest computers to share the same physical network adapter. While this helps to consolidate hardware, take care not to saturate the physical adapter. Use the following methods to ensure the health of the network used by the Hyper-V virtual machines:

• Test network latency – Ping each virtual machine to ensure adequate network latency. On local area networks, expect to receive less than 1ms response times.
• Test for packet loss – Use the pathping.exe utility to test packet loss between virtual machines. Pathping.exe measures packet loss on the network and is available with all versions of Windows Server since Windows Server 2000. Pathping.exe sends out a burst of 100 ping requests to each network node and calculates how many pings are returned. On local area networks there should be no loss of ping requests from the pathping.exe utility.
• Test network file transfers – Copy a 100MB file between virtual machines and measure the length of time required to complete the copy. On a healthy 100Mbit (megabit) network, a 100MB (megabyte) file should copy in 10 to 20 seconds. On a healthy 1Gbit network, a 100MB file should copy in about 3 to 5 seconds. Copy times outside of these parameters are indicative of a network problem. One common cause of poor network transfers occurs when the network adapter has “auto detected” a 10MB half-duplex network which prevents the network adapter from taking full advantage of available bandwidth.

Measure network utilization on the PHYSICAL host operating system

• Evaluate network bandwidth utilization: Use “\Network Interface(*)\Bytes Total/sec” – The percentage of network utilization is calculated by multiplying Bytes Total/sec by 8 to convert it to bits, multiply the result by 100, then divide by the network adapter’s current bandwidth. Use the following thresholds to evaluate network bandwidth utilization:
  • Less than 40% of the interface consumed = Healthy
  • 41%-64% of the interface consumed = Monitor or Caution
  • 65-100% of the interface consumed = Critical, performance will be adversely affected
• Evaluate network latency: Use “\Network Interface(*)\Output Queue Length” – The output queue length measures the number of threads waiting on the network adapter. If there are more than 2 threads waiting on the network adapter, then the network may be a bottleneck. Common causes of this are poor network latency and/or high collision rates on the network. Use the following thresholds to evaluate output queue length:
  • 0 = Healthy
  • 1-2 = Monitor or Caution
  • Greater than 2 = Critical, performance will be adversely affected.
• Ensure that the network adapters for all computers (physical and virtual) in the solution are configured to use the same value for maximum transmission unit (MTU). For more information about configuring the MTU value see “Appendix A: TCP/IP Configuration Parameters” at http://go.microsoft.com/fwlink/?LinkId=113716. If an output queue length of 2 or more is measured, consider adding one or more physical network adapters to the physical computer that hosts the virtual machines and bind the network adapters used by the guest operating systems to these physical network adapters.

Measure network utilization on the GUEST operating systems

If a network adapter on the Hyper-V root partition is busy as indicated by the performance monitor counters mentioned above, then consider using the “\Hyper-V Virtual Network Adapter(*)\Bytes/sec” performance monitor counter to identify which virtual network adapters are consuming the most network utilization.

Recommendations for managing your virtual SharePoint environment

This section discusses the tools and products you can use to manage your virtualized SharePoint environment. It is focussed on Microsoft products but recognises that there are other fantastic virtualization support products such as VMware Infrastructure. Please refer to their site for more information on what they provide.

System Center Virtual Machine Manager (VMM) 2008

Virtual Machine Manager 2008 is the ideal management tool for SharePoint admins tasked with the upkeep of virtualized SharePoint farms, as it provides valuable tools that allow for rapid provisioning of new SharePoint front-ends servers, conversion of physical servers to Hyper-V guests, and other highly useful management capabilities.

Virtual Machine Manager is an end-to-end management solution that can manage both Hyper-V guests and VMware Virtual Infrastructure 3 (VI3). It directly integrates with VMware’s VirtualCenter server to directly manage the VMware hosts. This allows for VMM to perform tasks specific to VMware, such as moving sessions from ESX hosts using VMotion technologies.

System Center Virtual Machine Manager (VMM) 2008 provides a straightforward and cost-effective solution for unified management of physical and virtual machines, consolidation of underutilized physical servers, and rapid provisioning of new virtual machines. It provides for critical virtual management functionality in environments that are considering virtualizing SharePoint farms, particularly when dealing with multiple servers and farms.

System Center Operations Manager

System Center Operations Manager 2007 is an end-to-end service management product which works seamlessly with Microsoft software and applications enabling greater control of the IT environment. Some features I think will be of interest to virtualized SharePoint environments include:

• PROTips integration in Virtual Machine Manager 2008: PROTips stands for Performance and Resource Optimization and is fully extensible. It is simply an extension of the existing Operations Manager management pack infrastructure. When PRO finds an opportunity for optimization, it generates a PRO tip, which comes complete with a remediation script. A VMM administrator can manually approve the PRO tip for implementation or configure PRO to automatically implement tips on all or a subset of your environment.
• Server health monitoring and management
• Performance reporting and analysis

System Center Configuration Manager

System Center Configuration Manager is the solution to comprehensively assess, deploy, and update your servers, clients, and devices—across physical, virtual, distributed, and mobile environments. Optimized for Windows and extensible beyond, it is the best choice for gaining enhanced insight into, and control over, your IT systems.

Some features I think will be of interest to virtualized SharePoint environments is patch management of offline virtual machines and management of software upgrades.

System Center Data Protection Manager

Focused on the primary Microsoft server workloads, DPM 2007 was specifically built to protect and recover SQL Server, Microsoft Exchange Server, SharePoint Portal Server, Microsoft Virtual Server, as well as Windows file services. In addition, DPM 2007 blends the best aspects of continuous data protection (CDP) with traditional tape backup. Some features I think will be of interest to virtualized SharePoint environments is virtual machine backup and restore and disaster recovery scenarios.

Other articles in this series

• Virtualizing SharePoint Series - Introduction
• Virtualizing SharePoint Series - Optimizing the performance of a virtualized SharePoint environment
• Virtualizing SharePoint Series - SharePoint server role recommendations in virtualized environmennts
• Virtualizing SharePoint Series - Monitoring and managing your virtualized SharePoint environment
• Virtualizing SharePoint Series - High availability and disaster recovery, deployment best practices, common mistakes and summary

This article was authored by:

Brian Wilson
Senior Consultant
Microsoft Consulting Services UK
Brian.Wilson@Microsoft.com

Click here for my bio page