Hyper-V: Clocks lie... which performance counters can you trust?
Many people have asked me how to do performance analysis using Hyper-V. The first thing I always tell them is clocks lie. Lets dig into this a little bit.
First some basic concepts. There are two types of partitions that the Hypervisor manages. The root partition which is special (manages physical devices, child partitions, … all except CPU and physical memory access which is the Hypervisors job) and the child partitions. The child partitions are where the “Guest” virtual machines run. The root partition has the “Root OS” or sometimes called “Host OS” (not technically correct in Hypervisor architectures).
Neither the Root nor the Guests VM’s control the PM Timer, APIC, … which means their concept of time in virtualized. Something else to be aware of is both the Root and Guest virtual processors share the physical processors in a Round Robin like fashion.
Now let’s take a very important Windows Performance Counter - % Processor Time. This performance counters details the total amount of CPU being used by a process, … (aka the CPU is not idling waiting for work). There are other % Processor Time counters for that detail process time on a CPU like Internet Explorer running.
Now for the lie. Check the picture below. On the guest I am running a CPU spinner program to make it 100% busy. You can see perfmon running in the guest shows 100% utilization and the root shows ~85% utilization using the Hyper-V Hypervisor Guest Run Time counters.
So which is correct? Actually they both are. The guest is using 100% of the CPU it has been given by the Hypervisor and each time the guest it running the CPU is busy running the spinner code. Thus the guest is using 100% or a Virtual CPU.
The Hyper-V Hypervisor Logical Processor counters are showing Logical (aka Physical) Processor utilization. This is what you are used to % Processor Time actually showing.
So if you are doing performance analysis and using performance counters be aware the counters in the Guest Virtual Machine “lie” so to speak. What you need to use are the Hyper-V Hypervisor Performance Counters in the Root to get Physical Processor usage.
