granth's blog : Performance

VSTS Pioneer TFS2010 Dogfood Server: Statistics October 2009

grantholliday — Fri, 06 Nov 2009 09:21:36 GMT

(See this summary on the VSTS Pioneer dogfood server and all the other posts.)

As with the last few months, October was another busy month for dogfooding TFS. Since the Pioneer server has been such a success for Work Item Tracking and Source Control, we’ve really ramped up our efforts to dogfood every single feature as much as we can. The benefits of having everything online really lets us make sure our end-to-end integration is working as well as feel the pain before we share it with our customers. The most notable changes this month are:

Each of the different business units within VSTS have setup their own SharePoint MOSS Dashboards with bug trends, etc.
We’re now using Team Build 2010 with Gated Checkin on our part of the tree (we still have to follow the divisional processes of custom nightly build scripts at this stage)
We enabled Test Case Management features and are now dogfooding the Test Case Management (TCM) features for our major test passes
We setup and enabled the Lab Management functionality on our server and we’re starting to dogfood that across the organization as well

Here’s the latest stats:

Recent Users: 915

Version Control

Version Control Users: 622
Files: 11,053,314
Compressed File Sizes: 284,480
Uncompressed File Sizes: 870,228
Checkins: 29,275
Shelvesets: 7,249
Merge History: 10,897,164
Pending Changes: 13,344
Workspaces: 1,607
Local Copies: 382,464,960

Work Item Tracking

Users with Assigned Work Items: 2,607
Total Work Items: 284,813
Areas and Iterations: 2,910
Work Item Versions: 2,892,664
Work Item Attachments: 115,501
Work Item Queries: 5,728

Commands (last 2 weeks in October)

Work Item opens: 154,788
Work Item queries: 120,537
Work Item updates: 40,242
Checkins: 2,630
Uploads: 10,440
Shelves: 4,553
Downloads: 251,032
Gets: 41,615

DevDiv TFS Server

On top of enabling all these new scenarios on the Pioneer server, we also managed to get the main DevDiv TFS server upgraded from SQL 2005 to SQL 2008. This was the first step to getting the server to TFS2010. I can’t say it was all smooth sailing, but it looks like we’re through the worst of it now (man, what a fortnight…). The good news is that all of the extra performance optimizations we had to make to cope with our dataset sizes also made it into TFS2010 product.

Although no install guide will tell you, I highly recommend you put on SQL 2008 SP1 CU4 if you’re running a large, busy TFS2008 or TFS2010 server – in the processes of testing the upgrade and after the actual upgrade to SQL 2008, we hit these bugs which the SQL team quickly reacted and fixed for us:

KB967178 - When you restore a SQL Server 2005 backup file in SQL Server 2008, the operation takes much longer than when you restore the same backup file in SQL Server 2005 (fixed in SQL08 SP1 CU1)
KB970823 - CPU cost increases suspiciously when the number of threads working on the same index increases (fixed in SQL08 SP1 CU3)
KB970133 - Backup Log hangs when using compressed backups and the destination drive runs out of disk space. SQL Restart is required (fixed in SQL2008 SP1 CU4)

Now, onto the next Pioneer upgrade.

TFS2010: SQL Queries for TFS Statistics

grantholliday — Fri, 23 Oct 2009 08:27:09 GMT

Brian has been posting TFS statistics from the Developer Division dogfood TFS server for many years. I’ve updated the queries from TFS2008 to work with Team Foundation Server 2010 Beta 2. Here are the statistics for the TFS2010 Beta 2 server that we have been using internally since August 2009.

The queries

With the default permissions in SQL, mere mortals will not have enough access to run these queries. In fact, only TFS administrators should be able to even connect to SQL at all. To run these queries, you essentially need db_datareader rights on the Tfs_Configration and any Tfs_Collection* databases that you want statistics on.

In general, you do NOT want to query the TFS operational databases. The only supported API is the Microsoft.TeamFoundation.* Object Model. If you build applications that read the SQL store directly, you are likely to impact TFS performance and it is likely to break in the next version if the database schema changes. You’ve been warned! (Running these queries are OK periodically though :)

-- Change this to the name of your collection DB. You’ll need to run these queries for each of your collection DBs.
USE Tfs_DefaultCollection
GO

-- Recent Users
select count(distinct IdentityName) as [Recent Users] from tbl_Command with (nolock)

-- Users with Assigned Work Items
select count(distinct [System.AssignedTo]) AS [Users with Assigned Work Items] from WorkItemsAreUsed with (nolock)

-- Version Control Users
select COUNT(*) AS [Version Control Users] from [Tfs_Configuration].[dbo].tbl_security_identity_cache as ic JOIN tbl_Identity as i ON i.TeamFoundationId=ic.tf_id where ic.is_group = 0

-- Total Work Items
select count(*) AS [Total Work Items] from WorkItemsAreUsed with (nolock)

-- Areas and Iterations
select count(*) AS [Areas and Iterations] from tbl_nodes with (nolock)

-- Work Item Versions
select count(*) AS [Work Item Versions] from (select [System.Id] from WorkItemsAreUsed with (nolock) union all select [System.Id] from WorkItemsWereUsed with (nolock)) x
-- Work Item Attachments
select count(*) AS [Work Item Attachments] from WorkItemFiles with (nolock) where FldID = 50
-- Work Item Queries
select count(*) AS [Work Item Queries] from QueryItems with (nolock)

-- Files
select count(*) as [Files] from tbl_VersionedItem vi with (nolock) join tbl_Version v with (nolock) on v.ItemId = vi.ItemId where VersionTo = 2147483647

-- Compressed File Sizes
select (sum(convert(bigint,OffsetTo - OffsetFrom + 1)) / (1024 * 1024)) AS [Compressed File Sizes] from tbl_Content with (nolock)

-- Uncompressed File Sizes
select (sum(FileLength) / (1024 * 1024)) AS [Uncompressed File Sizes] from tbl_File with (nolock)

-- Checkins
select max(ChangeSetId) AS [Checkins] from tbl_ChangeSet with (nolock)

-- Shelvesets
select COUNT(*) AS [Shelvesets] from tbl_Workspace with (nolock) where type='1'

-- Merge History
select SUM(st.row_count) AS [Merge History] from sys.dm_db_partition_stats st WHERE object_name(object_id) = 'tbl_MergeHistory' AND (index_id < 2)

-- Pending Changes
select count(*) AS [Pending Changes] from tbl_PendingChange pc with (nolock) join tbl_Workspace w with (nolock) on pc.WorkspaceId = w.WorkspaceId where w.Type = 0
-- Workspaces
select COUNT(*) AS [Workspaces] from tbl_Workspace with (nolock) where type='0'
-- Local Copies
select SUM(st.row_count) AS [Local Copies] from sys.dm_db_partition_stats st WHERE object_name(object_id) = 'tbl_LocalVersion' AND (index_id < 2)

-- Command Counts
select Command, count(*) as [Execution Count] from tbl_Command with (nolock) WHERE Command IN ('QueryWorkitems', 'Update', 'GetWorkItem', 'Get', 'VCDownloadHandler', 'Checkin', 'Upload', 'Shelve') GROUP BY Command, Application ORDER BY [Application],[Command]

These are the core statistics to get you started. I need to add in some more on Team Build, Web Access, Excel & Project usage, etc. Let me know what other statistics you think would be useful.

If you’d like to share your statistics with me and/or the TFS team, send me a link or email them privately. We’d love to know what sort of usage patterns people are using the product in.

I’m also working on a plugin for the forthcoming TFS Best Practices Analyzer that will allow you to run these queries on a schedule and publish your own TFS statistics. By running it on a schedule, you will be able to track growth in size and usage over time and help you do some capacity planning. Stay tuned for that.

VSTS Pioneer TFS2010 Dogfood Server: Statistics August 2009

grantholliday — Thu, 27 Aug 2009 21:44:00 GMT

(This blog post is part of a series of posts on the new VSTS Pioneer TFS dogfood server.)

Everybody loves statistics, so here are the statistics for the first month on the VSTS Pioneer server.

Recent Users: 415

Work Item Tracking

Users with Assigned Work Items: 2,454
Work Items: 111,319
Areas & Iterations: 2,811
Work Item Versions: 1,056,416
Work Item Attachments: 50,833
Work Item Queries: 2,965

Version Control

Files & Folders: 8,561,565
Compressed File Sizes (MB): 325,080
Uncompressed File Sizes (MB): 856,859
Checkins: 18,867
Shelvesets: 1,982
Merge History: 1,789,333
Pending Changes: 48,155
Workspaces: 727
Local Copies: 184,231,208

These statistics are captured by using the TFS Best Practices Analyzer and a custom configuration file. The configuration tells the BPA tool to run a series of SQL queries and dump the results to an XML file. I have then created an XSLT that transforms the results into a more user-friendly HTML file. The BPA tool is then scheduled to run every night and generate a fresh set of statistics. I’m working with the TFS BPA team to get this configuration built into the next version of the BPA tool – I don’t have an ETA on when that will ship, but stay tuned.

This month I’ve also pulled the last 30 days of PerfMon counters to see how our hardware choices are holding up. If you’d like to setup something like this for yourself, have a look at my previous posts.

Disk Performance

Things are looking OK on the IO front. 95% of the time we’re pushing 45MB/sec (Read) and 33MB/sec (Write) or less to and from our storage. The response time is <40ms which is nothing to worry about for the configuration we have.

LogicalDisk(_Total)	50th Percentile	95th Percentile	99th Percentile
Avg. Disk sec/Read	0.004	0.039	0.371
Avg. Disk sec/Write	0.003	0.023	0.063
Disk Read Bytes/sec	2,142,548	44,926,021	94,273,782
Disk Write Bytes/sec	1,908,518	33,029,252	70,802,346
Avg. Disk Bytes/Transfer	42,867	133,887	436,920

Memory

The memory usage on our SQL server has remained completely constant. This is because SQL is set to use a fixed amount and Analysis Services & Reporting Services share the rest.

CPU Performance

(Avg: 32%). The majority of this usage is SQL Server, with a small portion going to Analysis Services & Reporting Services. It’s nice to see that we still have plenty of capacity left for the usage to grow.

That’s all for statistics, stay tuned for more next month.

TFS Performance Report Pack now works with SSRS2005

grantholliday — Wed, 24 Jun 2009 06:40:14 GMT

Back in February, I released some of the reports that were created for our internal TFS servers on my blog as the TFS Performance Report Pack. I got countless emails and comments on the fact that some of them didn’t work with SQL Reporting Services 2005.

Fortunately, Jim Saunders from the Developer Support team at Microsoft (who have a great blog!) has re-created the three reports in the pack that required SQL2008 Reporting Services. Just download a different version of the reports from here and replace the ones in the original zip file.

From the support team blog:

...He lists SQL Server 2008 Reporting Services as a requirement, with a note that they should work in SSRS 2005. It turns out some of the reports function under both SSRS 2008 and 2005, some do not. Several posters have mentioned incompatibilities with some reports and SSRS 2005.

These reports are compatible with either SSRS version:

Server Status - Historical Performance Trends.rdl

Server Status - Recent Performance Trends.rdl

Server Status - Top Users Bypassing Proxies.rdl

These reports require SSRS 2008:

Execution Time Summary.rdl

Execution Time for User.rdl

Server Status - Source Control Request Queue.rdl

I have created versions of the three reports above that are functionally similar to Grant's reports, but also work with SSRS 2005. You can download my updated reports here. To use my reports, install Grant's TFS Performance Report Pack, and replace the three reports with the versions in the zip file I provided.

Large Team Foundation Server Performance Characteristics

grantholliday — Thu, 21 May 2009 20:27:42 GMT

Since joining Microsoft, I’ve become intimately familiar with running a TFS server for ~3,500 users in Developer Division and the performance characteristics of it.

One thing I’ve learnt is that Performance Counters rule. You might observe the server being “slow” and you might notice that it “takes a while” to do certain operations – but you need evidence to back up your claims before anybody will take you seriously. The evidence that everybody has access to, is reliable and people take seriously are the perf counters built into windows.

See my previous posts on Relogging Perfmon binary log files to SQL and Querying Perfmon data from SQL for more details on setting up performance counter logs.

If I think about the problems we’ve overcome in the last 12 months, the issues come down to these:

IO – If the “LogicalDisk\Avg. Disk sec/Transfer” perf counter for any of your disk drives is more than 0.030 (30ms) – then you’re hosed. This counter is a primary indicator of disk latency. Get that fixed before doing anything. (see below for more details)
Workspace Mappings – If you have unnecessary paths in your workspace mappings, then Get() will be much slower than it needs to be. E.g. DON’T map $/ to C:\Code and think that everything will be good. A root mapping isn’t truly a bad thing but if you aren’t careful it can lead to unexpected and potentially slower results.
Latency/Download requests – Proxy servers help here by offloading Download() requests from the main server. Doesn’t help Work Item tracking.
CPU – Processor performance isn’t linear. If you’re running higher than ~70% CPU for periods of time, then you need to increase your processing capacity.
SQL indexes/fragmentation – Sometimes the TFS SQL Jobs that update statistics & rebuild/reorganize indexes stop running, or don’t run for whatever reason. Check that the SQL jobs are running successfully and check for index fragmentation.

The tools you can use to diagnose performance issues are:

PerfMon. Setup a perfmon counter log for the important counters. Track them and work out what’s “normal” for your load/environment
TfsActivityLogging database. Dive into this database and look for trends, heavy users, heavy tools, etc. Understand where your load is coming from.
- Download & install my TFS Performance Report Pack and look at the Execution Time Summary report.
TfsServerManager.exe (Comes with the Team Foundation Server Power Tools, see Brian’s blog for more details)
- No shortcut gets created. Run it from "C:\Program Files\Microsoft Team Foundation Server 2008 Power Tools\TfsServerManager.exe"
- If users are reporting a problem, try and catch it while it’s currently executing. Look at the “Source Control Request Queue” report. Is their request on top?
- Run the following query a few times in SQL to see if any blocking is occurring. If the same spid hangs around for a while, run DBCC INPUTBUFFER(spid_here) to see what stored procedure it is and try and match that to a TFS command. e.g. prc_Get = Get()

SELECT a.status, a.*
FROM    sys.sysprocesses a
WHERE   spid > 50
        and spid <> @@spid
        and blocked = 0
        and EXISTS ( SELECT *
                     FROM   sys.sysprocesses b
                     WHERE b.blocked = a.spid)

Disk Latency

To determine if you are having significant issue with disk latency you should use the following performance counters:

Object: [Physical Disk] or [Logical Disk]
Counter: [Avg. Disk Sec/Transfer]
Instance: Ideally you collect this for individual disks however you may also use [_Total] to identify general issues. If [_Total] is high then further collections can be taken to isolate the specific disks affected.
Collection Interval: Ideally you should collect at least every 1 minutes. The collection should be run for a significant period of time to show it is an ongoing issue and not just a transient spike. 15 minutes is minimum suggested interval.
Issue Thresholds (seconds):
- < 0.020: Normal time and no I/O latency issues are apparent
- > 0.00 – 0.050: You may somewhat concerned. Continue to collect and analyze data. Try to correlate application performance issues to these spikes
- > 0.050 – 0.100: You are concerned and should escalate to SAN administrators with your data and analysis. Correlate spikes to application performance concerns.
- > 0.100: You are very concerned and should escalate to SAN administrators. Correlate spikes to application performance concerns.

Announcing TFS Performance Report Pack

grantholliday — Wed, 04 Feb 2009 03:00:00 GMT

Update 6/23/2009: Due to popular demand, Jim from the Developer Support team at Microsoft has re-created the three reports that required SQL2008 Reporting Services. See the support team blog for more details.

I’m on the team that runs the busiest Team Foundation Server at Microsoft. The Developer Division instance has over 3,500 active users and processes over 10,000,000 source control and work item tracking requests per day (not including the requests that our 5 proxy servers handle).

We have a responsibility to ensure that the server is performing as expected and to identify any efficiencies that can be made in the server or the tools. To do this, we have created a number of reports that we use ourselves and make available to our own users.

Download

We’re now making these reports available to you to install on your own server and monitor your TFS server’s performance. The plan is to eventually roll these into a power tool release or a subsequent release of VSTS but I’ll let you know more about that when it happens.

Download from here TFS Performance Report Pack.zip (~250Kb)
Download the SQL Server 2005 compatible reports from here SRS2005CompatibleReports.zip (~13Kb)

Requirements

SQL Server Reporting Services
A user with read-only access to the TfsActivityLogging database
A shared datasource to connect the report to (see below)
Command Logging enabled - This logging is enabled by default in TFS 2008.

In TFS 2005 you will have to enable the commandLogging flag in the Global Web.Config file (and the query for the report will be a little different).

Installing

The report pack consists of a ZIP file with a number of Report Definition (*.rdl) files. These files are designed to be deployed onto your existing TFS reporting server, e.g. http://your-tfsserver/Reports/

As a TFS administrator, extract the files to your PC
Open http://your-tfsserver/Reports/
Create a new folder in Reporting Services called “Server Status”
Create a new shared data source called “TfsActivityReportDS” and set the connection string to:
1. Data Source=localhost;Initial Catalog=TfsActivityLogging
2. Credentials: domain\user that has access to the TfsActivityLogging database
3. Use as windows credentials when connecting to the data source

Here’s an overview of what the reports look like and what questions you can answer with them.

Execution Time Summary

This report visualizes the load, in this case reflected by total execution time, on the server from two axis: users and commands.

Use this report when you want to know:

Which commands account for the largest load on the server?
Which tools / or users are putting the biggest load on the server?

Server Status - Source Control Request Queue

Source Control is undoubtedly the application that consumes the most resources on an Application Lifecycle Management (ALM) Server. Across the day, a series of requests get queued to be processed as transactions are committed to the database. This report provides a view into that queue.

Use this report when you want to know:

If a request is blocking source control operations and for how long
How healthy is the performance of version control on this hardware?

Lots of red means that you have some long running operations and you may have some problems.

Server Status - Top Users Bypassing Proxies

In the past, I’ve blogged a query to get this information – How many user’s are not using a TFS Proxy server? Internally, we have setup a scheduled subscription that emails this report twice a week.

IT departments strive to provide the best level of service to their users. Hardware requirements planning and setting up proxies are activities that ensure optimal performance for their internal teams when interacting with team Foundation Server. This report allows administrators a view into which users are not complying with internal guidelines and hence decreasing overall server performance.

Server Status - Historical Performance Trends

This report serves as a summary of the average response time for two of the Team Foundation Server subsystems: Work Item Tracking and Version Control

Use this report when you want to know:

How long are users, on average, waiting for a subsystem to process their request
Which days of the week are the most critical when it comes to performance

Server Status - Recent Performance Trends

This report provides more data granularity about the performance of the server. We start with a view into the server average response time, now looking at the entire picture instead of broken down by subsystem. We then follow with charts relating information about version control downloads and average response time distributions for the same time period.

Use this report when you want to know:

The correlation between degraded server performance and average response times by the subsystems
How does a large number of downloads affect overall server performance
Overall health indicator of the server

I hope that you find these reports useful. Please send any questions or feedback as comments to this post, or contact me via email.

Analyzing TFS performance with TfsActivityLogging and custom User Agents

grantholliday — Sat, 24 Jan 2009 10:13:38 GMT

Within Developer Division at Microsoft, we run a very busy server. We’re up to almost ~3,600 active users.

Just over 6 months ago, the custom tools that our build labs were using were putting an extraordinary strain on the server. More than 40% of the load on the server was just from these tools. After some fairly deep analysis into what the tools were doing and how they were using TFS, we identified some inefficiencies and were able to address them in the tools.

The two most useful resources for tracking down these issues and addressing them were:

TfsActivityLogging database
Setting a custom User Agent string

These are examples of the reports that we came up with to track our progress at making the efficiencies. They show total execution time from the TfsActivityLogging database split up by user and by command. You can also select a particular user and drill down and see just their commands. This showed us where our efforts were best spent.

Then the report also shows the details with actual Execution Time, Execution Count and Average Response Time (Time divided by Count) for each command.

Here’s a sample SQL query you can use to start looking at this data yourself:

SELECT
    Application,
        Command,
    SUM(ExecutionTime/1000) as TotalTime_ms,
    SUM(ExecutionCount)
FROM TfsActivityLogging.dbo.tbl_Command with (nolock)
GROUP BY Application, Command

Now I know that as soon as I post this, I’m going to get flooded with comments/emails asking “Can you send me the report file?”. Don’t worry, I’m working with Mario and we should be able to get some goodness posted up on Codeplex.

Setting a custom user agent for the TFS Object Model

Once we had identified which users were contributing the most load, we needed to map their scripts and tools to TFS commands.

One of the columns within the TfsActivityLogging database that gets logged with every request to TFS is the “UserAgent” column. It turns out that you can set this to an arbitrary value using the TeamFoundationServer.ApplicationName property in the TFS object model.

using System;
using System.Text;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.VersionControl.Client;
namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            TeamFoundationServer tfs = TeamFoundationServerFactory.GetServer("http://tfs-server:8080", new UICredentialsProvider());
           TeamFoundationServer.ApplicationName = "MyApplication v1.1 - Phase1";

            VersionControlServer vcs = (VersionControlServer)tfs.GetService(typeof(VersionControlServer));
            Changeset cs = vcs.GetChangeset(1);
        }
    }
}

By setting this value dynamically throughout the custom scripts, we were able to map stages in the build process to groups of actual TFS commands. Then we were able to refactor the build scripts and optimize them for better TFS performance.

TFS Management Pack for System Center Operations Manager 2007 shipped

grantholliday — Thu, 27 Nov 2008 05:01:17 GMT

The VSTS Rangers have shipped the TFS 2008 Management Pack for SCOM2007. It is also published in the official System Center Operations Manager 2007 Catalog on TechNet (choose “System Center Operations Manager 2007” from the dropdown list and enter “TFS” as keyword).

It takes out a new record for the longest name of a Microsoft download: Visual Studio Team System 2008 Team Foundation Server Management Pack for System Center Operations Management 2007 – English

The VSTS Rangers are a community of Microsoft field staff that are experts in VSTS. Their mission is to create re-usable solutions to aid adoption problems and influence future versions of VSTS based on lessons learned.

In the past, they’ve shipped other solutions such as:

This product is a significant improvement for monitoring and management of TFS 2008.

Overview

The Team Foundation Server (TFS) 2008 Management Pack monitors exposed services of TFS. This management pack includes event rules and monitors. It is designed to proactively monitor quality and availability of TFS services and even automate recovery in some scenarios.

Feature Summary

Auto discovery of TFS components
Leverages ASP.NET MP and SQL MP
Implements containment hierarchy reflecting logical architecture of the Product
Implements a proper health model using Monitors
Contains tasks, diagnostic and recovery for certain failures
Events indicating service outages
Alerts indicating configuration issues and connected data source changes
Verification that all dependant services are running
Targeted running of BPA against TFS Servers from Operator Console
New monitoring capabilities can be easily added

I’m looking forward to hearing what people are doing with it. Please contact me with any feedback you have of this in your environment.

Querying Perfmon data from SQL

grantholliday — Sat, 08 Nov 2008 03:41:33 GMT

In a previous post I talked about Relogging Perfmon binary log files to SQL. The next obvious step is to query the data from the database.

There are two ways to do this:

Use Perfmon itself to connect to the SQL data source
Use another tool to query the SQL data source directly

1. Using Performance Monitor

Once you open Performance Monitor, press the View Log Data button (Ctrl-L):

Then on the Source tab, if you have the ODBC System DSN configured to connect to your performance database, it will show up.

This allows you to view the log files directly from SQL, just as if you had loaded the binary log files.

2. Querying SQL directly

At first glance, the schema doesn’t seem very intuitive. But with a little digging, it’s not too bad.

Whenever you use PerfMon or relog to log to a database, it automatically sets up this schema for you. There are three tables:

DisplayToID – This lists all the log sets that are in the database
CounterData – This is where the actual counter values are stored
CounterDetails – This is where the metadata about each counter / object / instance is stored

Here are some queries that I’ve found useful:

Get all computers that have counter data logged:

SELECT DISTINCT MachineName
FROM CounterDetails
ORDER BY MachineName

Get available object names for a particular computer:

SELECT DISTINCT ObjectName
FROM CounterDetails
WHERE MachineName = ‘\\MYSERVER’
ORDER BY ObjectName

Get counter names for a particular computer and object:

SELECT DISTINCT CounterName
FROM CounterDetails
WHERE MachineName = ‘\\MYSERVER’
AND ObjectName = ‘Processor’
ORDER BY CounterName

Get instance names for a particular computer, object and counter:

SELECT DISTINCT InstanceName
FROM CounterDetails
WHERE MachineName = ‘\\MYSERVER’
AND ObjectName = ‘Processor’
AND CounterName = ‘% Processor Time’
ORDER BY InstanceName

Get counter values for a particular computer, object, counter and instance. Name the column appropriately:

SELECT
    CAST(LEFT(CounterDateTime, 16) as smalldatetime) AS CounterDateTime,
    REPLACE(CounterDetails.MachineName,'\\','') AS ComputerName,
    CounterDetails.ObjectName + ISNULL('(' + CounterDetails.InstanceName + ')','') + '\' + CounterDetails.CounterName AS [Counter],
    CounterData.CounterValue
FROM CounterData
    INNER JOIN CounterDetails ON CounterData.CounterID = CounterDetails.CounterID
    INNER JOIN DisplayToID ON CounterData.GUID = DisplayToID.GUID
WHERE CounterDetails.ObjectName = 'Processor'
    AND    CounterDetails.CounterName = '% Processor Time'
    AND    CounterDetails.MachineName = '\\MYSERVER’
    AND CounterDetails.InstanceName = '_Total'
ORDER BY CounterData.CounterDateTime

There’s a few nuances in the schema which creates complexity in this query. Let me try and explain:

CounterDateTime is a char(24) instead of a datetime. By trimming it to 16 characters, this allows it to be converted to a smalldatetime at the ‘minutes’ resolution – not seconds.
MachineName includes ‘\\’, we want to trim it to make it look pretty.
InstanceName can be null. An example is the Memory\Pages/sec object & counter. The instance value for this combination is null. So we only show the instance name if it’s not null.

Of course now that you have the data being collected, post-processed and queryable in SQL – the next logical step is to make it reportable. Here is a report that I’ve built to display this data. I’ll talk about it more in a future blog post.

PerfMon is not collecting LogicalDisk or PhysicalDisk statistics

grantholliday — Sat, 08 Nov 2008 01:46:41 GMT

If the LogicalDisk or PhysicalDisk objects don’t appear in PerfMon for you, then they may have been disabled automatically by the performance monitoring API.

This is what Mark Friedman has to say about it in Microsoft Windows Server 2003 Performance Guide from the Windows 2003 Resource Kit:

To maintain the integrity of the performance data and to improve reliability, the performance monitoring API disables any performance DLL that returns data in the incorrect format, causes an unhandled program fault, or takes too long to return the performance data. As a result of an error condition of this magnitude, a field is added to the registry in the HKLM\System\CurrentControlSet\Services\<Service-name>\Performance key named Disable Performance Counters. When Disable Performance Counters is set to 1, no performance monitoring application will be able to gather the counters that the disabled Perflib is responsible for gathering. Perflib Event ID 1017 and 1018 messages are written to the Application event log at the time the Disable Performance Counters flag is set.

In the case of disk counters, the service name is ‘PerfDisk’. Here’s what it looks like in the registry:

As well as re-enabling the collection, you can increase the Collect Timeout so that it doesn’t happen as often.

granth's blog : Performance

VSTS Pioneer TFS2010 Dogfood Server: Statistics October 2009

TFS2010: SQL Queries for TFS Statistics

VSTS Pioneer TFS2010 Dogfood Server: Statistics August 2009

TFS Performance Report Pack now works with SSRS2005

Large Team Foundation Server Performance Characteristics

Disk Latency

Further Reading

Announcing TFS Performance Report Pack

Download

Requirements

Installing

Execution Time Summary

Server Status - Source Control Request Queue

Server Status - Top Users Bypassing Proxies

Server Status - Historical Performance Trends

Server Status - Recent Performance Trends

Analyzing TFS performance with TfsActivityLogging and custom User Agents

Setting a custom user agent for the TFS Object Model

TFS Management Pack for System Center Operations Manager 2007 shipped

Querying Perfmon data from SQL

PerfMon is not collecting LogicalDisk or PhysicalDisk statistics