Jimmy May, Aspiring Geek: SQL Server Performance, Best Practices, & Productivity : Best Practices

Xperf: Install Windows Performance Toolkit (WPT) with 242MB Download, Not 2.5GB Windows 7 SDK: Part 2

Jimmy May — Tue, 24 Nov 2009 19:18:00 GMT

In my first post on Xperf, I described the simple 4.4MB download necessary to install the Windows Performance Analyzer (WPA) & the other tools which comprise the Windows Performance Toolkit (WPT), including Xperf.

I was surprised to learn that the WPT is no longer available in a discrete download.

As I explained in Part 1, the nominal way to install it now is through a humongous 2.5GB download of the Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 Service Pack 1.

Unlike my buddy & fellow MCM Jens Suessmeyer (twitter|blog) whose “Developer hearted / Relational minded” blog betrays his expertise, downloading multiple gigabytes of developer tools on my workstations would be a waste of bandwidth & disk space.

Fortunately there’s a much faster, easier way. Here are the 8 steps to do so. The key step is selecting only “Win32 Development Tools” in the SDK Installations Options (step 6).

1. Browse to the Windows 7 SDK download page
You can go directly to the download page (the fourth URL below), but let’s start with the WPA home page & drill through.

<ADD> Don’t get distracted now, but there’re lots of great items here to bookmark & peruse later. </ADD>

Windows Performance Analysis Developer Center
http://msdn.microsoft.com/en-us/performance/default.aspx

Windows Performance Analysis Tools
http://msdn.microsoft.com/en-us/performance/cc825801.aspx

Windows SDK
http://msdn.microsoft.com/en-us/windowsserver/bb980924.aspx

Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 SP1
http://www.microsoft.com/downloads/details.aspx?FamilyID=c17ba869-9671-4330-a63e-1fd44e0e2505

2. Run the Win 7 SDK Web Executable
Clicking the download button initiates the download of winsdk_web.exe. Save/run the file (493KB).

3. Installation Screen Shot: Windows SDK Setup Wizard
Click the Next > button on the first page of the Windows SDK Setup Wizard.

4. Installation Screen Shot: EULA
Choose “I agree” & click the Next > button on the End-User License Agreement page.

5. Installation Screen Shot: Install Locations
Select destination folders for "Documentation and Tools" & "Samples", then click the Next > button. The default location root is:

C:\Program Files\Microsoft SDKs\Windows\v7.0

6. *KEY STEP* Installation Screen Shot: Installation Options
This is the key step in avoiding the SDK’s 2.5GB default download (4.5GB installed). Clear all checkboxes except one: Win32 Development Tools, which is “only” a 242MB download requiring 529MB to install.

Developer Tools > Windows Development Tools > Win32 Development Tools

7. Install
Click Next > on the Begin Installation page. patiently watch the progress of the Installation Progress page. On the Installation Complete page, consider enabling the checkbox labeled, “Help improve our products by sending your installation report to Microsoft…” Click the Finish button.

8. Find & Install the Appropriate WPT MSI
Now that the selected SDK components are installed, we can install WPT. You’ll find the wpt_x86.msi & wpt_x64.msi files in the folder defined in step 5 above, by default:

C:\Program Files\Microsoft SDKs\Windows\v7.0

Walk through the Microsoft Windows Performance Toolkit Setup:

On the Welcome page, click Next >
Accept the EULA, click Next >
Choose Setup Type, click the Custom button
On the Custom Setup page, click the Browse button, & on the Change Current Destination Folder page consider modifying the default
C:\Program Files\Microsoft Windows Performance Toolkit\
to
C:\xperf\
Click OK on , Click Next
On the Ready to Install page, click the Install button to invoke the Installing Microsoft Windows Performance Toolkit page
Click the Finish button on the Completing… page

Jimmy May, MCM, MCDBA, MCSE, MCITP: DBA + DB Dev | Principal Performance Engineer: SQL Server | MS IT Assessment Consulting & Engineering Services (A.C.E.)
http://msinfosec.com
http://blogs.msdn.com/jimmymay
www.twitter.com/aspiringgeek
jimmymay@microsoft.com
317.590.8650
Don’t practice in front of the CIO. A professional prepares ahead of time.
—Pond's 12th Law

Xperf: Install Windows Performance Toolkit (WPT) with 242MB Download, Not the Entire 2.5GB Windows 7 SDK: Part 1

Jimmy May — Sun, 22 Nov 2009 23:00:00 GMT

Say It Ain’t So!
In my first post on Xperf, I described the simple 4.4MB download necessary to install the Windows Performance Analyzer (WPA) & the other tools which comprise the Windows Performance Toolkit (WPT), including Xperf.

I was surprised to learn that the toolkit is no longer available in a discrete download.

The only way to install it now is through the Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 Service Pack 1.

The default download is a honkin’ 2.5GB, 4.5GB installed, but all I want is the toolkit.

Why the Change?
The version of WPT that was independently posted was incompatible with Win7. The team wanted to remove it before a large number of people downloaded it and had issues with their freshly installed copy of the RTM bits.

How to Install WPT
Principal Development Lead Adam Smith helped me short-circuit the big download so we need download “only” 242MB.

Per Adam, the web install allows us to choose just the win32 dev tools for a far more reasonable download—242MB instead of 2.5GB.

In Part 2 I’ll detail how to do so.

Jimmy May, MCM, MCDBA, MCSE, MCITP: DBA + DB Dev | Principal Performance Engineer: SQL Server | MS IT Assessment Consulting & Engineering Services (A.C.E.)
http://msinfosec.com
http://blogs.msdn.com/jimmymay
jimmymay@microsoft.com
317.590.8650
Don’t practice in front of the CIO. A professional prepares ahead of time.
—Pond's 12th Law

SQLIOSim Parser by Jens Suessmeyer & Yours Truly

Jimmy May — Sun, 27 Sep 2009 14:32:00 GMT

New! Improved! v.3.3.0 Now Available on CodePlex!

Preamble
SQLIOSim Parser shreds the results of SQLIOSim.log.xml in seconds, allowing easier & more accurate interpretation of the output of SQLIOSim.

SQLIOSim is a tool written by CSS Engineer Bob Dorr. Many of you know Bob from the excellent insights he shares on the CSS SQL Server Engineers blog. Note that SQLIOSim is not a performance tool; it is designed to validate disk I/O subsystem correctness.

Back during MCM training I was asked about interpreting the output of SQLIOSim. I cited these references:

Understanding SQLIOSim Output
http://sqlblog.com/blogs/kevin_kline/archive/2007/06/28/understanding-sqliosim-output.aspx
MVP Kevin Kline’s conversation with yours truly

More Tidbits on SQLIOSim
http://sqlblog.com/blogs/kevin_kline/archive/2008/12/14/more-tidbits-on-sqliosim.aspx
Kevin’s conversation with Bob Dorr & Jose Fortuny

How It Works: SQLIOSim - Running Average, Target Duration, Discarded Buffers
http://blogs.msdn.com/psssql/archive/2008/11/12/how-it-works-sqliosim-running-average-target-duration-discarded-buffers.aspx
Bob Dorr documents key SQLIOSim metrics

How to use the SQLIOSim utility to simulate SQL Server activity on a disk subsystem
http://support.microsoft.com/kb/231619
Microsoft KB article

I’ll document these & other references in a subsequent post.

The Genesis of SQLIOSim Parser
During MCM training I lamented that, as cool as the tool is, I explained my practice of manually parsing the output, laboriously copying-&-pasting values into Excel in order to better wrap my brain around the results.

Serendipitously, Jens Suessmeyer was there & offered to write a parser.

Jens is not only a great friend, but one of the Microsoft’s rising stars. Before joining Microsoft he was an MVP, he’s a prolific contributor to our internal DLs, & he has an impressive depth & breadth of knowledge. Check out & subscribe to his blog. And now he’s provided yet another means to make our lives easier.

Fruitful Collaboration Results in Downloadable Version
I provided guidelines & Jens has followed through by writing SQLIOParserWin.exe. We’ve labored through several enhancements. Be clear, Jens did the heavy lifting here. The fruits of our collaboration are now available for review. Here's a link to the latest-&-greatest, currently v.3.3.0.

The parser has saved me loads of time whenever I use SQLIOSim & it will do the same for you. I’m on-site with a customer right now. I ran & parsed the real-life results of numerous tests in mere seconds instead of tedious & error-prone copying-&-pasting.

SQLIOSim Parser Usage

Note: A step ahead of me as usual, the latest-&-greatest version from Jens provides more sophisticated output than I've described here; stay tuned for details.

Download & execute SQLIOParserWin.exe
Point to a SQLIOSim log file (Figure 1)
Click the “Parse input file and show results” button to instantly display the ResultsPane window (Figure 2)

I analyze the results in Excel like this:

Type select & copy the results (Ctrl+A, Ctrl+C)
Paste into Excel
- Delete the empty first column
- Select all cells by typing Ctrl+A or clicking the Sheet Selector button (that’s the button at the intersection of the rows & columns)
  - Right-click a row button, click Row Height…, & change the value to 15 for all rows
  - With the sheet still selected, type Ctrl+1 to invoke the Format Cells dialog
    - On the Number tab, click Number, change “Decimal places” to 0 & check “Use 1000 Separator”
    - Click the Font tab & select something such as Consolas 9
    - Click OK to accept your changes & close the dialog
  - With the sheet’s contents still selected, “right size” the columns by double-clicking the boundary between columns A & B.
  - Select the TestTime column, type Ctrl+1 again, click the Number tab, select the Custom category, & enter this value for the Type: mm/dd/yyyy hh:mm:ss
  - Click OK to accept your changes & close the dialog

See Figure 3 for the final result. I’ll post the macro code on request. Note that Jens has even added row headers for us. He’s also added a row id so we can sort the results but still if we desire restore the original order.

Alternative Output Formats
There are two alternative ways to generate output:

Save the result in XML format by providing a path & file in the “Result OutputFileName” textbox & clicking the “Export result file to XML” button.

Click the “Display result Report” button to invoke an intermediate window which allows you to enter:
- CustomerName
- Path to customer logo
- Additional project information

The final report includes many relevant columns.

Feedback
We invite your feedback. Let us know what you think. Jens always eager to flex his coding skills. SQLIOSim Parser is great for ad hoc analysis. Future versions may include columns from the SQLIOSim config file which enhance enterprise compatibility.

Note: These images reflect: SQLIOParserWin.exe v.0.2.3.0. We'll be posting updated images sometime soon.

Figures
Here are the figures cited above.

Figure 1. Launch SQLIOParserWin.exe to expose the SQLIOSim Parser interface. Point to a SQLIOSim log file & click the “Parse input file and show results” button.

Figure 2. The ResultsPane.

Figure 3. Sample output after copy-&-pasting the results of SQLIOSim Parser into Excel & manipulating the worksheet’s format as described in the text.

Columns
SQLIOSim Parse provides the following columns. As stated, we may incorporate columns from the SQLIOSim config file in future versions.

RowId
DriveLetter
TestTime
FileName
Compression
Encryption
Sparse
TargetIODuration
RunningAverageIODuration
NumberOfTimesIOThrottled
IORequestBlocks
Reads
ScatterReads
Writes
GatherWrites
TotalIOTime
SectorSize
Cylinders
MediaType
SectorPerTrack
TracksPerCylinder
ReadCacheEnabled
WriteCacheEnabled
ReadCount
ReadTime
WriteCount
WriteTime
IdleTime
BytesRead
BytesWritten
SplitIOCount
StorageNumber
StorageManager

Jimmy May, MCM, MCDBA, MCSE, MCITP: DBA + DB Dev
Principal Performance Engineer: SQL Server
A.C.E.: Assessment Consulting & Engineering Services
http://msinfosec.com
http://blogs.msdn.com/jimmymay
www.twitter.com/aspiringgeek
Don’t practice in front of the CIO. A professional prepares ahead of time.
—Pond's 12th Law

Aspiring Geek Speaks This Week Yet Again: Perpetual Technologies (PTI) Back-to-School with Microsoft and Oracle Event

Jimmy May — Wed, 16 Sep 2009 16:47:00 GMT

Yesterday I flew to Minneapolis to speak at PASS MN on disk partition alignment. I left for the airport immediately afterward to get home to Indianapolis in order to speak at Perpetual Technologies, Inc. (PTI) 2nd Annual Back to School with Microsoft & Oracle Event.

PTI is home to some great folks, including my friends Kristin Sheibley, Ray Lucas (an IndyPASS colleague), & MVP nominee Arie “AJ” Jones, proprietor of the Programmer’s Edge blog. Here’s AJ’s post on the event.

I’ll be speaking along with many others, including my Microsoft colleagues Tom Pizzato & Joe Carver.

I’ve been asked to speak on two different topics:

Table Partitioning in SQL Server 2005 & 2008
Data can be stored by using the storage algorithms provided by Partitioned Tables and Indexes. Partitioning can make large tables and indexes more manageable and scalable. Learn keywords, key concepts, & see meaningful demonstrations on techniques related to table partitioning which are consistent with enhanced performance, scalability, & management.

Combining Transactional Replication & DB Mirroring
Database mirroring can be used in conjunction with replication to provide availability for the publication database. Database mirroring involves two copies of a single database that typically reside on different computers. Mirroring involves applying the transaction log from every insertion, update, or deletion made on the principal database onto the mirror database. Replication failover to a mirror is supported without the need to reinitialize replication. Learn & see how these High Availability options can work together in SQL Server, as well as pitfalls to avoid.

If you’re available, c’mon down!. There’s no charge, & you can register here.

Here’s the venue:

Junior Achievement Center
7435 North Keystone Avenue
Indianapolis IN 46240

sys.dm_db_index_operational_stats: Kick Your Assumptions: Part 1

Jimmy May — Mon, 03 Aug 2009 16:17:00 GMT

                  A two-part tale in which your humble correspondent interrogates
                  databases & Very Smart Persons (VSPs), & is forced to kick his ass-
                  umptions regarding a popular DMF.

Executive Summary
Documentation for the DMF sys.dm_db_index_operational_stats in BOL here & here as well as elsewhere is incorrect.

In fact, ops stats will not return information for all indexes, but lists only those indexes (& their partitions) in the metadata cache. On a newly re-started server, no user indexes will appear in the result set. This contradicts documentation.

An outer join to a system object such as sys.indexes on object_id & index_id is required to list all indexes & partitions.

Using op stats Returns Info ONLY on Indexes in the Metadata Cache
The following returns names & metadata only for those indexes (& their partitions) in the metadata cache. On a newly re-started, this will consist only of system objects.

--ops stats returns info only for a db’s indexes & their part’ns in cache
    --this contradicts documentation
--note: db context here is not mandatory
USE AdventureWorks;
GO
DECLARE @dbid sysname = DB_ID(); --SQL 2008 syntax
--DECLARE @dbid sysname; SET @dbid = DB_ID();
SELECT DB_NAME(@dbid) as db_name
    , OBJECT_NAME(object_id) as obj_name
    FROM sys.dm_db_index_operational_stats
        (@dbid, NULL, NULL, NULL)
    ORDER BY obj_name;

Use op stats for Info on All Indexes—System & User Objects
Obtaining information for all indexes (& their partitions) in a specific database—regardless of metadata cache status—including all system & user indexes, requires a query such as the following which uses an outer join to sys.indexes. (Note that both of the following queries are sensitive to the database context. Note also not validating the db & passing an invalid id to the DMF will produce undesired results.)

--to get ops stats to work as advertised
--return all rows for all indexes (& partitions) for a specific db
    --use an outer join between ops stats & sys.indexes
--note: db context is relevant
USE AdventureWorks;
GO
DECLARE @dbid sysname = DB_ID(); --SQL 2008 syntax
--DECLARE @dbid sysname;    SET @dbid = DB_ID();
SELECT DB_NAME(@dbid) as db_name
    , OBJECT_NAME(i.object_id) as obj_name
    , ops.*
    FROM sys.dm_db_index_operational_stats
            (@dbid, NULL, NULL, NULL) as ops
        RIGHT JOIN sys.indexes as i
           ON ops.object_id = i.object_id AND ops.index_id = i.index_id
    ORDER BY obj_name , i.index_id , i.partition_number;

Use ops stats for Info on All User Indexes
The following uses op stats returns information for all user indexes in a specific database—regardless of metadata cache status:

--use op stats to return all rows for all *user* indexes (& partitions)
    --for a specific db
    --combine outer join between ops stats & sys.indexes
    --w/ OBJECTPROPERTY
--note: db context is relevant
USE AdventureWorks;
GO
DECLARE @dbid sysname = DB_ID(); --SQL 2008 syntax
--DECLARE @dbid sysname;    SET @dbid = DB_ID();
SELECT DB_NAME(@dbid) as db_name
    , OBJECT_NAME(i.object_id) as obj_name
    , ops.*
    FROM sys.dm_db_index_operational_stats
            (@dbid, NULL, NULL, NULL) as ops
        RIGHT JOIN sys.indexes as i
           ON ops.object_id = i.object_id AND ops.index_id = i.index_id
    WHERE OBJECTPROPERTY(i.object_id,'IsUserTable') = 1
    ORDER BY obj_name , ops.index_id , ops.partition_number;

To Be Continued...
Stay tuned for Part 2 for additional details & documentation, including a walk-through, widespread misconceptions even amongst VSPs, & internals.

Jimmy May, MCM, MCDBA, MCSE, MCITP: DBA + DB Dev
Senior Performance Consultant: SQL Server
A.C.E.: Assessment Consulting & Engineering Services
http://msinfosec.com
http://blogs.msdn.com/jimmymay
Politics leads you in the direction of a belief. Data, if you follow them, lead you to truth.
—Michael Crichton

SQL CAT New Whitepaper: Data Compression: Strategy, Capacity Planning and Best Practices by Sanjay Mishra, et al.

Jimmy May — Fri, 29 May 2009 21:07:00 GMT

Data Compression: Strategy, Capacity Planning and Best Practices
Hot off the presses, the SQL CAT team has just published a new whitepaper for which I had the opportunity to provide a technical review. The author is SQL CAT best practices maven Sanjay Mishra. Contributors include SQL CAT member Sunil Agarwal and architects Marcel van der Holst & Peter Carlin. Besides yours truly, tech reviewers were Stuart Ozer, Lindsey Allen, Juergen Thomas, Thomas Kejser, Burzin Patel, Mike Ruthruff, & Prem Mehra of SQL CAT as well as Joseph Sack, Cameron Gardiner, MVP Glenn Berry, Paul Randal (SQLskills.com), & David P Smith (ServiceU Corporation).

Put the Big Squeeze on Your Data

<image source>

The data compression feature in the Microsoft SQL Server 2008 database software can help reduce the size of the database as well as improve the performance of I/O intensive workloads. However, extra CPU resources are required on the database server to compress and decompress the data, while data is exchanged with the application. Therefore, it is important to understand the workload characteristics when deciding which tables to compress. This white paper provides guidance on the following:

How to decide which tables and indexes to compress
How to estimate the resources required to compress a table
How to reclaim space released by data compression
The performance impacts of data compression on typical workloads

See Sanjay’s post at the SQL CAT blog For more information, refer to the whitepaper Data Compression: Strategy, Capacity Planning and Best Practices.

Administrivia

Jimmy May, MCDBA, MCSE, MCITP: DBA + DB Dev
Senior Performance Consultant: SQL Server
A.C.E.: Assessment, Consulting, & Engineering Services
http://msinfosec.com
http://blogs.msdn.com/jimmymay

This post was written with the PracticeThis.com plugin for Windows Live Writer

The first thing to do in a cardiac arrest is to take your own pulse.
—The Fat Man, House of God, Samuel Shem

Disk Partition Alignment (Sector Alignment): Make the Case: Save Hundreds of Thousands of Dollars

Jimmy May — Fri, 08 May 2009 21:15:00 GMT

I recently collaborated with Microsoft PFE Daniel Janik to create a template to make the case for disk partition alignment. Perhaps your customers or stakeholders within your organization can benefit. This work was recently broadcast throughout PFE DLs as well as the April 2009 SQLRAP Newsletter. Thanks also to Cindy Gross & Ward Pond for their keen eyes.

This information is available elsewhere on my blog & the Internets. It’s provided here as public service for the first time for convenient access.

Make the Case for Disk Partition Alignment (Sector Alignment)

Introduction
Disk partition alignment is a best practice and must be applied during disk setup. Partition alignment provides a significant increase in system performance. Microsoft engineers have shown again-&-again that disk alignment can improves performance by not merely the nominal 10 – 15% in RAID systems claimed in official documentation, but commonly 20%, 30%, or more.

How to Save Hundreds of Thousands of Dollars
If your SAN cost $1,000,000, & partition alignment provides 10%, 20%, 30%, or more throughput & IOPs at better latency, then alignment arguably provides a direct savings of $100,000, $200,000, $300,000 or more.

Likewise, failure to do so is like throwing that money away.

Look at it another way. Say you have 24 disks on three shelves dedicated to SQL Server. Wouldn't it be great to have the equivalent of an additional shelf of disks? Say you have 200 disks. How different would your life be if you were able to get the performance of the equivalent of 250 disks or more? Imagine the perf gain for hundreds or thousands of disks. As my friend Danielle Nguyen said, we can "make huge perf gains and save huge $$ on additional hardware...especially when a data center has 100s of servers".

Typical Results
Here’re the results of an experiment which have been shown to be typical.

Data was collected for disk latency, duration, and other relevant metrics. The Avg. Disk Transfers/sec counters of the PhysicalDisk and LogicalDisk performance objects were used to measure disk latency. Disk latency is a fundamental measure of disk performance.
The experiment was simple, yet convincing. The results were consistent and significant.

Analysis resulted in the following conclusions:

Disk alignment produced significant improvement compared to nonaligned disks. The measurements document enhancements in excess of 30% for disk latency and duration.

The performance of six aligned disks was comparable to or better than eight nonaligned disks.

The Fundamental Physics
When the file allocation unit (cluster) size is defined at 64KB per best practice yet partition alignment is not performed, multiple IOPs are required to satisfy single requests.

The following diagrams document the phenomena for cluster sizes of 4KB & 64KB. For the latter, each-&-ever I/O request requires two IOPs!

The Remedy
The remedy is simple but has a big gotcha. The good news is that partition alignment is simple to perform; the bad news is that partition alignment must be done at partition creation time, prior to partitions being formatted. This is great if you have a new SAN, but it might be painful to convert large amounts of existing data on misaligned partitions.

Use the command line utility DiskPart to implement alignment when creating new partitions.

Two Essential Correlations; Three Variables: Partition Offset, File Allocation Unit Size, & Stripe Unit Size
There are two correlations which when satisfied are a fundamental precondition for optimal disk I/O performance. The results of the following calculations must result in an integer value:

Partition_Offset ÷ Stripe_Unit_Size
Stripe_Unit_Size ÷ File_Allocation_Unit_Size

Of the two, the first is by far the most important for optimal performance.

Starting Partition Offset
Use this formula to obtain the starting partition offsets for existing partitions:

wmic partition get BlockSize, StartingOffset, Name, Index

File Allocation Unit Size
Run this command for each drive to see the file allocation unit size reported in bytes per cluster.

fsutil fsinfo ntfsinfo c:
fsutil fsinfo ntfsinfo d:
etc...

Stripe Unit Size
The value for stripe unit size must be obtained from your SAN man (or woman).

Dynamic Volumes
Note that the dynamic volumes complicate matters a bit. See this post for more information.

A Common Misalignment Example
The following demonstrates a common misalignment scenario: Given a starting partition offset for 32,256 bytes (31.5 KB) and stripe unit size of 65,536 bytes (64 KB), the result is 0.4921875. This is not an integer; therefore the offset & strip unit size are not correlated. This is consistent with misalignment.

Disk I/O Subsystem Configuration
Configuring optimal disk performance is often viewed as much art as science. Yet an understanding of best practices can result in significant improvements in performance. Some of the many factors which affect disk I/O performance include the number, size, & speed of disks; file allocation unit size; configuration of HBAs & fabric switches; network bandwidth; cache on disk, controllers, & SAN; whether disks are dedicated, shared, or virtualized; RAID level; bus speed; number of paths from disk I/O subsystem to server; driver versions for all components, stripe size, stripe unit size, & workload. Disk partition alignment is the foundation for optimal disk performance. Failure to do so is incompatible with performance & scalability.

References

¡New!
Disk Partition Alignment (Sector Alignment) Best Practices: Characterization, Analysis, and Configuration for Optimal Performance of Windows Disks—Technical Note Series
http://sqlcat.com/whitepapers/archive/2009/05/11/disk-partition-alignment-best-practices-for-sql-server.aspx

Disk Partition Alignment (Sector Alignment) for SQL Server: Part 1: Slide Deck
http://blogs.msdn.com/jimmymay/archive/2008/10/14/disk-partition-alignment-for-sql-server-slide-deck.aspx

Disk Partition Alignment (Sector Alignment) for SQL Server: Part 4: Essentials (Cheat Sheet)
http://blogs.msdn.com/jimmymay/archive/2008/12/04/disk-partition-alignment-sector-alignment-for-sql-server-part-4-essentials-cheat-sheet.aspx

An updated version of the Disk Partition tool for Windows Server 2003 is available
http://support.microsoft.com/default.aspx?scid=kb;en-us;923076&sd=rss&spid=3198

Pre-deployment I/O Best Practices (Volume Alignment and NTFS Allocation Unit Size)
http://www.microsoft.com/technet/prodtechnol/sql/bestpractice/pdpliobp.mspx

Disk Subsystem Performance Analysis for Windows
http://www.microsoft.com/whdc/archive/subsys_perf.mspx

Administrivia

<Note: Edited for clarity on 20090510>
<Note: Link to whitepaper added & "Disk I/O Subsystem Configuration" section added>

Jimmy May, MCDBA, MCSE, MCITP: DBA + DB Dev
Senior Performance Consultant: SQL Server
A.C.E.: Assessment, Consulting, & Engineering Services
http://blogs.msdn.com/jimmymay

This post was written with the PracticeThis.com plugin for Windows Live Writer

If it is fast and ugly, they will use it and curse you; if it is slow, they will not use it.
—Computer science professor, billionaire, & entrepreneur David Cheriton

A Language for Software Architecture by JD Meier in The Architecture Journal

Jimmy May — Tue, 28 Apr 2009 16:41:58 GMT

J.D. Meier is a Principal Program Manager for Microsoft’s Patterns & Practices team. He’s a great writer & a clear thinker. J.D.’s Sources of Insight is devoted to getting results & balancing life.

His latest-&-greatest publication is available now:

A Language for Software Architecture
J.D. Meier, The Architecture Journal, #19, TechEd 2009 Special Edition
Article URL: http://short.ie/msdnlang4arch
Blog URL: http://short.ie/jdlang4arch

His starts out like this:

It's a simple language for helping you get in the ballpark when you're traversing the very large space of software architecture. By framing and naming the space, we can more effectively share our principles, patterns, and practices for application architecture. This also helps consolidate all the great information spread over time and space and threads and heads.

I know how important it is to establish a common framework for communication. In my deck on disk partition alignment & forthcoming white paper, I have a section devoted exactly to doing so. For example, engineers must discriminate between ambiguous & often misused terms such as block, sector, partition, & volume.

I recently returned from three weeks in the Middle East. During many of our meetings, a translator was required. I’m seldom accused of talking succinctly, yet for every word I spoke, it seemed the translator needed ten. I was reminded of the translation preceding the fight scene in Dwayne “The Rock” Johnson’s The Rundown. (Apparently nothing was lost in translation—the gig turned out swell.)

J.D.’s work transcends my anecdotes by a wide margin. The best practices he proselytizes are practical, scalable, & platform agnostic.

…This also helps consolidate all the great information spread over time and space and threads and heads. More importantly, if we simplify how we talk about architecture, we can move up the stack as well as pave paths for others and help mentor others in our field. Instead of asking basic questions like what is architecture, we can ask things like how do we define archetypes for the cloud or how do improve product line engineering for common systems and application types? In our case, we're using the language to help rationalize our portfolio of assets in our patterns & practices product line…

J.D. lucidly outlines the following:

Why the Map
Usage Scenarios
Key Concepts
The Map

His post & article are not merely worth reading & re-reading, but learning. Don’t take my word for it. Decide for yourself & share it with your team. Doing so will provide a scalable framework for effective communication and success.

If it is fast and ugly, they will use it and curse you; if it is slow, they will not use it.
—Computer science professor, billionaire, & entrepreneur David Cheriton

Administrivia

Jimmy May, MCDBA, MCSE, MCITP: DBA + DB Dev
Senior Performance Consultant: SQL Server
A.C.E.: Assessment, Consulting, & Engineering Services
http://blogs.msdn.com/jimmymay

This post was written with the PracticeThis.com plugin for Windows Live Writer

Wait Stats by Joe Sack

Jimmy May — Mon, 27 Apr 2009 18:15:00 GMT

Wait stats rock. As I shared recently with friends, when it comes to SQL Server performance, wait stats are the #1 arrow in my quiver, the first tool out of my toolbox, the Vera of guns (see the Firefly episode Our Mrs. Reynolds to divine that reference).

In a previous post I stated the importance of wait stats & cited several references.

Joe Sack has recently published the best deck on the topic I’ve seen to date:

Presentation Deck for "Performance Tuning with Wait Statistics"
http://blogs.msdn.com/joesack/archive/2009/04/22/presentation-deck-for-performance-tuning-with-wait-statistics.aspx

Joe’s presentation includes:

What are wait stats?
What to collect? How to collect?
Where to look for wait stats info, what to collect, what to ignore, & what to pay attention to…
Top 10 Patterns from the field
- Parallelism: CXPACKET
- CPU pressure: SOS_SCHEDULER_YIELD
- Network I/O: ASYNC_NETWORK_IO
- Long term blocking: LCK_X, LCK_M_U, & LCK_M_X
- Buffer I/O latch: PAGEIOLATCH_X
- Buffer latch: PAGELATCH_X
- Non-buffer latch: LATCH_X
- Memory grants: RESOURCE_SEMAPHORE
- Tran log disk subsystem: WRITELOG & LOGBUFFER
- General I/O issues: ASYNC_IO_COMPLETION & IO_COMPLETION

But wait, there’s more!

References
Usage Scenarios
Benefits
Wait stats in a nutshell
Execution model
Query life cycle
Why Waits-&-Queues
sys.dm_os_wait_stats & sys.dm_os_waiting_tasks
Reporting made simple, Performance Dashboard, & the Management Data Warehouse & Data Collector
Extended events introduction
Interpretation: Resource, Queue, & External
Runnable vs. Running
Preemptive waits

I’ve transcribed & attached the code. I invite you to download both Joe’s deck & my transcript.

There is no fate but what we make.
—John Connor

Administrivia

Jimmy May, MCDBA, MCSE, MCITP: DBA + DB Dev
Senior Performance Consultant: SQL Server
A.C.E.: Assessment, Consulting, & Engineering Services
http://blogs.msdn.com/jimmymay

This post was written with the PracticeThis.com plugin for Windows Live Writer

Disk I/O Case Study: Classic Case of SAN Over-Subscription

Jimmy May — Sun, 15 Mar 2009 16:21:00 GMT

SAN Over-Subscription
There are a variety of challenges related to I/O performance. One common problem is "SAN Over-Subscription", which is characterized by the the following symptoms:

High disk latency and
Low IOPs & Throughput

As MS PFE & fellow perf engineer Shane Creamer describes it, think of sucking your favorite adult beverage through a pinched straw.

Preamble
Recently I had the pleasure of working with my A.C.E. colleague Jon Picariello & MS IT SAN Admin Russell Folsome on an internal Microsoft application. This post describes that gig's big win.

I/O Data: The Table
The customer was experiencing significant I/O challenges. Jon compiled the following table from PerfMon logs he captured.

I/O Data: The Chart
I used his information to generate the following chart.

Let's examine the chart, which displays the following five dimensions:

Lower Right Axis
- Drive letter: H:, I:, & X:
- Disk latency counter: Disk sec/Read & Disk sec/Write
Lower Left Axis
- FA count: 2, 4, & 6
- Disk Latency metric: Avg & Max
Left Vertical Axis
- Disk Latency in seconds (yes, whole seconds not milliseconds!)

The columns are divided into four quadrants

Max latency values for Disk sec/Read
Max latency values for Disk sec/Write
Avg latency values for Disk sec/Read
Avg latency values for Disk sec/Write

It doesn’t take pattern recognition skills commensurate with membership in Mensa to see the trends. Max & Avg Read & Write latency for all three drives improved dramatically with the each addition of Fiber Adapter pairs.

I/O Data: The Discussion

Thresholds
For systems from which high performance is expected, average disk latency should be no more than 5 – 10ms for OLTP db data files. Occasional spikes may be acceptable, but extended plateaus are not. Values for log files should not exceed 0 – 2ms

I'll spare you the screen scrapes, yet were I less merciful you’d see that the IOPs consumed by H:, I:, & X: ought to require only 3 disks, 9 disks, & 6 disks respectively—a total of 18 disks—to handle the maximum loads with ease. Instead, as you see we have latency values out the wazooticles—orders of magnitude higher than acceptable.

The fact that this isn’t happening means either:

We don’t have enough disks
The disks are too busy servicing other requests
A bottleneck in the SAN

The Usual Suspects? No!
High latency is a symptom. Jon nailed it. But we needed to know what was causing the latency. Jon confirmed that disk partition alignment had already been done. (Did you know that disk partition alignment may be the best kept secret in SQL Server performance?) Russ the SAN man confirmed that HBA queue depth had been elevated to 64. Elevation of of this setting from the typical default of 32 to 64 (& sometimes more) is a best practice. (See SQL CAT's Mike Ruthruff's Predeployment I/O Best Practices for more details.) Per best practice, file allocation unit size was 64KB.

Too Few Disks? No!
We were told the SAN presented a total of 400 disks to us. Not taking RAID into account, even 10K disks should support at least 40,000 IOPs & 40,000 MB/s. (You might come up with somewhat different numbers; the point is the actual throughput is orders of magnitude lower than theoretical.)

Too Busy Disks? No!
The disks aren’t too busy. Examine the idle times in the table or the chart—virtually all above 50%.

Note on % Idle Time
The % Idle Time counter is a bit controversial & misunderstood. Like disk queue length, it can be & often is misused. I will be publishing a series of posts in which I talk about these disk counters. Be prudent in using & interpreting values from this counter!

The Perpetrator Revealed: SAN Over-Subscription
As described above, 18 disks could theoretically provide the observed performance, so 400 disks are plenty. The disks are largely idle. That leaves only one other option: a bottleneck in the SAN.

Why is the I/O that is being provided delivered at such high latency? Why can’t 400 disks—disks which PerfMon reports are idle half the time—produce the throughput even of only 18? By the definition I shared in the opening paragraph, this describes a classic case of SAN Over-Subscription.

Viola!
It turns out that the number of Fiber Adapters was on the shy side. Thanks to the prompt & enthusiastic efforts of MSIT SAN Man Russell Folsome, the count was elevated from two to four & eventually to six.

The improvements are remarkable. Note that the vertical axis is logarithmic in order to display all data values which transcend five orders of magnitude. Improvements exceeding two orders of magnitude were achieved in virtually every category.

Their world is now a different, much better place to be in. As my friend the cellist says, “Viola!”

Query Tuning
In addition, we counseled the application team that it was their responsibility to remediate the expensive queries we identified. Doing so would provide compliance with best practices & provide head room.

Fundamental Troubleshooting
You might think, "Well, duh, Jimmy May, you threw more hardware at it, so of course it's going be faster". Well, that's not necessarily so. For example, if we'd've thrown more disks at the problem, or more memory, or more CPU, the numbers wouldn't've changed much. By analyzing a simple PerfMon log, we were able to point to the SAN. Fundamental performance troubleshooting definitively identified the area of the bottleneck. By collaborating with a cooperative SAN admin, we were able to remediate the problem.

Related Posts

PerfMon Objects, Counters, Thresholds, & Utilities for SQL Server
Disk I/O: Microsoft SQL Server on SAN Best Practices from SQL CAT's Mike Ruthruff (& Prem Mehra)
Stay tuned for my series on Disk I/O counters.

Acknowledgements
In addition to A.C.E. colleague Jon Picariello & MS IT SAN Man Russell Folsome, thanks also to the internal team with whom we worked, especially Matt Eldridge without whose full cooperation we may not have been able to pull this off. As always, I am grateful for the consistently excellent guidance from my fellow MS perf phreaks Clint Huffman, Ken Brumfield, Shane Creamer, & Robert Smith. Thanks also

Administrivia

Jimmy May, MCDBA, MCSE, MCITP: DBA + DB Dev
Senior Performance Consultant: SQL Server
A.C.E.: Assessment, Consulting, & Engineering Services
http://blogs.msdn.com/jimmymay

If it is fast and ugly, they will use it and curse you; if it is slow, they will not use it."
—Computer science professor, billionaire, & entrepreneur David Cheriton
This post was written with the PracticeThis.com plugin for Windows Live Writer

Architecture Journal: Green IT in Practice: SQL Server Consolidation in Microsoft IT

Jimmy May — Tue, 10 Mar 2009 18:12:00 GMT

I'm a lucky guy. Besides the fact that I'm healthy, alive, & living on the planet earth "in interesting times" as the fortune cookie says, I had the opportunity to contribute to an article which appeared in the Green IT issue of The Architecture Journal:

Green IT in Practice: SQL Server Consolidation in Microsoft IT
http://msdn.microsoft.com/en-us/architecture/dd393309.aspx

Mark Pohto, α-Geek #1
The primary author is Mark Pohto, an engineer for MS IT responsible for architecting Microsoft's SQL Server consolidation solution. He's taken the lessons learned & documented them to make the planet a better place on which to live—especially if it's your job to create such a solution. Mark's work provides an excellent head start. Remember, this ain't no theoretical mumbo-jumbo, this is how Microsoft actually gits-'er-done.

The article demonstrates the convergence of the best interests of business, people, & the planet—which happens a lot more than some folks think—but that discussion is for a different forum...

Ward Pond, α-Geek #2
There's my name, there at the end of the article, along with Ward Pond. Ward is an Architect for Microsoft's IT Operations Excellence group. Both Mark & Ward were instrumental in crafting the SQL Server Ranger program, part of the path to the Microsoft Certified Architect: Database certification. To be associated with these folks in any capacity is quite an honor. (See "lucky guy" comment above as well as this & this.)

Ward, always a step or three ahead of me, already cited the article on his blog.

<ADD>If you're not already a subscriber to the fine resource which is Ward's blog, I invite you to do so. His timely & entertaining prose—technical & personal—is always a welcome respite from the usual humdrum.</ADD>

Critical Noteriety
The article has received critical noteriety, including the following:

Server consolidation and Green IT Computing
Kimberly L. Tripp, SQL Skills

SQL Server Consolidation Initiative Under Way at Microsoft
SQL Server Magazine

Where's the Beef?
Enough fluff! What does the article actually say? OK, OK, alright, already, hold your equine beasties! Here's the ToC:

Data Center Energy Consumption Trends
Opportunities
Initial Situation
Desired Situation
Solution Mindset
Solution Implementation
Results
General Benefits
Conclusion
Resources

Getting it Right
Yesterday Mark shared the latest-&-greatest high-level diagram of our environment. Mark's dedication to getting this "right" has been impressive, an exquisite example of the iterative process associated with the following: Perception, Reality, & Incrementally Tuning World-Class Applications.

Normalizing Disparate Platforms: Compute Units (CU)
Among others, one of the most interesting new concepts was normalizing disparate platforms through the use of Compute Units, CU.

A data-driven discussion. Microsoft IT developed the RightSizing initiative to ensure effective utilization of servers in the data center and in managed labs. Because significant underutilization occurs, one of the initiative’s first tasks was for Microsoft IT to identify underutilized servers that might be good candidates for virtualization...

To accurately compare the performance of server platforms of varying architectures, Microsoft IT has developed a Compute Unit (CU) formula for each server platform that utilizes an industry standard, architecture-agnostic, benchmark suite from the Standard Performance Evaluation Corporation (SPEC). The SPEC benchmarks are developed in such a way to allow a repeatable test with strict result submission requirements. The Microsoft IT CU formula uses a baseline (not peak) benchmark that measures the rate of integer calculation work a server platform can perform in a given amount of time.

According to the Technical Case Study, Identifying Server Candidates for Virtualization:

The benchmarks that Microsoft originally used, the SPEC CPU2000 - CINT2006 Rates - Baseline Rates, have been retired. Microsoft IT is now using the SPEC CPU2006 - CINT2006 Rates - Base Results. For more information about the SPEC benchmarks and ratings for individual servers, visit the Standard Performance Evaluation Corporation Web site at http://www.spec.org.

SQL Server 2008 & Windows Server 2008 Hyper-V Best Practices
For maximum benefits, couple the concepts from Mark's consolidation article with this one from the SQL Server Customer Advisory Team (SQL CAT):

Running SQL Server 2008 in a Hyper-V Environment - Best Practices and Performance Recommendations
Authors: Lindsey Allen, Mike Ruthruff, Prem Mehra
Reviewers: Cindy Gross, Burzin Patel, Denny Lee, Michael Thomassy, Sanjay Mishra, Savitha Padmanabhan, Tony Voellm, Bob Ward

Administrivia

Jimmy May, MCDBA, MCSE, MCITP: DBA + DB Dev
Senior Performance Consultant: SQL Server
A.C.E.: Assessment, Consulting, & Engineering Services
http://blogs.msdn.com/jimmymay
"If it is fast and ugly, they will use it and curse you; if it is slow, they will not use it." —Computer science professor, billionaire, & entrepreneur David Cheriton
This post was written with the PracticeThis.com plugin for Windows Live Writer

Disk I/O: Microsoft SQL Server on SAN Best Practices from SQL CAT's Mike Ruthruff (& Prem Mehra)

Jimmy May — Sun, 01 Mar 2009 19:44:00 GMT

While at the PASS Community Summit in November 2008, I had the pleasure of attending a handful of excellent presentations. One of the best was delivered by Mike Ruthruff (& not just because he shilled for my presentation on disk partition alignment later that day—though I suspect he contributed to my session being SRO).

Mike is a member of the SQL Server Customer Advisory Team (SQL CAT). He authored the deck with contributions from SQL CAT patriarch Prem Mehra.

Most of you probably know Mike because he is the primary author of the landmark white paper we all know-&-love & have read over-&-over again because we know how unbelievably valuable it is:

SQL Server Predeployment I/O Best Practices
http://www.microsoft.com/technet/prodtechnol/sql/bestpractice/pdpliobp.mspx

Mike gave me a copy of his latest-&-greatest deck for publication here:

Microsoft SQL Server on SAN Best Practices
http://tinyurl.com/SQLonSAN

The deck includes the following:

Characteristics of SQL Server I/O operations
Best practices

SQL Server Design Practices
Storage Configuration
Common Pitfalls

Monitoring performance of SQL Server on SAN
Emerging Storage Technologies
Additional Material In Appendix Section (not covered during session)

How to validate a configuration using I/O load generation tools
General SQL Server I/O characteristics
How to diagnose I/O bottlenecks
Sample Configurations

I think you'll enjoy this presentation—one of the best, perhaps the best of its kind ever assembled. ¡Yo! Only first-rate decks on this blog. Besides which, SQL CAT does nothing but the best. Get ready to be wowed by 50 slides of geekly goodness.

Administrivia

Jimmy May, MCDBA, MCSE, MCITP: DBA + DB Dev
Senior Performance Consultant: SQL Server
A.C.E.: Assessment Consulting & Engineering Services
http://blogs.msdn.com/jimmymay
Microsoft: Change the world or go home.
This post was written with the PracticeThis.com plugin for Windows Live Writer

SSMS Query Results: “Include column headers when copying or saving the results"

Jimmy May — Sun, 22 Feb 2009 18:40:00 GMT

In my work with external customers, I collaborate with some excellent DBAs & Architects. But I've noticed many of them are not aware that you can capture column headers from query results to grid in SSMS.

When a very bright engineer I work with—not a SQL SME yet one of the smartest guys I know—told me he didn't know about it either, I decided to document the simple protocol. I'm not the first person to post this topic, but I wouldn't need to do so if everyone already knew about it.

There are two ways to get the column headers from SSMS into, say, Excel.

The Hard Way

You can simply type out the column names by hand.

The Right Way

Make it the SSMS default:

Open SSMS
Click Tools > Options
In the Options dialog, expand Query Results > SQL Server > Results to Grid, then check “Include column headers when copying or saving the results”
Click OK
Close & restart SSMS

As my friend the cellist says, "Viola!"

The steps are the same for SQL Server 2005 & 2008.

Additional Resources

Explore these topics for additional information on SSMS options & increase your SSMS productivity.

Administrivia

Jimmy May, MCDBA, MCSE, MCITP: DBA + DB Dev | Senior Performance Consultant: SQL Server
A.C.E.: Assessment Consulting & Engineering Services
http://blogs.msdn.com/jimmymay
Oh, the weather outside is frightful, but SQL Server is delightful! —Brian Moran

This post was written with the PracticeThis.com plugin for Windows Live Writer

sp_reset_connection Does NOT Reset TRANSACTION ISOLATION LEVEL: Unexpected Behavior By Design

Jimmy May — Mon, 02 Feb 2009 20:44:00 GMT

I've long been an advocate of the best practice of explicitly setting the transaction isolation level in my scripts.

At my old day job as an enterprise architect I mandated that along with SET NOCOUNT ON, the preamble for all SProcs explicitly must declare the appropriate transaction isolation level.

In addition, all ad hoc queries by DBAs had to be prefaced with a read uncommitted declaration. To save time & promote compliance, I distributed a SQL template for quick access (set_tx_iso_level.tql) which contains two lines:

SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED
-- DBCC UserOptions

<ADD> We'll save the dirty reads & data integrity debate for another day, eh? </ADD>

Many customer applications exercise connection pooling. Doing so enhances performance & preserves system resources. For more information, see this MSDN article:

Using Connection Pooling with SQL Server

The system SProc sp_reset_connection facilitates the re-use of connections from the connection pool. The SProc is so ubiquitous that I typically filter it from SQL Trace output.

My friend, buddy, pal PFE Gennady "Dr. G" Kostinsky brought to my attention that sp_reset_connection does NOT reset the transaction isolation level to the server default from the previous connection's setting.

To my surprise, this unexpected behavior is by design.

O the humanity! See for yourself on Connect:

http://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=243527

This has profound implications. Imagine the havoc wrought to applications by everything from intended declarations to developer abuse to vendor apps—the list goes on-&-on...

In light of sp_reset_connection's behavior, the explicit declaration of the isolation level is not merely a best practice but is a mandatory practice. Engineering discipline demands it.

Perception, Reality, & Incrementally Tuning World-Class Applications

Jimmy May — Wed, 07 Jan 2009 15:32:00 GMT

Performance & the 20% Rule

I was reading the post from Ed Glas, Make Your Performance Work Count: The 20% Rule.

He shared the words of computer science professor & billionaire entrepreneur David Cheriton:

If it is fast and ugly, they will use it and curse you;
if it is slow, they will not use it.

Indeed.

Ed provided this quote in the context of the book Designing and Engineering Time: The Psychology of Time Perception in Software by Steve Seow. Steve is an expert on performance engineering at Microsoft and has a deep understanding of the human element and psychology behind software performance. An intriguing take-away is that users do not notice changes in performance below a certain threshold. Research has shown that for durations under 30 seconds, users do not detect differences up to 18%. Steve's rule of thumb is that users really won't notice performance changes of +/- 20%.

Ed Glas is a big-time test manager for Microsoft. He relates that when setting goals for performance improvement, or considering performance against a competitive product, the threshold for success should be to gain at least 20% over the baseline measurement. Likewise when considering whether to fix a performance regression, users likely won't detect the degradation less than 20%. He suggests we’re better off spending effort elsewhere if the 20% threshold seems out of reach for a given area of testing.

Perception is Reality

I was struck by the (not uncommon) idea that Perception is Reality. (This has been reinforced by my holiday reading of Stumbling on Happiness by Daniel Gilbert.) So, to get ooh’s & ah’s from our users or customers—especially before we’ve gained their trust—we must either provide the 20% enhancements Mssrs. Glas & Seow discuss, or set the expectations low (which is sometimes appropriate but hard to do), &/or when incremental changes smaller than the 20% perception threshold we must provide appropriate metrics to document what we did.

My Microsoft Assessment, Consulting, & Engineering (A.C.E.) team & I often provide performance enhancements substantially larger than 20% in terms of all manner of metrics including duration.

World-Class Applications Are the Result of Incremental Improvements

Yet let’s not neglect that like many aspects of development, performance & scalability enhancements are incremental. Dittos for those changes which are significant in magnitude, yet don’t result in a significant performance enhancement visible to the user. For example, I recently documented how a simple configuration change resulted in an immediate, persistent 30% CPU enhancement and reduction in SQL Server wait stats of 78%. The customer was thrilled—yet the user experience was scarcely affected.

In addition, best practices often demand many things the transition to which may not in-&-of-themselves show obvious immediate benefits. It is my position that incremental improvements in performance & scalability made possible by compliance with best practices provide a foundation fundamentally critical for success.

An analogy would be the forces binding the DNA helix. Hydrogen bonding is one of the weakest molecular bonds in biology, any individual one of which can be cleaved quite easily. However, billions of discrete H-bonds result in a virtually indivisible unit the cleavage of which requires specific enzymatic action.

Likewise the result of multiple tuning iterations will result eventually in only incremental changes beyond the ability of humanoids to appreciate, yet the aggregate effect can be vital to an applications & organization’s success.

Why World-Class Applications Seek Incremental Improvement

A deck from CS193H High Performance Web Sites by Stanford Professor Steve Souders contains this information:

Company	Delay	Business Loss
Google	500ms	20% traffic
Amazon	100ms	1% sales

Of course, we go initially for the most bang for our buck, & pick off the low hanging fruit first (& other clichéd metaphors).

This data demonstrates that incremental improvements translate to big bucks in mature, world-class applications.

As an architect out in the real-world, I constantly strived to introduce best practices into our applications. After several years on the job, incremental improvements were all we had left, yet it still made business sense to pursue them.

A proper, disciplined amalgamation of best practices can result in scalable, robust, performant, world-class applications—a classic example of the whole being greater than the sum of its parts (there's another one of those darned clichés).

Perception Problems are People Problems

So, it’s great to produce 20% & greater performance enhancements, yet such improvements can’t be sustained round after round after round of tuning.

It is important—isn’t it always?—to perform appropriate expectations setting when only incremental changes are possible. As Alik Levin shares with us, "No matter how it looks at first, it's always a people problem".

What’s your perception of implementing best practices?
What’s your perception of performance?
Tell me.

Administrivia

Jimmy May, MCDBA, MCSE, MCITP: DBA + DB Dev Senior Performance Consultant: SQL Server
A.C.E.: Assessment Consulting & Engineering Services
http://blogs.msdn.com/jimmymay
Performance is paramount: Asking users to wait is like asking them to leave.

This post was written with the PracticeThis.com plugin for Windows Live Writer