Version 2 of the Quick Reference Guide has now been published. This guide is chock-full of information gathered through real experience with the product. This new version contains 97 new or updated articles.
Special thanks to Geoff Gray for once again spearheading this effort. Geoff is one of a handful of elite VS load test practitioners. As he encounter unexpected behavior or bugs in the product, or gets undocumented information on the internals or edge cases, he captures the info for the quick reference guide.
Here is a screen shot of some of the really nice artwork in the guide:
Here is a list of topics in the guide (the numbers after each article is the page number):
NOTE FROM THE AUTHOR HOW IT WORKS How Web Tests Handle HTTP Headers 9 General Info (including order of execution) of load and web test plugins and rules 9 Client Code does not execute because Web Tests Work at the HTTP Layer 12 File Downloads, Download Size and Storage of files during Web Tests 12 When is the “Run unit tests in application domain” needed? 12 How the “Test Iterations” Setting impacts the total number of tests executed 12 Test timeout setting for load test configuration does not affect web tests 13 How user pacing and “Think Time Between Test Iterations” work 13 Load test warmup and cool down behaviors 13 What is the difference between Unique, Sequential and Random Data Sources 14 Comparing new users to return users 14 Goal based user behavior after the test finishes the warmup period 17 Threading models in Unit tests under load 18 Simulation of Browser Caching during load tests 19 The difference between Load Test Errors and Error Details 20 How parameterization of HIDDEN Fields works in a webtest 21 Testing execution order in Unit Tests 23 How machines in the test rig communicate 25 Changing the Default Port for Agent-Controller Communication 26 How to Add Agents To A Test Rig 26 ITEMS NEW TO VS 2010 “Find” feature now available in Webtest playback UI 27 “Go To Web Test” feature now available in Webtest playback UI 28 Recorder Log Available 29 Add extraction rule directly from the playback UI 30 New “Reporting Name” property for web requests 31 LoadTestResultsTables now differentiate between GET and POST requests 32 Virtual user visualization now available 33 New Excel reporting features built into load test results 39 New Load Test and Load Test Rig Licensing and configurations 40 New test mix: “Sequential Test Mix” 44 Query String and FORM POST URLs get parameterized 46 New options on Load Test Scenarios 47 Loops and Conditionals 48 CONFIGURATIONS AND SETTINGS How to Change the Location Where Agents Store Run Files 50 How to set a proxy server for web tests 50 How to configure Web Tests so Fiddler can capture playback info 50 Controlling the amount of memory that the SQL Server Results machine consumes 51 How to configure the timeouts for deployment of load tests to agents 51 How to set the number of Load Test Errors and Error Details saved 52 Multi-proc boxes used as agents should have .NET garbage collection set to server mode 53 Location of list of all agents available to a controller 53 NETWORKS, IP SWITCHING, TEST STARTUPS IP Address Switching anatomy (how it works) 54 Gotcha: IP Address Switching is ONLY for WEB TESTS 54 Gotcha: IP Addresses used for switching are not permanent 54 How to Setup IP Switching 55 Troubleshooting invalid view state and failed event validation 58 Startup: Slowness Restarting a Test Rig with Agents Marked as “Offline” 58 Startup: Multiple Network Cards can cause tests in a rig to not start 59 Startup: Slow startup can be caused by _NT_SYMBOL_PATH environment variable 59 Startup: tests on a Rig with Agents on a Slow Link 60 “Not Bound” Exception when using IP Switching is not really an error 60 How to configure the timeout for deployment of load tests to agents 61 PERFORMANCE COUNTERS AND DATA Customizing the Available Microsoft System Monitor counter sets 62 Performance Counter Considerations on Rigs with slow links 64 Increase the performance counter sampling interval for longer tests 65 Changing the default counters shown in the graphs during testing 65 Possible method for fixing “missing perfmon counters” issues 65 How and where Performance data gets collected 66 DATA AND RESULTS Custom Data Binding in UNIT Tests 67 Verifying saved results when a test hangs in the “In Progress” state after the test has finished 67 The metrics during and after a test differ from the results seen. 68 How new users and return users affect caching numbers 69 data sources for data driven tests get read only once 70 Consider including Timing Details to collect percentile data 71 Consider enabling SQL Tracing through the Load Test instead of separately 72 How to collect SQL counters from a non-default SQL instance 72 How 90% and 95% response times are calculated 72 Transaction Avg. Response Time vs. Request Avg. Response Time 73 Considerations for the location of the Load Test Results Store 73 Set the recovery model for the database to simple 73 How to clean up results data from runs that did not complete 74 InstanceName field in results database are appended with (002), (003), etc. 74 Layout for VSTS Load Test Results Store 74 How to view Test Results from the GUI 75 SQL Server Reporting Services Reports available for download 75 How to move results data to another system 75 Load Test Results without SQL NOT stored 76 Web Test TRX file and the NAN (Not a Number) Page Time entry 77 Proper understanding of TRX files and Test Results directory 78 Understanding the Response Size reported in web test runs 79 ERRORS AND KNOWN ISSUES 80 CSV files created in VSTS or saved as Unicode will not work as data sources 80 Incorrect SQL field type can cause errors in web tests 80 Leading zeroes dropped from datasource values bound to a CSV file 80 Recorded Think Times and paused web test recordings 80 After opening a webtest with the VS XML Editor, it will not open in declarative mode. 81 Calls to HTTPS://Urs.Microsoft.Com show up in your script 81 Possible DESKTOP HEAP errors when driving command line unit tests 81 Goal based load tests in VSTS 2008 do not work after applying SP1 82 Using Named Transactions in a Goal-Based Load Profile can cause errors 82 Debugging Errors in Load Tests 83 Debugging OutOfMemory Exceptions in Load Tests 83 Memory leak on load test when using HTTPS 83 “Not Trusted” error when starting a load test 84 Detail Logging may cause “Out of disk space” error 85 Error details and stack traces no longer available in VSTS 2010 85 VSTS does not appear to be using more than one processor 85 Changes made to Web Test Plugins may not show up properly 85 Socket errors or “Service Unavailable” errors when running a load test 86 Error “Failed to load results from the load test results store” 87 Hidden Field extraction rules do not handle some fields 87 Test results iteration count may be higher than the max test iterations set 87 In flight test iterations may not get reported 88 Completion of Unit Test causes spawned CMD processes to terminate 88 Bug with LoadProfile.Copy() method when used in custom goal based load tests 89 Errors in dependent requests in a Load Test do not show up in the details test log 90 WCF service load test gets time-outs after 10 requests 92 Loadtestitemresults.dat size runs into GBs 92 TROUBLESHOOTING How to enable logging for test recording 93 Diagnosing and fixing Web Test recorder bar issues 93 User Account requirements and how to troubleshoot authentication 94 How to enable Verbose Logging on an agent for troubleshooting 95 Error that Browser Extensions are disabled when recording a web test 95 Troubleshooting invalid view state and failed event validation 96 Troubleshooting the VSTS Load Testing IP Switching Feature 97 Troubleshooting Guide for Visual Studio Test Controller and Agent 99 HOW TO, GOTCHAS AND BEST PRACTICES How to call one coded web test from another 111 How to use methods other than GET and POST in a web test 111 How to filter out certain dependent requests 111 How to handle ASP.NET Cookie-less Sessions 112 How to use Client-side certificates in web tests 112 How to remove the “If-Modified-Since” header from dependent requests 113 How to handle custom data binding in web tests 113 How to add a datasource value to a context parameter 113 How to test Web Services with Unit Tests 114 How to add random users to web tests 114 How to add think time to a Unit Test 114 How to add details of a validation rule to your web test 115 How to mask a 404 error on a dependent request 116 How to parameterize Web Service calls within Web Tests 117 How to pass Load Test Context Parameters to Unit Tests 117 How to create Global Variables in a Unit Test 117 How to use Unit Tests to Drive Load with Command Line Apps 118 How to add Console Output to the results store when running Unit tests under load 118 How to add parameters to Load Tests 119 How to Change the Standard Deviation for a NormalDistribution ThinkTime 119 How to programmatically access the number of users in Load Tests 120 How to create a webtest plugin that will only execute on a predefined interval 120 How to support Context Parameters in a plug-in property 121 How to stop a web test in the middle of execution 122 How To: Modify the ServicePointManager to force SSLv3 instead of TLS (Default) 122 How To: Stop a Test in the PreRequest event 123 How to make a validation rule force a redirection to a new page 123 How to add a Web Service reference in a test project 127 How to remotely count connections to a process 129 How to hook into LoadTest database upon completion of a load test 129 How to deploy DLLs with MSTEST.EXE 130 How to authenticate with proxy before the test iteration begins 131 How to enumerate WebTextContext and Unit TestContext objects 132 How to manually move the data cursor 132 How to programmatically create a declarative web test 133 How to modify the string body programmatically in a declarative web test 134 Gotcha: Check Your Validation Level in the Load Test Run Settings 134 Gotcha: Do not adjust goals too quickly in your code 134 Gotcha: Response body capture limit is set to 1.5 MB by default 134 Gotcha: Caching of dependent requests is disabled when playing back Web Tests 135 Best Practice: Blog on various considerations for web tests running under load 135 Best Practice: Coded web tests and web test plug-ins should not block threads 135 Best Practice: considerations when creating a dynamic goal based load test plugin: 136 Best Practice: Add an Analysis Comment 136 Best Practice – Using comments in declarative webtests 136 EXTENSIBILITY New Inner-text and Select-tag rules published on Codeplex 138 ITEMS NOT SPECIFIC TO THE VSTS TESTING PLATFORM Using the VSTS Application Profiler 146 VSTS 2008 Application Profiler New Features 146 Using System.NET Tracing to debug Network issues 146 Logparser tips and tricks 147 Logparser WEB Queries 147 LogParser Non-Web Queries 148 OLDER ARTICLES Content-Length header not available in Web Request Object 149 SharePoint file upload test may post the file twice 149 Some Hidden Fields are not parameterized within AJAX calls 149 (FIX) Unit Test threading models and changing them 149 Bug in VSTS 2008 SP1 causes think time for redirected requests to be ignored in a load test 150 New Load Test Plugin Enhancements in VSTS 2008 SP1 150 Four New Methods added to the WebTestPlugin Class for 2008 SP1 150 INDEX