Patterns & Practices Test

Workload Characterization for Performance Testing

2005-03-19T09:58:00Z

Test Load definition is one of the steps for a performance test plan specification. It is a view account status of the most frequent use resource action against a computing system. Test Load is generated by creating virtual users, in a simulation case where the goal is to reflect how users utilize a system. When developers or testers are load testing a system, they usually have no numbers that allow creating a load profile. Let us consider a box that contains a processing unit, it can be as specific as a disk subsystem or complex as an entire intranet. If we consider customers C arriving at a rate R to the box and spending T time utilizing the box, we can say that C=R*T. This is known as Little’s Law. The proof of this simple but important performance law can be found if you search the Internet. For a given system the throughput of a system can be measured by dividing the number of users with the time spent in the box (R=C/T). Now let’s assume that users will wait a To time in between requests, which we know as a think time. This is an interval typical for users to interact with the system. So from the C=R*T we can expand and infer that the number of users in think time will be Co=R*To. But the number total of users in such a case will be CTotal = C+Co = R*T + R*To = R(T+To). So CTotal = R(T+To). R=C/(T+To) where T is time spent in the box(response time), To is the average think time, C number of users and R the throughput.

If we had a system with 200 users requesting services with 1600 request for 15 minutes and response time average of 2 seconds we can characterize that think times would be C/R-T = 200/1.77 – 2=110.2 seconds average. Now if we wanted to reproduce the same workload characterization without think times in the test environment we could use safely the equivalence of two cases R=C/T or R=CTotal/(T+To). So C/T=CTotal/T+To. In our hypothetical case above C/2=200/(2+110). 2C+110C=400. C=4 users. One could easily extrapolate any workload, provided there was a well known production profile or a performance test goal in mind, and apply the law correctly.

Carlos

http://www.gotdotnet.com/Workspaces/Workspace.aspx?id=4713168a-9073-40e2-854a-a4b9ca217de9 http://www.microsoft.com/resources/practices/default.mspx

Architecture Testing

2005-03-18T20:09:00Z

I often wonder how many IT professionals say “we could have caught these bugs up front had we been involved in the planning and design phase at project proposal”. How true can it be and what does it take to convince the business leaders, architects and other stakeholders that testing should occur at design time? A time where value propositions are discussed to increase the company’s ROI based on customer requirements and business trends.

Another way of saying this is how much money can be added to the ROI if testing begins at design time? I have seen numbers that state for every 100,000 lines of code released to production a company can spend up to $300,000 dollars in maintenance and support cost over the lifecycle of the product. Some would say this is too high while others would argue it’s too low. What are your thoughts?

· Can the cost be attributed to not having a comprehensive quality control system in place at design time?

· Can you agree that implementing an architecture testing process at design time can?

§ Reduce rework and maintenance cost

§ Provide better fault detection and test coverage at development and implementation time? (increased productivity)

§ Provide a more proven deployment and operations plan (faster time to market)

§ Provide better traceability through the product life cycle (predictability)

If you answered yes to all or some of these questions, then you either have a quality control process in place to cover architecture testing or you are in need of implementing one.

The patterns and practices test team is in the process of developing guidance that addresses architecture testing. Our years of experience have proven the earlier you identify issues in the product life cycle, the more you will save in the rears. It’s not just saving money; architecture testing is saving and expanding the business with collaborative partnerships and customer satisfaction.

One high severity bug or a missed opportunity can cost you dearly in your enterprise application space. Your business domain should be validated against each solution (application) as well as the application itself. So if you were to assemble an architecture test team, what skill sets would require? Better yet:

· How would you define architecture testing?

· Are you faced with this challenge in your business today?

· Do you test at Design Time?

§ If not, why not?

§ If so what’s the approach and benefits?

You can also visit patterns & practices: Test & QA: CodeGallery Project and click downloads at:

· http://www.gotdotnet.com/codegallery/codegallery.aspx?id=4713168a-9073-40e2-854a-a4b9ca217de9 and .net application block testing at:

· http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpag2/html/mtf.asp

Which test artifacts would you find more useful for you when working with p&p blocks?

2005-03-18T03:15:00Z

Hi,

This question is for those developers and testers who consumed p&p blocks within their apps. Our test team provided different types of test artifacts within some of the blocks. I would like to get your feedback in regard to type/format of test data that you would most likely find useful for you when consuming, customizing, and integrating these app blocks. Possible test artifacts (and feel free to recommend outside of this list):

High level test plans (documents) for an app block
Lower level test cases (documents) for an app block
nUnit test scripts for an app block
Non block specific test guides like this one http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpag2/html/mtf.asp

Lets us know what is more useful for you …. Thanks

Welcome to the Patterns & Practices Test Blog

2005-03-14T20:26:00Z

This blog is dedicated to testing patterns & practices guidance and code blocks. An example of our latest test guidance “Testing .Net Application Blocks” located at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpag2/html/mtf.asp provides information on how to test and integrate p&p code blocks into existing or proposed business applications. Our next test deliverable is targeted at Architecture Testing. This guidance will enable customers to begin testing at design time starting at the business layer of the enterprise architecture stack and ending at the operations layer where the proposed business solution is deployed into production. Look for updates on Architecture Testing in the coming weeks.

Our team is made up of 5 test engineers. Our blog goal is simple - discuss test guidance over a variety of issues, which includes deliverables we have shipped, current projects, things of interest - or suggest one for us to drill down on.

Between the 5 of us we can cover a wide range of testing topics based on a combined 50 years of experience in test. You can also visit patterns & practices: Test & QA: Workspace at:

· http://www.gotdotnet.com/Workspaces/Workspace.aspx?id=4713168a-9073-40e2-854a-a4b9ca217de9 and

· http://www.microsoft.com/resources/practices/default.mspx for all p&p deliverables

Patterns & Practices Test Team