Srikanth R : VSTS

Links to Learning more about Team system

srikanth_r — Fri, 25 May 2007 11:00:00 GMT

The below links should be very useful to customers to clarify what exactly they will need to purchase in order to enable source control/bug tracking/testing scenarios within VSTS /Team Foundation Server

Presales website

VSTS datasheet

Cheers !

SnagIt! integration with TFS

srikanth_r — Fri, 25 May 2007 10:31:00 GMT

I just came across this news from bharry's blog. This is a great news for testers to create bugs and improve their productivity !

Heres the contents of the blog...

I just got mail from TechSmith - makers of SnagIt! that they have added support for Team System. Being an occasional user of SnagIt! myself, I tried it out on our dogfood server and it worked like a charm. SnagIt! allows you to take screenshots and then automatically create a bug with the screenshot attached at the click of a button - very nice. SnagIt! also has powerful capabilities for editing/annotating the screenshots. It's a nice tool. Check out the new update at:

http://www.techsmith.com/community/blogcomments.asp?thread=356

Outsourced Projects using Visual Studio Team System

srikanth_r — Sun, 20 May 2007 12:55:00 GMT

I came across this interesting blog by Richard Murillo and thought of sharing it. In this post he explains lessons learnt and how to leverage Visual studio Team System to execute outsourced projects. Here's the content of what he has to say

"Having been included in two separate teams where our outsourced projects don't go as well as expected I cannot help but to take a step back and ask why things are consistently failing with separate teams, and separate organizations.

The commonalities in both projects were fixed-bid (i.e. fixed price, features and time) using a modified waterfall model (to account for geographic locations). I know—you say the waterfall model doesn't work. Be that as it may organizations revert to this model and are thus handicapped; unable to make adjustments to the changing business and unable to make definitive decisions on how to improve their process, but more on that later. Other commonalities included high level and low level designs, unit test cases, test cases, etc.

I have heard that groups similar to mine are having great success with outsourced projects, and I myself have also been involved with projects that were executed to great success, however I assert that for us there are areas of improvement. At a high level, I can sum it up in one word: process.

Where have my two projects failed? IMHO it is in the analysis of the requirements. The requirements had the vendors pegged implementing features on product technologies they were unfamiliar with—some churn in requirements and with new technology. Traditionally engineering teams would perform proof of concepts and experiments to validate design, and clickable prototypes to share with customers to stabilize requirements (business and technical). While some of that was done here, I still see where it could have been improved. In both instances, we started with a solid vendor relationship that quickly begins to fall apart due to lack of trust. IMHO the biggest areas we are lacking are:

Trust between teams
High SNR in communication (i.e. direct, candid communications on issues, risk and status)
Close coordination between said teams

IMHO in both projects are considered high risk when examining the vendor skill set, staffing model, and experience with projects of this complexity. In most situations you can apply the Pareto principle (AKA 80-20 rule) where 80% of staff is offshore while 20% is onsite for R&D phases, moving to a 90-10 planned iterative model for build and less onsite still for stabilization.

In both projects the offshore team encountered additional issues during implementation that require workarounds for resolution, R&D, or a complete shift in approach. On both projects we tried to mitigate risk by having frequent check-ins with the offshore team and checking progress on a tight basis (one was almost daily, the other bi-weekly). Alas when significant and unexpected delays arose the relationship fell to pieces and in response we put in-house staff on the project to sure it up and get it going in the right direction.

Lessons Learned

Complete as many iterations as you can: That is, don't disrupt forward progress on the project for the sake of getting a work product from the team, but do get deliverables as often as it makes sense. Some of our projects get some weekly, others monthly.
Use collaborative tools to increase confidence in deliverables: Your in-house staff does not have to wait until the end of the formal delivery cycle to perform review or check in with the offshore team. Utilize tools like LiveMeeting Net Meeting, Windows Live Messenger, etc. to increase collaboration
Code review, code review, code review: DO NOT UNDERESTIMATE THIS. A majority of issues we found on one project were found in code review by in-house staff familiar with the business rules. In both projects the adherence to offshore code reviews prior to sending to our in-house staff was not maintained, resulting in many bugs raised later in the cycles. In the reviews we took code that was a representative sample (chosen at random, but was a deep slice) and did a review. Additionally we looked at key strategic areas for performance bottlenecks and security to ensure code conventions, standards, and requirements were being met
Integrate as frequently as possible: Our teams use branches to logically separate the project or feature into areas for work. As a matter of practice, the more frequently these branches can be integrated the faster you can identify and resolve defects.
Where possible, leverage frameworks: Use application blocks (Enterprise Library), Guidance Automation (Web Client Software Factory, Smart Client Software Factory) to solve common problems. Where possible, create your own frameworks and reuse them on projects to reduce the amount of custom code required to be written, which decreases development time and improves quality.
Use Design Patterns: These patterns illustrate common ways to solve problems. Being standard, they can convey meaning across geographical boundaries. Further, by implementing patterns maintenance complexity is reduced and documentation and clarity is improved. A different vendor in a different country, maybe even in-house staff, may need to understand and modify the code—using a proven design allows ramp up to easily occur.
Use useful metrics to measure quality and progress: Build metrics into SLAs and look at meaningful metrics, such as defect density, bug reactivations, code churn, code coverage (unit test or otherwise), project velocity, etc.

Leveraging Visual Studio Team System

I have found that VSTS handles the needs of geographically dispersed development teams (and their inherent challenges) in a variety of ways:

"Better" communication and coordination: Team Foundation Server provides the ability to track work items (tasks, bugs, issues, requirements, risks, change requests) in addition to source code and documentation. The architecture, utilizing web services and cache servers, is optimized for distributed teams working over slow or unreliable connections. The product also allows for customizable notification of events to give you the clarity you need from the projects and help improve onsite/offshore communications
"Better" status reports: Coupled with the data warehouse maintaining statistics and work item history, reports can be generated to give you insight in to some of the metrics I mentioned above. The platform is also extensible, utilizing SQL Analysis Services (SSAS) and SQL Server Reporting Services (SSRS) to allow team members to gather information with various levels of granularity.
"Better" code quality: the product also includes a unit testing framework that works with the centralized build and reporting portions to give indications of code churn, unit test coverage, and pass rates. Check in policies can be configured to check if the code compiles, run static code analysis, even run selected tests

Each one of these allow the onsite team to ensure compliance with corporate standards and gain trust in confidence in the offshore team's ability to deliver a quality, stable product."

In his another blog he explains how VSTS can help prevent failures in outsourced projects

How can Visual Studio Team System help?

Gartner advises companies that plan on offshoring work to figure out their IT process maturity and identify gaps in your process. As previously mentioned, it is important to set all expectations clearly up front with your vendor. When using Visual Studio 2005 with Team Foundation Server, several mechanisms out of box enable teams to work effectively in these environments:

Process: out of the box, Visual Studio Team Foundation server includes MSF for CMMI Process Improvement Level 3 and MSF for Agile Development work item templates. By utilizing the templates and the process behind them, teams can effectively work across physical boundaries with increased confidence and transparency, allowing software development activities to be predictable and success repeatable.
Communication: Visual Studio Team System facilitates the transparency between individuals and teams with work items, a shared team portal, integrated change management, and a common data repository. The availability of information, and insight into an individual's progress, creates a more unified work environment regardless of physical location. Project managers can stay informed on an individual's progress without having to visit each individual—having real time information about each individual's work and their progress allows project managers to create precise schedules and report more accurately to management
Productivity: utilizing the common repository, managers and leads can answer common questions such as: What's in the current build that QA can test today; are requirements being met; are my teams adhereing to quality standards; is the product ready. Further, it provides the single team portal for integrating source code, issue tracking, project plans, vision statements and others that are critical assets to a project team.

This should definitely be useful !

How to change process guidance template on existing project?

srikanth_r — Fri, 18 May 2007 09:12:00 GMT

I found this interesting question in the VSTS MSF Forum that many customers have and answer from Alan Ridlehoover . Heres the answer for it.....

The short answer is no, there is no way to change the process template of an existing project.

Think of it like this: A process template in TFS is similar to a document template in Word.

In Word, all documents are based on a template. The default template in Word 2007 is Normal.DOTX. It contains your default font and other formatting preferences. Other templates, such as UrbanLetter.DOTX, contain different formatting as well as placeholders for content. Once a document is created based on a specific template, you can change the content and/or formatting without affecting the underlying template.
In TFS, all projects are based on a process template. There is no default process template. Instead, you must choose from either MSF for Agile or MSF for CMMI. Other templates are also available from third parties. Once a project is created, you can change the content (specific work items, etc.) and/or formatting (work item type definitions) without affecting the underlying template. (More on this below.)

That said, if you still wish to begin using the CMMI template, my advice is to begin fresh, with a new project. To do this:

Move your source code either by getting it out of the old project and checking it in to the new project (which will not bring history), or by starting the new project off a branch of the old source tree (which will bring history)
Move your work items via Excel (which will not bring history - to my knowledge, there is no way to keep history when moving work items between projects)
- In the old project, create a query with all the fields and records you want and export it to Excel
- In the new project create a query with all the fields you want to import and export it to Excel
- Cut & paste (or write a macro)

Alternatively, it is also possible to modify the individual work item type definitions within an existing project using WITExport.exe and WITImport.exe. Together these programs import and export work item type definitions from an existing project (as XML documents). So, you can export a WIT, modify it, and import it. If you go this direction, you will definitely want to check out the Visual Studio 2005 SDK 3.0 - September 2006 RTM (which you can find here), as it contains the documentation for the XML format you'll be modifying. Note: Changing a WIT for a specific project does NOT update the underlying process template. Also note that going this direction may have negative side-effects, such as broken reports. The safest method is to create a new project as described above.

Team System Widgets

srikanth_r — Thu, 17 May 2007 18:05:00 GMT

Here's is a cool list of various Visual Studio 2005 and Team Foundation Server add-ins, add-ons, widgets, and extensibility solutions.

Cheers

Srikanth

Work Item Creator is now on CodePlex

srikanth_r — Thu, 17 May 2007 10:26:00 GMT

Work Item Creator is a Standalone application that enable you to create Work Items and organize them in a hierarchical way.
This software also includes WINetwork, a Team System Alert Web Service that is aware of Work Item changes, and update related ones using predefined schemas.

Here's the screenshot of the tool

How Tos (Visual Studio Team System)

srikanth_r — Mon, 14 May 2007 11:06:00 GMT

Find the answers for the below questions here

How To: Perform a Baseless Merge in Team Foundation Server
How To: Step Through Creating a Custom Check-in Policy for Team Foundation Server 2005
How To: Step Through Creating Your Source Tree in Team Foundation Server 2005
How To: Structure Your ASP.NET Applications for Team Foundation Server
How To: Structure Your Source Control Folders in Team Foundation Server
How To: Structure Your Windows Applications for Team Foundation Server

Happy Reading !

Visual Studio Code Metrics

srikanth_r — Sat, 12 May 2007 12:36:00 GMT

The new Code Metrics feature for Visual Studio ‘Orcas’! Available in Visual Studio Team Developer and Team Suite, this new feature allows users to generate code metrics for projects and solutions and displays the results in the Code Metrics Results tool window. An example Code metrics Results window is shown below

This feature can generate five different metrics: Maintainability Index, Cyclomatic Complexity, Depth of Inheritance, Class Coupling, and Lines of Code.

Maintainability Index - Measures ease of code Maintanance and higher values for this is better

Cyclomatic Complexity - Measures the number of branches and lower values for this is better

Depth of Inheritance - This measures the lenght of object inheritance hierarchy and lower values for this is better

Class Coupling - Measures number of classes that are referenced and lower values are better

Lines of Code - Measures lines of executable code and lower values are better

All metrics are averaged at the type, namespace, and assembly levels with the exception of Class Coupling. The Class Coupling metric displays the total number of distinct types referenced at the method and type levels rather than the total number of type references.

As you can see in the figure Visual Studio gives Red or Green icon for the index based on which you can change your code to improve code quality.

Users will are able to sort the results in the window by column. For example, above the results are sorted by the Maintainability Index column. Note that the proper hierarchy is maintained after sorting

Users can also filter the results from a particular metric for values between a specified minimum and maximum value. As you can see in below the results of filtering out any results with a Maintainability Index greater than 100 are displayed.

Users can also export these results into Excel where they can perform their own calculations and transformations.

To generate code metrics, simply do the following:

In Solution Explorer, right-click on your solution/project and choose Generate Code Metrics

Download Orcas and try it out !!!

Teamprise Announces Java SDK for Visual Studio Team Foundation Server

srikanth_r — Wed, 09 May 2007 03:18:00 GMT

Teamprise today announced plans for the creation of a Java Software Development Kit (SDK) for Microsoft Visual Studio 2005 Team Foundation Server. The Teamprise Java SDK for Team Foundation Server will enable software developers to create custom applications in the Java programming language that have access to the full set of features in Team Foundation Server, including source code control and work item tracking.

Check out for more information here

Advanced Load Testing Features of Visual Studio Team System

srikanth_r — Tue, 08 May 2007 12:23:00 GMT

Visual Studio Team Test Edition provides two capabilities that can allow users to create load tests of web sites quickly and easily: the Web Test recorder and the Load Test wizard. In fact, these tools make this process so easy that it is tempting to use the web tests and the load tests that result from using these tools without much modification. However, to use the web testing and load testing capability most effectively it is beneficial to understand how to use other web test and load test properties that are not set by these tools. This note describes some of the most important considerations for load testing with Visual Studio Team System that are not addressed by the web test recorder and the load test wizard.

For an excellent description of the process of effective web site load testing that is not specific to Visual Studio Team System, see the paper The Science of Web site Load Testing by Alberto Savoia at http://www.cmgitalia.it/download/seminario_marzo_2004/scienze_web_site_load_testing_savoia.pdf.

Verify web tests and unit tests

Before running a load test, it is a good idea to make sure that all of the tests contained in the load test will pass when run by themselves by running them from either the Test Explorer or Test View windows, or from the Web Test editor in the case of a web test. For data bound web tests, run through all of the data values.

See Josh Christie’s excellent paper Web Test Authoring and Debugging Techniques at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnvs05/html/WTAuthDebug.asp for more details on creating and verify web tests. Some additional web test properties that are useful when running load test are described later in this document.

Choose an appropriate load profile

Choose a load profile for each Scenario in your load test that is appropriate for your test goals. Here is a summary of the three choices for load profile.

Using a Constant Profile
A Constant Load Profile can be used to run the same user load throughout a load test. Be careful about using a Constant Load Profile with a high user count; doing so may place an unreasonable and unrealistic demand on your server(s) at the beginning of the load test. For example, if your load test contains a web test that starts with a request to a home page, and you set up the load test with a constant load of 1,000 users, the load test will submit the first 1,000 requests to the home page as fast as possible. This may not be a realistic simulation of real-world access to your web site. To mitigate this, consider using a Step Load Profile that ramps up gradually to 1,000 users, or specify a warm-up period in the Load Test Run Settings. If a warm-up period is specified, the load test will automatically increase the load gradually during the warm-up period.

Using a Step Load Profile
A Step Load Profile can be used to increase the load on the server(s) as the load test runs so that you can see how performance varies as the user load increases. For example, to see how your server(s) perform as the user load increasing to 2,000 users, you might run a 10-hour load test using a Step Load Profile with the following properties:
Initial User Count: 100
Maximum User Count: 2000
Step Duration (seconds): 1800
Step Ramp Time (seconds): 20
Step User Count: 100
These settings have the load test running for 30 minutes (1800 seconds) at user loads of 100, 200, 300, up to 2,000 users. The Step Ramp Time property is worth special mention here because it is the only one of these properties that is not available to choose in the Load Test Wizard. This property allows the increase from one step to the next (for example from 100 to 200 users) to be gradual rather than immediate. In this example, the user load would be increased from 100 to 200 users over a 20 second period (an increase of 5 users every second).

Using a Goal-Based Load Profile
A Goal Based Load Profile is useful when you want to determine the number of users that your system can support before reaching some level of resource utilization. This option works best when you have already identified the limiting resource (i.e. the bottleneck) in your system. For example, if you know that the limiting resource in your system is the CPU on your database server, and you want to see how many users can be supported when the CPU on the database server is approximately 75% busy, you could use a Goal Based Load Profile with the goal of keeping the value of the performance counter “%Processor Time” between 70% and 80%. One thing to watch out for is that if some other resource is limiting the throughput of the system, the goal specified by the Goal Based Load Profile may never be reached, and the user load will continue to rise until the value specified for the Maximum User Count is reached. This is usually not the desired load, so be careful about the choice of the performance counter in the Goal Based Load Profile, and also make a conscious decision about the value for the Maximum User Count to place an upper bound on the user load.

Choosing the location of the Load Test Results Store
When the Visual Studio Team Test Controller is installed, the Load Test Results Store is set up to use an instance of SQL Express that is installed on the controller computer. SQL Express is limited to using a maximum of 4 GB of disk space. If you are going to run many load tests and want to keep the results for a while, you should consider configuring the Load Test Results Store to use an instance of the full SQL Server product if available. See the Visual Studio Team Test documentation for instructions on setting up the database to be used as the Load Test Results Store.
Increase the performance counter sampling interval for longer tests
Choose an appropriate value for the “Sample Rate” property in the Load Test Run Settings based on the length of your load test. A smaller sample rate, such as the default value of five seconds, requires more space in the load test results database. For longer load tests, increasing the sample rate reduces the amount of data collected.
Here are some guidelines for sample rates:

Load Test Duration Recommended Sample Rate
      < 1 Hour                   5 seconds
    1 - 8 Hours                 15 seconds
   8 - 24 Hours                 30 seconds
     > 24 Hours                 60 seconds

Consider including Timing Details to collect percentile data
There is a property on the Run Settings in the Load Test Editor named "Timing Details Storage". If Timing Details Storage is enabled, then the time to execute each individual test, transaction, and page during the load test will be stored in the load test results repository. This allows 90th and 95th percentile data to be shown in the load test analyzer in the Tests, Transactions, and Pages tables. The amount of space required in the load test results repository to store the Timing Details data may be very large, especially for longer running load tests. Also, the time to store this data in the load test results repository at the end of the load test is longer because this data is stored on the load test agents until the load test has finished executing at which time the data is stored into the repository. For these reasons, Timing Details is disabled by default. However if sufficient disk space is available in the load test results repository, you may wish to enable Timing Details to get the percentile data. Note that there are two choices for enabling Timing Details in the Run Settings properties named "StatisticsOnly" and "AllIndividualDetails". With either option, all of the individual tests, pages, and transactions are timed, and percentile data is calculated from the individual timing data. The difference is that with the StatisticsOnly option, once the percentile data has been calculated, the individual timing data is deleted from the repository. This reduces the amount of space required in the repository when using Timing Details. However, advanced users may want to process the timing detail data in other way using SQL tools, in which case the AllIndividualDetails option should be used so that the timing detail data is available for that processing.

Consider enabling SQL Tracing
There is a set of properties on the Run Settings in the Load Test Editor that allow the SQL tracing feature of Microsoft SQL Server to be enabled for the duration of the load test. If enabled, this allows SQL trace data to be displayed in the load test analyzer on the "SQL Trace" table available in the Tables dropdown. This is a fairly easy-to-use alternative to starting a separate SQL Profiler session while the load test is running to diagnose SQL performance problems. To enable this feature, the user running the load test (or the controller user in the case of a load test run on a rig) must have the SQL privileges needed to perform SQL tracing, and a directory (usually a share) where the trace file will be written must be specified. At the completion of the load test, the trace file data is imported into the load test repository and associated with the load test that was run so that it can be viewed at any later time using the load test analyzer.

Don’t Overload the Agent(s)
If an agent machine has more than 75% CPU utilization or has less than 10% of physical memory available, add more agents to your test rig to ensure that the agent machine does not become the bottleneck in your load test.

Add an Analysis Comment
After the load test is complete and you have spent some time analyzing the results, you can a short one line description and an arbitrarily long analysis comment to be stored permanently with the load test result. To do this, in the load test result viewer, right click and choose the “Analysis” option. This brings up a dialog that allows you to enter your analysis text which is stored in the load test results database you click OK to close the dialog.

Consideration for Load Tests that contain Web Tests
The following considerations apply to load tests that contain web tests, but are not applicable to load tests that contain only unit tests.

Choose the Appropriate Connection Pool Model
The property WebTest Connection Model found on the RunSettings property sheet in the Load Test editor allows you to choose between two models for managing the connections that the load test runtime engine uses for connecting to the target web site(s):

The ConnectionPerUser model most closely simulates the behavior of a real browser: each virtual user running a web test creates one or two connections to the web server that are dedicated to that virtual user. The first connection is established when the first request in the web test is issued. A second connection may be used when a page contains more than one dependent request; these requests may be issued in parallel using the two connections. These connections are re-used for subsequent requests within the web test, and are closed when the web test completes. The only drawback to the ConnectionPerUser model is that the number of connections held open on the agent machine may be high (as high as 2 times the user load), and the resources required to support this high connection count may limit the user load that can be driven from a single load test agent. The extra resources used are the memory associated with the connection and extra processing time to close and reopen the connections as web tests complete and new web tests are started.

The ConnectionPool model conserves the resources on the load test agent by sharing connections to the web server among multiple virtual web test users. When the Connection Model is ConnectionPool, the Connection Pool Size specifies the maximum number of connections to make between the load test agent and the web server. If the user load is larger than the connection pool size, then web tests executing on behalf of different virtual users will share a connection. This means that one web test will need to wait before issuing a request when another web test is using the connection. The average time that a web test waits before submitting a request is tracked by the load test performance counter “Avg. Connection Wait Time”. The value of this performance counter should be very low; if it is not, you need to increase the connection pool size to prevent it from limiting the throughput of your load test.

Consider setting response time goals for web test requests
One of the properties of a web test request is a response time goal. If you define response time goals for your web test requests, when the web test is run in a load test, the load test analyzer will report the percentage of the web tests executed within the load test for which the page response time was less than the goal. Note that the response time goal is for the entire page, meaning the time to receive the response for the request specified in the web test including the time to receive the response for any dependent requests. By default there are no response time goals defined.

Consider setting timeouts for web test requests
Another property of a web test request is a timeout value. By default there is no timeout value specified on web test requests, so if the web server never responds to the request, the web test request (and the web test) will never complete and no error will be reported. If a timeout value (even a high one such as 300 seconds) is set, then the web test request will eventually timeout, an error will be reported, and the web test will continue.

Choose a value for the “Percentage of New Users” property
There is a property on each scenario in a Load Test named "Percentage of New Users”. This property affects the way in which the load test runtime engine simulates the caching that would be performed by a web browser. The default value for the “Percentage of New Users” property is 100%. This means that each web test run in a load test is treated like a first time user to the web site who doesn't have any content from the web site in their browser cache from previous visits. Thus all requests in the web test (including all dependent requests such as images) are downloaded (except in the case where the same cacheable resource is requested more than once in a web test). If you are load testing a web site that has a significant number of return users who are likely to have images and other cacheable content cached locally, then using the default setting of 100% for “Percentage of New Users” will generate more download requests than would occur in real-world usage. In this case, you should estimate the percentage of visits to your web site that are from first time users of the web site, and set “Percentage of New Users” accordingly.

Consider setting the “ParseDependentRequests” property of your web test requests to false
By default, when a web test is executed, HTML responses received from the web server are parsed and any dependent requests such as images or style sheets are automatically submitted. This is usually desirable as it places the most realistic load on the server. However, if the goal of your load test is to place the maximum load on your application server as opposed to your web server, you may wish to disable this behavior by setting the ParseDependentRequests on all of your web test requests to false.

For more information please visit here

Running Tests with Code Coverage

srikanth_r — Tue, 08 May 2007 12:23:00 GMT

Three common work flow situations with steps to successfully collect code coverage data. These scenarios were built around different models of access to source code because that seems to be the underlying issue customers experience.

Every scenario minimally requires product binaries, product PDBs, test binaries, and Visual Studio Team System. For better, richer, and more flexible experiences you’ll need more, like access to test and/or product source code. In the end, you just want code coverage data so it doesn’t matter how you go about it. Pick a method that works for your setup and enjoy what VS Team System has to offer..

Scenario 1: Local Product and Test Sources

This is the most ideal scenario. One has the product and test projects loaded in VS. One has full ability to build both which provides the most flexible experience.

This is probably a developer or a tester who work directly on the product. They are trusted to and it is logistically possible to have full access.

Gathering Steps

Load the product and test sources in a single solution in Visual Studio Team System 2005.
Perform a full build.
Open the Run Configuration you intend to use for this run (‘localtestrun.tesrunconfig’ is the default)
Select the Code coverage page in the resulting Dialog
Check the Binaries you wish to be instrumented
Execute the tests you desire to run, either through Test Manager or Test View.

Review Code Coverage data

When tests are finished, open the Code Coverage window and review the results.

Scenario 2: Product Binaries and Test Sources

This is probably the most likely scenario. One has product binaries (and PDB files!) from a daily build. One has access to the test sources and can build test binaries. This nearly provides the same flexibility as Scenario 1.

In this model, the user may be the same developer or tester as in Scenario 1, but they may be encouraged to only test against “official” builds from a common source. This could also be a person testing offsite from core development who receives regular drops of builds from the development team. They may be interested in gauging the comprehensiveness of their testing as measured by code coverage, or this activity may have been requested by the development team. In the latter case, the code coverage results file will be sent to the development team for analysis.

Gathering Steps

Load the test sources in Visual Studio Team System 2005.
Copy the desired product binaries and PDB files locally.
Build the test sources.
Open the Run Configuration you intend to use for this run (‘localtestrun.tesrunconfig’ is the default)
Select the Code coverage page in the resulting Dialog
Check the Binaries you wish to be instrumented
Execute the tests you desire to run, either through Test Manager or Test View.

Review Code Coverage data

When tests are finished, open the Code Coverage window and review the results. If you don’t have access the source code, you will only be able to see the blocks covered etc., rather than full detail and coloring of the source code.

Scenario 3: Product and Test Binaries Only

This scenario requires the most work and provides the least amount of benefit, but unfortunately it is all too common. All the great work VS does for you when you have your own test project is gone because you only have test binaries now. Additionally, this could include running manual tests so there might be no automation to run at all.

That said the scenario still works. You can get code coverage data. Also, if that data is shared with someone who has access to source code, then they will get full benefit from the results as they can see code coverage detail and line coloring.

Gathering Steps

Copy the desired product binaries, product PDBs, and test binaries files locally.
Instrument the product binaries:
1. Open up a Visual Studio 2005 Command Prompt (Start | All Programs | Microsoft Visual Studio 2005 | Visual Studio tools | Visual Studio Command Prompt).
2. Change to the directory where your binaries are located
3. Execute “vsinstr –coverage <assemblyname>” for all your binaries. e.g. If your binary name is ‘coolproduct.dll’, then type “vsinstr –coverage coolproduct.dll”.
Start the Code Coverage monitoring process
1. Return to the previous command prompt
2. Type “vsperfcmd –start:coverage –output:outputFilename.coverage”.
Execute your tests
1. For automated tests, navigate to the directory where your test binaries are located and execute “mstest /testcontainer:TestDll.dll”.
2. Run manual tests, if you have them
End the Code Coverage monitoring process
1. In the command prompt opened earlier, type “vsperfcmd –shutdown”. This shuts down the coverage monitor and saves the coverage data to disk.

Note: There are alternatives to manually executing these commands, but they are slightly more complex.

Start VS and create a test project
Add your production DLLs as references to the project
Open the Run Configuration you intend to use for this run (‘localtestrun.tesrunconfig’ is the default)
Select the Code coverage page in the resulting Dialog
Check the Binaries you wish to be instrumented
Save the runconfig file
Exit the IDE
Copy the Product DLLs and Test DLLs into to the correct place along with the runconfig
Execute “mstest /testcontainer:testdll.dll /runconfig:runconfig.testrunconfig”

This will instrument and execute the tests, while correctly starting & shutting down the coverage collector. If you still wish to instrument the binaries yourself, then you still need a dummy-dll to be instrumented. The tools do not start the coverage monitor unless the tool itself has instrumented a DLL.

Review Code Coverage data

Open the .coverage file in Visual Studio.

Got this information here

Prescription Guidance for Visual Studio Team System

srikanth_r — Thu, 03 May 2007 21:21:00 GMT

Many customers must be looking for more guidance on how best to use our VSTS in the real world, the VSTS product group embarked on a project to build a definitive Body of Guidance (BOG) for VSTS. This includes How Tos, Guidelines, Practices, Q&A, video-based guidance, and more.

The VSTS Guidance Project contains information on all the latest prescriptive guidance developed by the VSTS Guidance Project team.

You can also visit J.D.Meier's blog for more information

VSTS Cool case Studies

srikanth_r — Thu, 03 May 2007 09:15:00 GMT

Total number of Case studies for Team System increased to five:

New! KBC Bank used Microsoft Visual Studios Team System and Team Foundation Server to improve application-building methodologies, increase productivity, and accelerate application deployment.
New! EDS used Microsoft Visual Studio Team System to support a more efficient global development strategy, realigning its internal software development assets and improving the productivity of its developers and testers.
New! Dell: By using Microsoft Visual Studio Team Foundation Server to deploy a global source code management platform, Dell is able maintain a centralized repository for all source code and provide it to developers on a just-in-time basis regardless of their location. Improved source code management has also enabled Dell to consolidate its source code onto fewer servers, redeploy 100 system administrators, and improve the productivity of its developers.
A social networking Web site used Microsoft Visual Studio Team System to improve the productivity of its developers and project managers as well as to reduce software costs.
A global manufacturer deployed Microsoft Visual Studio Team System to improve the consistency and efficiency of internal software development, enabling it to support adoption of the widely recognized Capability Maturity Model for software development. This enabled the company to increase developer, project manager, and tester productivity while improving software quality.

Each of these presents a compelling case for the benefits of adopting Team System. The most interesting sections are the ROI, key benefits / costs and lessons learned.

Happy Reading !

Visual Studio Team System - Future Releases

srikanth_r — Wed, 02 May 2007 18:53:00 GMT

A lot of great information about the roadmap for Visual Studio Team System future releases has been posted at Visual Studio Team System Future Releases . If you're interested in what's coming soon in "Orcas" and after that in "Rosario", check this out.

This is great information to consider as you are planning your adoption, mapping out deployment, or even evaluating specific functionality.

Also included in the list are more details on power tools...

Link to Visual Studio Team System - Future Releases