This morning, I wanted to address some of the questions and concerns I've received on some of the Unit Testing features in Visual Studio Team System 2005.Thank You for Your FeedbackThe response to VSTS has been amazing. The questions coming in the form of blogs, newsgroup posts, email questions, etc are very insightful and much appreciated. Thanks for taking the time to voice your concerns and help us build the best product possible.One of the largest contributors to the set of questions I address here is jarnold in his blog: http://weblogs.asp.net/jarnold/archive/2004/09/15/229893.aspx
Question: Why bother with unit testing in VSTS?
Answer: On one level, Visual Studio Team System is about expanding Visual Studio so that roles beyond the developer and functions beyond standard code development can be addressed. One of many roles we added support for is the Tester and one of the many roles we expanded functionality for is the Developer. Adding built-in support for Testers and treating software testing as a first class citizen and expanding Developer functionality to include unit testing are great things for the software engineering community. The overwhelming customer response we’ve received, and one we agree with whole-heartily, is ‘it’s about time’ these features made their way into Microsoft products. Question: What about other tools?
Answer: By adding Unit Testing, Web Testing, Load Testing, Code Analysis, Profiling, Manual Testing, Centralized Reporting, etc, etc, etc into Visual Studio Team System we’re trying to give many roles involved in the product lifecycle a great set of tools. This is not to say we believe Microsoft can and will create every tool you’ll need. It’s clear that our customers want a diverse set of functionality and the tools any one vendor can release will only be part of the story. This is why Visual Studio has always had an extensibility story and it’s why we expanded our extensibility story in Visual Studio Team System. Adding unit testing into VSTS was not a statement about the validity or relevance of any other unit testing tool in the community. Adding unit testing into VSTS was about empowering our customers.
Question: Isn’t NUnit cool?
Answer: Yes. We think NUnit is a great tool and we believe NUnit has a place in the community. Additionally, we’re not trying to stop NUnit or any other tool from plugging into Visual Studio or Visual Studio Team System. You should use the tools that provide you the most value. We hope our tools integrated into Visual Studio are compelling. If something else works for you, and you use VS or VSTS for some other support, we’ll do are best to provide a platform so 3rd party tools can target integration into Visual Studio Team System and take advantage of our tools ecosystem.
Question: Is Microsoft going to provide some guidance around unit testing and unit testing features?
Answer: Yes. We have our MSF guidance, and we’re adding Test and Development specific sections to MSF along with help and other forms of guidance around our features. Check out: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnvsent/html/vsts-msf.asp
Question: Does the Unit Test Generation Dialog assume a One-to-One Mapping between Development Code and Unit Test Code?
Answer: The Unit Test Generation dialog lets you decide which elements of your code you want to generate tests for. Yes, you can get very specific and generate a single test per code element. That said, I’d stop short of saying the tool ‘assumes that there is a one-to-one mapping’ and instead say the tool allows you to generate tests at a one-to-one mapping granularity. The Unit Test Generation feature isn’t trying to force the way you scope your unit tests. The Unit Test Generation feature is trying to support as much functionality as it can while still be useful for most users. In that pursuit, the feature enables customers to get very granular about code element selection.
Question: Does the Unit Test Generation dialog promote a test-last culture?
Answer: The Unit Test Generation dialog helps you if you have an existing code base and you want to generate a set of unit tests quickly. The Unit Test Generation feature is not about promoting one methodology over another (Test-First vs. Test-Last). Instead, the feature is intended to help customers that have an existing code base and want to generate a set of tests quickly. Those customers that prefer ‘Test-First’ may not use the Unit Test Generation feature. Those customers that prefer ‘Test-Last’ or some how end up with an existing code base and need to use a ‘Test Last’ strategy will have this feature to use.
Question: Is Microsoft making a statement about the way I should test (features vs. implementation)?
Answer: We try to give our customers a useful set of features without dictating a process. Each customer needs to pick the right methodology \ approach to development and test that makes sense.
Question: Does coupling your tests to private implementation hinder or facilitate refactoring?
Answer: My general rule of thumb is that the more dependencies a piece of code has, the higher the cost of refactoring (or at least the higher the percentage chance of hitting issues). As a result, if a customer chooses to test a private method, there most likely will be some overhead in terms of refactoring. In some cases, this overhead may not be worth it. In other cases, the customer may want to pay the cost because the gain is significant enough.
Question: Isn’t offering a Unit Test Code Generation feature based off of existing development code a step backward in terms of Test Driven Development?
Answer: I think many customers face a mixed environment in terms of ‘Test-First’ vs. ‘Test-Last’. In certain cases, Test-First simply doesn’t make sense (an existing code base without unit tests). In other cases, some customers may simply prefer ‘Test-Last’ to ‘Test-First’. The Unit Test Code Generation feature is certainly helpful when Test-Last is used. However, I don’t think having the technology is a step-back in terms of the TDD community.
Question: The refactoring support in VS isn’t good enough… Will it get better?
Answer: We are very aware of the other products on the market that do more in terms of refactoring and code generation from test code to development code. This is something we are planning on improving and is certainly an evolutionary process. In that vein, customers have indicated to us that Generate Method Stub is a great beginning.
Question: The default names in the Unit Test Code Generation dialog don’t work for me… How can I change them?
Answer: The Unit Test Code Generation dialog allows you to enter specific names for specific tests. It also allows you to make use of little helpers like ‘[Class]’ and ‘[Method]’ to reference the Class of the development code or the Method under test.