I was at TechEd2009 a few weeks ago and I was so sad we weren’t able to talk more about VS2010 there as much as we wanted to. One of my highlights of the conference was standing in the lunch line while a group of folks from a variety of different companies discussed the new historical debugger work we are doing in VS2010. They were clearly excited and had a bunch of questions so I introduced myself as the Development manager for the team that worked on that and we sat and had lunch. Although there is clearly more we can do in this area than we will be able to ship in VS2010, it was clear these folks were looking forward to picking up the Beta. Of course at that time, the Beta was slowly cooking, but now it’s ready, and we can have a good conversation. The team I’m responsible for worked on bringing this pipeline of value and innovation in VS2010:
NEW FEATURE AREA Historical Debugger
- Explore important diagnostic events that happened in your application every time you press F5. Think of this as having things like Procmon/Filemon/Regmon and Fiddler integrated directly into your debugging experience! Except we do ones like exceptions and events specific to the libraries you are using like WinForms, Webforms, WPF etc.
- Go back to previous break states in the debug history. Ever written a value from the autos window or locals window down on a post-it and continued on debugging? Now just go back to the point you stopped at and see the value.
- Turn the historical debugger dial to high and see a history of the code execution, the method calls, the object states and call sites in the tree view of the Debug History window. Walk back to those points in time and understand what happened.
- Open logs of both diagnostic events and application recording created from running the new Visual Studio Test tools so you can figure out why test are opening this issue on you when clearly - “It works on my machine”.
NEW FEATURE AREA Test Impact Analysis
- Understand which tests to run to verify the changes you are making, as opposed to running every test.
- Understand the specific methods changed that are causing the requirement to run a specific test.
- Ask the question – if I changed this method – what tests would need to be run to verify it?
Debugger
- New: Tear off your datatips and leave them floating on the page while you debug. No need to go and hover over things time and again.
- New: WPF visualizer. Inspect the state of visual elements in WPF applications. No need to hunt down the specific property you have set based on which base class it is set in, use the filter mechanism in the visualizer to find it. See the rendered visual of the element in the preview pane.
- New: WPF logging. Track issues such as WPF DataBinding rather than asking “Why is may control/window black”
- New: minidump summary page. Use File->Open to open a minidump and see the data about the environment and modules that were present in the minidump. Search for your module in the list.
- New: Managed minidumps. Create minidumps for managed code running on the v4.0 runtime. Open those minidumps in Visual Studio and explore them.
- New: Thread window. A whole new thread window designed to help you find the important threads and figure out what they are doing. Show threads from all the processes you are debugging. Show the callstacks for all of those threads in the threads window itself. Group them by flagging them. Filter threads to only threads that have a particular pattern in their stack.
- New: Export your breakpoints. Tag and filter them in the breakpoints window. Export and give to a co-worker to let them import them and figure out a bug in some code you know
- New: Support for tasks in the task libraries as first class citizens
- New: New parallel stacks window shows the stacks of your application in a graphical format
- Improved: Support for opening executables as a project and using that as the debug target
- Improved: Support for inter-op debugging of x64 processes.
- Improved: C++ – support strcmp/strncmp in conditional breakpoint expressions
Profiler
- New: Tier interaction profiling. Understand the cost of the calls to the database from your mid-tier. See the SQL queries that make up those costs.
- New: Performance rules. Get guidance on common performance issues that the profiler spots in the performance results.
- New: Thread contention profiling. Understand which resources your threads are most contending on. See the contention pattern between threads fighting over locks, and drill down directly to the line of code trying to acquire the resource.
- New: Parallel performance analyzer. Understand core utilization and how parallel your app is and could be.
- Improved: Directly attach to managed CLR 4.0 applications without pre-setting environment variables.
- New: Profiler works on VPC, Hyper-V and VMWare
- New: Just my code support.
- New and improved: Integration with LoadTest. Switch on the profiler in the test settings for the load test. Open the load test result after the test and click on the profiler results icon in the tool bar. Drill right down to the code hotspots for your code under load.
- Improved: Summary page shows CPU usage charts and hot stacks rather than just methods. An action link pane provides support for “one click to action”.
- New: Function details page. Understand graphically how time is being spent in this method, and the children of that method. Visualize the source code and show the hot lines line by line with performance statistics visualized in the code at the line level.
Code Coverage
- Improved: Support for code coverage on 64bit code under test.
Whew! That should keep my blog full for a few months anyway!
Please pull down the Beta and give it a spin. It’s available from the product page here:
http://www.microsoft.com/visualstudio/en-us/products/2010/default.mspx
So far I have been running Beta1 and VSTS2008SP1 Side by Side with no issues. We spent a lot of time in this release focusing on setup, side by side and uninstall so I do think this is way better than in previous releases.
Fix those bugs, and stay posted for more deep dives in all of these areas. Sorry I didn’t drill deeper, but I wanted to get this post out today.
John
Every so often my team likes to make me feel useful. I don't get to write code and fix bugs as much as I used to when I arrived at Microsoft, but I do have a good few tricks up my sleeve. Someone from the profiler team was looking at a stack overflow in the analysis code that occured from some recent changes. As ever, these come from cyclic recursion patterns, and so you get the same few methods on the stack, and no good way to identify "when to stop" and what to inspect just before the exception:
A
B
C
A
B
C
...
My trick here is simple, and it's to do with "hit count" breakpoints. Set a breakpoint in one of the methods, but go ahead and add a hit count. You do this via:
Right click on the breakpoint->Select Hit Count...->hit when the hit count is greater than or equal to
Crank the hit count up to something insane (I tend to use 65536). Now let the program run to the crash point and see that the breakpoint never fires. Now go to the breakpoints window and look at the hit count for your breakpoint and it will tell you how many times that the code executed that breakpoint. Now go ahead and change the breakpoint hit count to only hit at that count or one or two below, and now you can debug through the last throes of the stack overflow....
Happy slaying! (Sorry for the lack of pictures)...
John
One of the more interesting things about working at Microsoft is the close collaboration the Microsoft Research (MSR) group has with academic research in many universities around the world. Today I was able to go to a talk form one of the researchers I most admire in the software engineering field: Prof. Andreas Zeller of Saarsland University in Germany. Prof. Zeller has a long history of research into bugs - what causes them, how to detect them and how to fix them. His book "Why programs fail" is one of very few books to cover this area.
Prof. Zeller's talk was on the subject of mutation testing to understand test suite effectiveness. You can find out more, including watching a pre-record of pretty much the same talk I attended, here:
http://www.st.cs.uni-saarland.de/mutation/
While I had some questions about the mutations that Prof. Zeller's team used, and my own concerns about how well test team's manage to make their test suites model after user's behaviours, there is no doubt that this research area is very interesting. The team is as yet unable to make any conclusions about the effectiveness of these techniques over existing coverage metrics.
My own perspective here is that anything that gets people thinking more about the test effectiveness, and less about specific numbers and metrics in things like statement coverage, the better. I'm continually asked about "what number of coverage should I shoot for" and "how much coverage does Microsoft aim for". This is a fruitless task in my opinion. Our coverage tells you one thing and one thing only - what code was not exercised at all by my test bed. You then need to make a judgement call about what to do about it. Is the functionality important? Is it really dead code? All of these go into the mix when making the decision to do work and write new tests.
Whatever your own opinion on these, Prof. Zeller's work continues to be thought provoking, so please go check it out,
John
I'll be out at TechEd this year and delivering the next version of my Debugger Tips and Tricks talk, which I last gave at PDC.
http://channel9.msdn.com/pdc2008/TL59/
In the version of the talk for this year [DTL323], I'll have similar content on VS2005 and VS2008, but will have an updated version of VS2010 to show off some new debugger features if you all are interested. I may even do a booth talk on VS2010 in more depth since I want to keep the breakout talk balanced with things you can take advantage of today. If you are headed to TechEd, please go ahead and sign up for this session early. Last time the room was way too small, and I really like the opportunity to have a meaningful Q&A session during and after the talk. Not much ability to do that when you are a mile away!
See you all there!
John
A fix for a widely reported crash in the Visual Studio 2008 IDE is now available. This fixes a crash that occurs after changing layouts (undocking/docking windows) and then changing modes (start stop debug, or switch to/from full screen. This bug was introduced in Service Pack 1 and the fix for this is now available for download via Code Gallery at: http://code.msdn.microsoft.com/KB960075
John
Even when I switch to "Release builds"? We recently got a connect bug on this so I thought I'd share my thoughts.
In the debugger, we always try to minimize the impact we introduce on the debuggee, but there is always some work that we do in a synchronous way. We alway deal with breakpoints, even when they are in "tracepoint mode", we always do work when receiving a module load or thread create event. As I say, we try to minimize the impact of this work, and are constantly looking to improve the performance (and therefor usually the perturbance) on the target application. However the biggest contributor I see to app performance degradation is the fact that when launching under the debugger, the operating system uses the debug heap.
The VS debugger uses the standard Windows CreateProcess call to create the debuggee. See docs for that here: http://msdn.microsoft.com/en-us/library/ms682425(VS.85).aspx. We pass in the create flag DEBUG_PROCESS as documented here: http://msdn.microsoft.com/en-us/library/ms684863(VS.85).aspx
When we do that, Windows enables three heap debugging features:
- Enabling a tail check. This put a marker on the end of each allocation and when it is freed, the heap manager checks the marker is intact.
- Enabling free check. If the process writes to the block after free, the heap manager detects this.
- Checking paramaters. The heap manager checks for valid parameters in each call to any of the heap allocation/free functions.
1. and 2. have a sizable performance impact when freeing memory. For more information on this subject, read Mark Russinovich and David Solomon's book "Windows Internals" Chapter 7.
There are two workarounds for this issue.
- Use CTRL+F5 and attach to your application instead. I realize this is not always ideal.
- Add the setting _NO_DEBUG_HEAP=1 to the environment settings in the debugger settings for your launch project. This only works in C++ projects today. You could force inheritance here by adding that to Visual Studio's environment, which your target application inherits from.
Remember however that you are giving up a lot of validation that the OS does on your behalf. Catching memory corruption issues early is very important, so don't take this step blindly.
Happy hunting
John
This is an opinion piece, and my opinion at that, not of my employer.
I was watching a debate on one of the internal support email lists for debugger related things last year when an interesting event occured. There is currently a limitation in the Windows OS that prevents ModuleLoad events being sent after ~500 of those events have been sent. The conversation wandered into why you get some gains from performance if you keep the number of dlls down, and ensure they have their base adresses set. To which there was a reply of (paraphrase):
"I don't want to think about that, the operating system should keep all that from me, if I want to have thousands of dlls I should be able to"
Which set off alarm bells for me. The hyperbolae of this statement is of course "don't bother me with constraints". It also surfaces in how people treat memory allocation and usage in managed code ("Hey - there's a garbage collector - there are no leaks. Um - why am I running out of memory - it must be a CLR bug"). Complete ignorance of the constraints of systems is a dangerous place to be.
I self identify as an engineer. But of course many of us who are in professional software development came through Computer Science courses as their degrees. Looking back on my own education, there were very few courses that armed me with the engineering skills I grew in my first few years as a developer. I learned (the hard way in many cases) what it takes to engineer solutions that work within the constraints of the systems my customers expected. I'm in no way bashing Computer Science. Where would our industry be without the deep thinking and experimentation that carries on to this very day. But when onboarding software developers, I think we need to do more to make them aware of engineering practices and principles in a more structured way. In our own team we do this through a combination of mentoring and training, but by no means do we have it down pat (who does). Different situations may call for different solutions. Until recently I never thought of how much that science vs. engineering discipline is actually implicitly demonstrated as opposed to being explicitly stated, but now I think on it, it's a useful conversation to have with those new hires.
Something to ponder as the next wave of scientist/engineers walk through the doors into industry from university. Please discuss through the comments section!
John
Last year's CTP of VSTS 2010 was one of the major milestones on the road to shipping what we think will be a great release. There's a lot of value, and much of it previewed in the CTP, but I've been feeling like we didn't get a huge amount of community feedback to help us fine tune the plan. Selfishly - I'm interested in your thoughts on two of the major innovations our team was responsible for:
Eliminate the “no-repro” bug with application flight recording.
How many times has a developer used the phrase “works on my machine” when responding to a bug? I know I have :) In many cases that may be true. By default the VSTS 2010 testing system will record a host of information:
1) Screenshots of the test in progress so developers can see the symptoms
2) An environment snapshot to aid in configuration mismatch diagnosis
3) An application flight recording that can be replayed in the debugger to allow developers to quickly get to the root cause of any issue
The application flight recorder technology is a key element to actually finding and solving bugs faster. Not only can developers see symptoms, they can understand what their code was up to at the time and get to the problem faster. To do this, we took advantage of one of Visual Studio’s core experiences – the VS debugger (which we are responsible for too). Many developers spend a large portion of their time using the VS debugger for both software construction and debugging. So we now provide a new set of debugging experiences that focus on looking at the application through the recording, and seamlessly integrate that with the debugger’s existing views, as opposed to building a new tool that people would have to learn and get some experience with. Any developer will be able to get value directly out of the gate. This also frees testers from extra rounds of validation (did that bug really exist), or working to reproduce a bug that only show up intermittently or even going to the trouble of setting up a test environment for the dev to repro and debug on live.
Understanding the test impact of code changes
A common question that testers in software organizations ask is “which tests should I run to ensure quality”. This is especially critical for teams with a test bed of many thousands of tests as they approach the release point of their software. Code changes can break the product, but also simply break the tests associated with them. All of this is cost, and as time progresses from the date the issue was added, it becomes more expensive to understand and fix.
VSTS 2010 intends to provide guidance around which tests have been impacted by changes in newer builds. As tests run in either automated or manual test scenarios, information is being recorded about which tests touch which code elements in a given project. This mapping is then transmitted to the test server which maintains a database of mappings. Team System then makes this information available to all the participants in the software process.
1) Test managers have a built in report of the test impact of any given new build. As well as understanding the tests impacted by changed source code, they can see which requirements have changed, or which bugs have been resolved, and by extension the test cases that are related to those. They can then decide to promote a new build to be “the testable build”
2) An individual tester can see this same information for their area, and make decisions on which of their tasks they ought to work on first
3) Developers can see the upstream impact of any change they make, and potentially run the impacted tests to understand if there is a breaking change, or if they have broken the product.
As we get closer to Beta, we should be going deeper on how these features work and what you can (and can't :) ) do with them, but until then, please go test drive them and give us the feedback!
Keep on fixing,
John
For the last few weeks we have had reports of stepping issues caused by people installing VS 2008 SP1. You can see our acknowledgement of this issue here:
http://social.msdn.microsoft.com/Forums/en-US/vsdebug/thread/f3fcb4fb-8a08-4fa0-8d58-9ed6f3eb1193
The symptoms are that while managed debugging some step commands do not complete as expected, and some breakpoints may be missed if running on hyperthreaded, multi-core or multi-proc hardware. We finally got to the root cause, and I'm glad to announce that we have now made a fix available for this issue. You can grab that here:
http://code.msdn.microsoft.com/KB957912/Release/ProjectReleases.aspx?ReleaseId=1796
John
The channel 9 rendering of my talk at PDC is now up for your general enjoyment/education/amusement.
http://channel9.msdn.com/pdc2008/TL59/
The topics I covered tricks in stepping, breakpoints, symbols, multi-threading and post-mortem. Judging from the evals the reception was pretty good, and I really wish I had the time to go deeper and go over more stuff. I was especially pleased with being highest rated talk in the TL track for job relevance :). I'd like to hear if folks would like to see more of this at PDC09. We are considering whether we should have pre-con sessions on debugging, profiling and diagnostics to really go deep. For the chap "D" from company "C" who was able to answer my opcode question, please get in touch. Other talks that I would encourage you to check out:
Steve doing his "Let's pretend this says System.String" line is one of my favourites of all time.
I'm not sure hte title of Habib and Justin's talk did justice to what we were showing off, which was the first introduction to historical debugging coming on line in VSTS 2010. However we received the highest evaluation scores for all of PDC (by some margin) for the Hands On Lab that Justin produced to demo this feature. We literally think it will change the way people debug, and will become increasingly important in diagnosing issues as platforms become more distributed and cloud based, e.g. Windows Azure.
Cheers
J
So I finally have some of my schedule figured out for the upcoming PDC. Most important for me is to actually turn up to my own talk :).
Visual Studio Debugger Tips & Tricks
Tue 10/28 | 12:45 PM-1:30 PM | 409A
I'll be going over a few of my personal favourites, talking about how and why we do these features, and showing a few things that will be new in VS 2010. I don't want to spoil the surprise too much though. After that (at 1:30), John Robbins is setting up an open space on "the future of debugging". I'm pretty excited by that obviously, and myself, David Gray, and a host of others from the team are looking forward to discussing this with you all.
The most interesting talks for me are the following, although I'm not sure I'll actually make it to most of the sessions.
|
|
|
|
|
|
|
TL19 Microsoft Visual Studio: Bringing out the Best in Multicore Systems |
Hazim Shafi |
10/27 |
01:45 PM – 03:30 PM |
Room 502A |
|
TL25 Parallel Programming for C++ Developers in the Next Version of Microsoft Visual Studio |
Rick Molloy |
10/27 |
03:30 PM – 04:45 PM
|
Room 408B |
|
TL03 Microsoft Visual Studio Team System: Software Diagnostics and Quality for Services |
Habib Heydarian; Justin Marks |
10/27 |
03:30 PM – 04:45 PM |
Room 515A |
|
BB39 Logging, Diagnosing, and Troubleshooting Applications Running Live in the Cloud |
Steve Garrity, Mark Gilbert |
10/29 |
01:15 PM – 02:30 PM |
Room 515B |
|
TL24 Improving .NET Application Performance and Scalability |
Steve Carroll;
Ed Glas |
10/29 |
01:15 PM – 02:30 PM |
Room 153 |
|
PC49 Microsoft .NET Framework: CLR Futures |
Joshua Goodman, Ian Carmichael |
10/30 |
10:15 AM – 11:30 AM
|
Room 153 |
|
TL26 Parallel Programming for Managed Developers with the Next Version of Microsoft Visual Studio |
Daniel Moth |
10/30 |
10:20 AM – 11:45 AM |
Petree Hall CD |
The ones marked in bold will be given by members of our team. Sadly there are a lot of conflicts. Grrr.
I'm particulalry excited by Justin and Habib's talk, as it lifts the lid on the application flight recorder / historical debugging feature that you've heard about in the news. Hopefully you can all make it to that to get a sense of where this is at and where we are going with it.
I'll also be hanging around the lounge and HOL areas, so drop by and say hi.
See you all there,
John
It's nice to at last be able to publicly discuss some of the features we've been working on recently. Brian Harry's blog entry here provides the best overall view of what our focus for this release is, and what features are delivered as part of that. Our team has been working on two of the three features listed here, specifically "Eliminate No-Repro Bugs" and "Identify the Test Impact".
Over the coming weeks, leading up to PDC, you'll see a series of blog posts from me discussing these areas, as well as the work we have coming on line for both the profiler and code coverage. We also are working on new features in the area of core debugging.
Stay tuned,
John
I was in a colleague's office the other day and ran into the now ubiquitous issue that people still buy or install the "Architect Edition" thinking it has everything. Although this was true in previous versions of VS, this changed with VSTS 2005 when role based editions came into being. If you need the box that has everything, you need the Team Suite edition. The Architect and Developer editions are now siblings rather than parent child (since I call myself a dev makes me feel better :)). For a more detailed breakdown of what features are in what box check out this link:
http://msdn.microsoft.com/en-us/vsts2008/products/bb933734.aspx
Cheers,
John
It's been a while, but the time has come again for a Microsoft Professional Developer's Conference. I'm aiming to go to the show, I really enjoyed my time at PDC05. I look forward to meeting as many of you - the MS developer community - as I can. I'm also looking forward to Ray's keynote. Our team will be participating in one of the key sessions that have been announced so far:
VSTS: Software diagnostics and quality for services
In this session we will present processes and tools from the upcoming Visual Studio Team System Codename "Rosario" release and Microsoft Research and show how we deliver on quality, scalability, and experience goals for the new class of applications that demand rich UI, service consumption, and frequent release.
So - I'm looking forward to that and just getting together and finding out what you like and don't like about our tools today, as well as sharing some of the "where we are going" with upcoming releases. For more details visit http://www.microsoftpdc.com/
Hope to see you there in October,
John
As promised it is time to follow up with the PM jobs from our debugger expansion (Test to follow soon, promise!). There are three main areas we are hiring into the debugger for:
Native C/C++ debugging PM
As mentioned in my previous posting, this is an area that will see us considerably staff up. We're looking for someone who can connect with this customer segment, and drive new experiences in this space. See more details for this job here.
Debugger strategy / partner relationship PM
This post is for a senior PM who will drive our relationship with external partners such as Intel, AMD and people extending the debugger in VS, as well as our internal platform partners like Windows and the CLR. This person will be excellent at building relationships and working with others. They will be responsible for developing the next generation of debugger extensibility APIs. See the details here.
HPC debugging/diagnostics PM
This post is for a senior PM who can quickly come up to speed on the HPC customer segment, understand their needs in the debugging and diagnostics space, and start to build a product and business plan around tools in this space. This person will be customer focused, and be comfortable presenting product and funding plans to senior management. Ideally, you'd be able to code some prototypes too. Get the skinny here.
Recently our profiler/code coverage PM headed to pasture's new, so if you are interested in this area, look at the job description here.
Again, if you do end up applying, feel free to mention the blog, or contact me through the blog if you want more information or details,
Cheers now,
John
