It's been a while, but the time has come again for a Microsoft Professional Developer's Conference. I'm aiming to go to the show, I really enjoyed my time at PDC05. I look forward to meeting as many of you - the MS developer community - as I can. I'm also looking forward to Ray's keynote. Our team will be participating in one of the key sessions that have been announced so far:
VSTS: Software diagnostics and quality for services
In this session we will present processes and tools from the upcoming Visual Studio Team System Codename "Rosario" release and Microsoft Research and show how we deliver on quality, scalability, and experience goals for the new class of applications that demand rich UI, service consumption, and frequent release.
So - I'm looking forward to that and just getting together and finding out what you like and don't like about our tools today, as well as sharing some of the "where we are going" with upcoming releases. For more details visit http://www.microsoftpdc.com/
Hope to see you there in October,
John
As promised it is time to follow up with the PM jobs from our debugger expansion (Test to follow soon, promise!). There are three main areas we are hiring into the debugger for:
Native C/C++ debugging PM
As mentioned in my previous posting, this is an area that will see us considerably staff up. We're looking for someone who can connect with this customer segment, and drive new experiences in this space. See more details for this job here.
Debugger strategy / partner relationship PM
This post is for a senior PM who will drive our relationship with external partners such as Intel, AMD and people extending the debugger in VS, as well as our internal platform partners like Windows and the CLR. This person will be excellent at building relationships and working with others. They will be responsible for developing the next generation of debugger extensibility APIs. See the details here.
HPC debugging/diagnostics PM
This post is for a senior PM who can quickly come up to speed on the HPC customer segment, understand their needs in the debugging and diagnostics space, and start to build a product and business plan around tools in this space. This person will be customer focused, and be comfortable presenting product and funding plans to senior management. Ideally, you'd be able to code some prototypes too. Get the skinny here.
Recently our profiler/code coverage PM headed to pasture's new, so if you are interested in this area, look at the job description here.
Again, if you do end up applying, feel free to mention the blog, or contact me through the blog if you want more information or details,
Cheers now,
John
We've been working for a while now on a plan to re-invest in one of Visual Studio's core strengths - the debugger.
I'm very proud to be associated with one of the pieces of VS that is used by every developer almost every day, from the folks who actually use the debugger as a creation tool in a RAD mode, like the VB users, to those debugging complex distributed problems in the enterprise, to native C++ developers trying to figure out some crashdump from their application in the field.
Needless to say, the debugger is impacted by the wealth of platforms that Microsoft brings to the developer community, and that means we've not moved the debugging experience as far forward as we would have liked. So down to business - we're hiring. We have positions available in Dev, PM and Test. I've broken down the dev jobs by the major areas we are trying to invest in:
Native C/C++ debugging
We're looking for someone to lead the charge and take the great native debugging experience we have today to new heights. You would work on everything from multi-threaded and crashdump debugging, to stronger support for inter-op and extensibility in our debugger. We're going to be staffing up in this area, so this is the first position of more to come. Read more details here.
Silverlight/WPF debugging
WPF and by extension Silverlight come with their own particular challenges. We're looking for someone to build new debugger features to specifically help customers develop the next wave of rich internet and client software. Find out more about this position here.
AJAX/Asp.Net/Script/WCF debugging
In VS2008 we made some great advances in our script debugging story (better data visualizations, breakpoint mapping from asmx to client script), but the job is still not complete. This particular position has a very large scope, touching on a lot of platforms, so we're looking for a pretty senior dev. Get the full scoop here.
Debugger user experience (aka GUI!)
The debugger has one of the largest user experience surface areas in all of Visual Studio. Choices made in the debugging experience could literally save developers thousands of hours in their daily lives. Want to have that kind of impact? Love building great experiences? Check out the details here.
I'll be following up in the next few days with links to our positions in Test and PM. We're a very collaboration oriented team, encouraging people to work across disciplines and with peers to come up with the best solutions. The team's really excited to see us expand in this way. We know the folks who can help us define the next wave of debugging support are out there, and we look forward to working with you!
If you do end up applying, feel free to mention you saw the positions on joc's blog.
Cheers now,
John
I'll be at EclipseCon next week, but just the Wednesday and Thursday. If you read the blog and want to meet up or have some questions you'd like answered, leave me a note at the Microsoft booth that we are sharing with the folks from TeamPrise (http://www.teamprise.com/).
Happy bug slaying,
John
I've seen many folks in our ISV community use the _NT_SYMBOL_PATH environment variable to set the path for symbols loading in VS. They often set this with symbol server included. Due to the current architecture, the environment variable is actually consumed at a pretty low level, so the debugger UI does not have any control in switching off paths etc. My suggestion for those using primarily VS is not to use _NT_SYMBOL_PATH, but instead to use the Tools->Options->Debugger->Symbols dialog.
This dialog affords you much tighter control over when and where you are loading symbols. In large scale apps, my suggestion is to leave entries in the symbol list (Microsoft symbol servers, your own companies symbol shares or symbol servers), but to check the box that says "Search the above locations only when symbols are loaded manually". Although this is a little wordy it is accurate. With this box checked, the debugger will continue to load symbol files that are either exactly where they were built, or the matching symbol file is right next to the exe or dll. It's then a trivial matter to go to the callstack window and load symbol files for when you need them (like kernel32.dll and ntdll.dll are on the stack). With a trimmed down set of symbols loaded, you will also find things like setting breakpoints are considerabley faster, and Devenv's memory use is down.
In coming releases, we are going to focus on continuous improvements in this area, as symbol loading is a perf and usability concern for us. You'll see the first changes in the Service Pack for VS2008. Any feedback you have on what you'd like to see in this area, please let me know with a comment here.
Happy bug slaying,
John
Jackson on our team recently posted a sample on building a debug engine. In our architecture, the debug engine is the part that does all of the communication with the debugging/diagnostic APIs for a given platform, and communicates those back to the UI through a common set of interfaces. The debugger ships debug engines for managed code, native code, script, native devices and t-sql in the box. The debugger engine extension point gives people writing new platforms the ability to provide an excellent VS debugging experience.
Take a look at the sample here:
http://code.msdn.microsoft.com/debugenginesample
John
Happy New Year! Thought I'd start the new year with some more off-road fun on how to interact with the diagnostic components in VS.
People have often asked me "what is the format for a VSP file John". The answer is "it depends". That sounds a bit glib, but realistically, the VSP is too raw for most people to get any value out of. It contains the blobs of data that we have read at profile time, and varies in format depending on profile type used. It has not done any analysis of what the data means and does not contain any symbolic information (i.e. no names of functions or classes). To make sense of it, you'd really have to duplicate our analysis engine, which seems kinda pointless. The analysis basically has two major outputs:
1) An ADO.Net dataset with the aggregate tables like function, module, type
2) A calltree that gets represented in the call tree view
In future we hope to build a real extensibility and automation model that is fully supported, but for the moment let me mirror what I did with code coverage in previous articles but in a much shorter way. Note, this is using VS2008 bits. This was not available in VS2005, we essentially re-architected the engine in VS2008 to set us up to provide richer extensibility. Here's a code snippet to read a VSP and print the methods from the method table.
using System;
using System.Collections.Generic;
using System.Text;
// There are actually 4 references you will need to add to
// your project that includes this code :
// VSPerfPresentation, VSPerfAnalysis, VSPerfData, VSPerfReader
using VSPerfPresentation;
using VSPerfAnalysis;
using VSPerfReader;
namespace VSPMethodDump
{
class VSPMethodDump
{
string _filename;
ProfileDataProvider _pdp;
// A simple method dumper program that dumps profiler data for all the
// methods/functions in a VSP file
static void Main(string[] args)
{
VSPMethodDump vmd = new VSPMethodDump(args[0]);
vmd.Dump();
}
VSPMethodDump(string filename)
{
_filename = filename;
// The ProfileDataProvider class is the class that allows
// reading and analysis of profiler data in VSP files
// First parameter is a reghive for when PDP is working inside VS
ProfileDataProvider.Initialize(null);
_pdp = new ProfileDataProvider();
}
void Dump()
{
// The option store gives a variety of ways for analysis
// to be performed on the target file. We'll keep it to a simple
// Function summary
VSPerfAnalysis.OptionStore ops = new VSPerfAnalysis.OptionStore();
ops.Clear();
ops.FunctionSummary = true;
// Noise reduction is the folding you see in the VS2008 profiler
_pdp.EnableNoiseReduction = false;
// Read the file and anlayze. There are certain operations that can
// be performed after the open step, which is basically a read of the index
// of the file
_pdp.OpenFile(_filename, ops);
_pdp.Analyze();
// A ProfileDataViewer is a class that allows access to the analyzed data
// in this case the Function summary, which is the same as the Function view
// in VS2008. They are pre-populated with a set of default visible columns
ProfileDataViewer pdv = _pdp.GetViewer("Function");
WriteHeaders(pdv);
// A flat provider mimics the flat table view as seen in the profiler. Others
// are available when the view is more tree like, e.g. CallTree
IFlatProfileViewer flat = pdv as IFlatProfileViewer;
// Each IProfileRow has the data for a given method/function in this example
IList<IProfileRow> rows = flat.GetRows();
foreach (IProfileRow row in rows)
{
WriteRow(row,pdv);
}
// Close is needed to ensure proper closure of symbol files and files
_pdp.Close();
}
void WriteHeaders(ProfileDataViewer viewer)
{
// As well as visible columns you can inspect AvailableColumns to see
// what could be written here. If you want to write said columns, they
// need to be turned visible by updating the "Visible" property
for (int i = 0; i < viewer.VisibleColumns.Count; i++)
{
string colString = viewer.VisibleColumns[i].MediumName;
Console.Write(colString + ",");
}
Console.WriteLine();
}
void WriteRow(IProfileRow row, ProfileDataViewer viewer)
{
for (int i = 0; i < viewer.VisibleColumns.Count; i++)
{
string colString = viewer.GetText(row, i);
Console.Write(colString + ",");
}
Console.WriteLine();
}
}
}
Sadly I'm not good enough at blogging to have this as an attachment. Next time out on the ramps, I'll show some other areas of the API that can provide more insight.
Enjoy!
John
A question that is common from folks in the field is "who's using the profiler at Microsoft". Recently we went on a tour of alot of the groups within Microsoft to understand how they are using profiling tools and to see what we could do in the VSTS profiler to help them better. It was really great validation to hear that most groups were already active users of the profiler. Many of them were new to us, and they had been happily using the profiler for years now. We have a pretty close relationship with Exchange and SQL, who use the profiler on a very regular basis to understand perf regressions and highlight areas for improvement. We also work closely with the Windows Server folks on their needs for profiling, where we provide a large part of the solution, but there are some needs (such as kernel mode) that we don't cover. What was surprising to us is that many of the groups in MSN, including Live Search and the Ads group, for instance, were already using the profiler widely too, and getting real gains out of it. Recently we've started partnering with more teams in the Office org. So you see the reach is pretty broad.
Of course there were a few asks, but it was nice to see that many of those are covered by the improvements we've made in VSTS 2008.
For me, that's a big reason to be in Developer Tools. I love that idea that my peers' lives are made easier by tools that our team builds. A personal thing, but one that keeps me going when the death march is on :).
John
ScottGu just posted info on something our team has been working on in conjunction with Shawn Burke (NDP) and the Windows debugging team. When VS2008 releases, people will be able to debug into the .NEt framework libraries. We will be providing full source and symbols to the VS debugger through a web server.
I think this is a great stride forward for the diagnosability of solutions built on the .Net framework - and Shawn can take a bow as primary driver. See details at the following links
http://blogs.msdn.com/sburke/archive/2007/10/03/making-net-framework-source-available-to-developers.aspx
http://weblogs.asp.net/scottgu/archive/2007/10/03/releasing-the-source-code-for-the-net-framework-libraries.aspx
John
I thought I'd share a little bit of organizational info about the team I work in as a way to dust off this old blog. I'm trying to make sure I don't leave it as dusty as I have been prone to.
About a year and a half ago, I worked with a few folks in the VS organization to bring the debugger team and profiler team into one umbrella org we call "VS Diagnostics". The team is not a huge one and we cover a variety of efforts including all of the native and managed debuggers (x86, x64, IA64), T-SQL debugging, SQLCLR debugging, Script debugging, Code coverage (managed and native) and Profiling (managed, native, x64, x86).
Obviously that's a bunch of work for a team in the low 10s of people (I don't think I am stepping over the line there, but if Soma wanders into my office it could be that "I'm just going outside; I may be away some time."). We believe, and feel free to correct me if I am wrong, that powerful integrated debugging and diagnostic experiences are some of the reasons our customers love VS. But we think there is room for improvement. Essentially, the way we debug and diagnose issues has not radically improved in the last 15 years. Now look at the same period in software construction:
1) Libraries have moved on, there are many declarative based paradigms (think things like WPF), and a myriad more frameworks than plain old VBRun, Win32 and the CRT. These platforms and frameworks have become more complex and require a bunch of domain specific knowledge for people to understand them.
2) The support for automating construction has evolved with numerous wizards, designers and code samples
3) The code editing experience is now so much richer with intellisense and refactoring
Creating VS Diagnostics has given us the potential to drive forward on a number of fronts including:
1) Building a platform core that is flexible enough that framework builders can provide the right diagnostic experience for their framework
2) Building a much more extensible core of diagnostic technologies than we have done to date.
3) Developing new experiences and paradigms for how people solve problems with their software today.
We still take very seriously our need to support our existing customers, so we're working to ensure Orcas is a quality release, and understanding which features we can build that remove people's current dissatisfaction with the debugger and profiler. We're hard at work in all of these areas. We know that our work touches every developer, but it's that very touch that keeps us coming to work every day.
My job at the moment is Development Manager for this group, which I've done now for close to a year, and it has been some of my best time at MS. I'm really happy to see the stuff we got done for Orcas, and am looking forward to more in the releases beyond, including the upcoming Rosario. Stay tuned for further posts on Rosario as and when I can in the coming months.
Cheers,
John
I love Orcas. I would say that of course, but for me personally I've been involved in so much of the blood, sweat and tears (yes - tears) that went into our features (debugging, profiling, code coverage) that it's great to see it finally be broadly available in the recently announced Beta2.
Soma created a blog entry recently on our features - see it here: http://blogs.msdn.com/somasegar/archive/2007/09/14/debugging-and-profiling-features-in-vs-2008.aspx
I kinda miss the hunt the thread game mentioned in that article. My highscore was 23. That and the fact that I actually wrote a version of the hotpath feature for V1 that got abandoned :(. Being able to drill down fast in the calltree view to the major bottlenecks is something that needed almost as much hand-eye co-ordination as "Hunt the Thread" (I wonder if I can TM that game).
It's amazing the amount of time that we put into the "infrastructure" this time around too. Things like Vista UAC support, WCF support, WPF support, LINQ support and x64 profiling suck up a lot of time to "just keep features working", especially in an area such as diagnostics where you are that bit closer to the metal (platform). So please, give the tools a spin on all of these areas and technologies. Have fun.
John
For those of us that use ILDASM and Lutz's reflector tool often, "Friend of the VS Debugger" (I may try and make that something official) Andrei Faber has created a cool add-in that visualizes IL in the current context.
Go here for details http://debug-companion.com/IL_View.aspx
Thanks Andrei!
If anyone else knows of cool debugger add-ins, including visualizers and the like, please post comments,
John
Whoa - didn't realize I had let this blog get so moldy! Apologies.
Recently I've had a similar conversation with many folks in the Microsoft consulting and field sales organizations about the value of Team System to C++/Native Win32 developers, folks not developing on .Net today. As an ex VC++ team member, this is a subject close to my heart. The truth is there is a lot of value in the server product in terms of its source control, work item tracking and reporting. But beyond that, we've added a host of advanced development tools that support the C++/Native Win32 developers too. These are all in the Developer edition of the Team System family.
The Visual Studio 2005 Team Edition for Software Developers (whoa - gotta love that name from marketing - we talk about it as TeamDev) includes:
-
A Code profiler that works well on native code, it is based on profiling technology that has been used to ship SQL server and Windows in the past.
-
Code analysis tools for native C++ that have been widely adopted and are part of the "minimum shipping bar" for most native code coming out of Microsoft.
-
Unit test tools that can be targeted at native code (with a little work), and that can collect coverage data on native dlls.
-
Integration of applicaiton verifier, a runtime checker that checks for common Windows programming mistakes and some memory and security issues.
We think it is a compelling offering for folks developing with native code and C++.
Even if these tools are not compelling for you to step to the TeamDev box, Visual C++ 2005 is an absolutley excellent product for those folks still developing native code. The compiler and libraries have gone from strength to strength, and I can speak from personal experience that the editing/browsing and debugging features in this edition make it a natural upgrade for anyone using previous versions of C++ today. And I include VC++ 6.0 in that.
Hope this helps folks understand our unwavering commitment to great tools to make every one of your software projects successful.
J
When I was out talking to customers over the last couple of months I was surprised to hear so many of them ask "how do you do it?" in terms of our software processes, and especially with respect to the roll out of team system. Eric Lee, John Lawrence and I were discussing that and got together with the channel9 team and Jeff Beehler to discuss this very thing. See the clip here:
http://channel9.msdn.com/ShowPost.aspx?PostID=139114#139114
John
It's a tale of a wronged brother seeking vengance. A tale of intrigue, mystery and politics. Bolder and braver, it goes places the first installment feared to tread. Well actually it's none of those things, but it is the sequel to my article on using the team system code coverage collection mechanisms "Off-road". This installment can be read here:
http://blogs.msdn.com/ms_joc/articles/495996.aspx
John
