Sometimes you need to debug a process, and you need to attach the debugger right away, but you cannot launch the process under the debugger. For example, if the process you need to debug is a Windows Service, the Windows Service Manager must launch the process. How can you debug it?
Here are the two easiest solutions:
#1: Windows includes registry settings under the ‘Image File Execution Options’ key that can help you debug this. I blogged about this before back in February of 2005 (http://blogs.msdn.com/greggm/archive/2005/02/21/377663.aspx). Most of the information is that blog is still correct. A few updates worth mentioning though:
- Visual Studio 2008 can support session 0 processes on Vista or Server 2008.
- Visual Studio 2008 can support 64-bit operating systems. The only remaining 64-bit restriction is that 64-bit managed processes cannot be debugged this way since the .NET Framework does not support managed+native debugging of 64-bit processes.
- Visual Studio 2005 does not support this feature on Windows Vista or Server 2008. You will need to upgrade to 2008 for these operating systems.
#2: You can add code to your app to cause it to pause waiting for a debugger to attach. I am attaching an example header file for making this work. We ship several executables that embed similar code for this reason. Just call ‘RuntimeDiagnostics::CheckPauseOnStartupOption()’ as the first line of code in your main routine. You can do something verify similar for managed code as well. Just call ‘System.Diagnostics.Debugger.IsAttached’ instead of IsDebuggerPresent.
int _tmain(int argc, _TCHAR* argv[])
{
RuntimeDiagnostics::CheckPauseOnStartupOption();
return 0;
}
Then set this registy key if you need your app to pause:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\sample.exe]
"PauseOnStartup"=dword:00000001
Today’s blog is about writing your own script debugger.
Visual Studio has provided ‘Script’ debugging for a long time. In this context, ‘Script’ refers to script run by jscript.dll or vbscript.dll such as the script code in Internet Explorer, classic ASP scripting, or the cscript/wscript programs. Long long ago, someone provided a sample for creating your own script debugging utilizing the active script debugging interfaces (ex: IRemoteDebugApplication).
However, this sample relied on a Microsoft executable called mdm.exe (IMachineDebugManager/CLSID_MachineDebugManager). After Visual Studio 2003, Visual Studio stopped shipping mdm.exe. So today, I wanted to provide a brief summary of how one could port an mdm.exe-based script debugger to the newer interfaces. Mdm.exe provided essentially two related features -- it allowed a debugger determine what script applications where currently running, and there was an eventing interface that let a debugger be informed when a new script application started. These features worked because the debugger component inside of the target process (pdm.dll) would connect to mdm.exe whenever a new script application was created.
Today, these features have been moved to pdm.dll. So now, pdm.dll is loaded in both the debugger and in the target process. In the target process pdm.dll still continues to do the same work as it has always done. However, now one can also load pdm.dll into the debugger process to obtain the old MDM features.
PDM now has a new CLSID for the work that it does in the debugger process – CLSID_MsProgramProvider. It exposes an AD7 interface for obtaining this information: IDebugProgramProvider2.
There is sample code attached to this blog which demonstrates how to launch or attach to a program using these new interfaces.
Background
Sometimes Visual Studio crashes. It would be nice if this wasn’t the case, but unfortunately it happens. However, like many Microsoft applications, Visual Studio has Error Reporting so that when a problem happens, you can tell us about it in one click. However, sometimes you would like to actually tell a live human about your crash. There are a number of advantages to doing so:
- A human might be able to tell you about a work around that Windows Error reporting doesn’t know about.
- A human can tell you if this crash was something that we could diagnose.
- It helps to make sure your issue is really fixed. When a developer at Microsoft looks at a Windows Error report, all we see is where the crash happened and some local variables. Sometimes we can figure out what was happening and create a repro to confirm our fix. This is the best case, but it often doesn’t happen this way. More often we need to either guess what a fix is but not test the fix, or we are unable to even guess what a correct fix might be.
For these reasons you might want to also report the issue through the Visual Studio product feedback center.
What to do
Step #1: Go to http://connect.microsoft.com/site/sitehome.aspx?SiteID=210 and enter a bug.
Step #2: In the bug, include the ‘bucketing information’. You can find this information by:
- Open the event viewer (Right click on Computer and go to ‘Manage’; click on the ‘Event Viewer’ tree item).
- Open the ‘Application’ section
- Look for the entry that happened about the correct time and had the source set to ‘Application Error’
- Copy the content and paste it into the bug, it should look something like:
Faulting application devenv.exe, version 9.0.30428.1, time stamp 0x4815597f, faulting module scriptle2.dll, version 9.0.21022.8, time stamp 0x47317e18, exception code 0xc0000005, fault offset 0x0000328c, process id 0xfbc, application start time 0x01c8bc39887cac69.
In case you missed it, Visual Studio 2008 SP1 Beta has been released:
http://www.microsoft.com/downloads/details.aspx?familyid=CF99C752-1391-4BC3-BABC-86BC0B9E8E5A&displaylang=en
For the debugger, Visual Studio 2008 SP1 contains:
- As with all service packs, fixes for all the significant bugs that we found since Visual Studio 2008 RTM.
- Improved support for downloading symbols and PDBs for the .NET Framework and other Microsoft products.
- Improvements for using Visual Studio with non-local symbol stores (such as the Microsoft public symbol servers)
- Step into specific for managed. We have had this feature with native for years, with SP1 we added this support for managed as well. If a line contains multiple function calls, you can right click on the line in the editor and select which function you would like to step into.
- Step filtering for managed. There is now an option to stop stepping into property get routines.
We want to try and get the RTM version into customer’s hands as soon as we can, so this means there isn’t too long for us to act on customer feedback. So if you are interested, give it a whirl soon.
One very important note: If you have installed any Visual Studio 2008 hot fix, make sure that you run this tool _before_ installing Visual Studio 2008 SP1 Beta:
http://code.msdn.microsoft.com/RemoveKB944899
If you would rather not run a tool, manual steps can be found here:
http://blogs.msdn.com/heaths/archive/2008/05/16/kb944899-should-be-removed-before-installing-visual-studio-2008-sp1.aspx
I was asked today -- Why does the Visual Studio Remote Debugger Service need to be run as an administrator?
Since it doesn't appear that this information is documented, I figured I would provide an answer. Running the remote debugger service as an administrator is really a recommendation rather than a requirement. The actual requirement is:
- Account must have the 'logon as service' privilege
- Account must be able to connect 'backwards' to the Visual Studio computer over the network. For this reason, on a domain, its easiest if the service is running under Local System, Network Service, or a domain account. If you want to run it as a local account see: http://blogs.msdn.com/greggm/archive/2004/10/04/237519.aspx
- Account must have rights to debug the target process. This means the service needs to either run under the same account as the process to be debugged, or the service needs to run as an administrator.
We recommend running the service as an administrator because:
- The service only allows connections from administrators, so there is no security reason why running it as an administrator is bad.
- Running it without administrative privileges may put the user in a situation where the Visual Studio user has access to debug a particular process, but since the remote debugging service lacks these rights the Visual Studio user is not allowed to debug the process.
- It’s a lot easier to setup this way, and there is enough that can go wrong with remote debugging already.
Recently I have found the need to have an exception filter in some C# code that I have written. Since there is no language support for this, I wrote an IL instrumentation tool:
http://code.msdn.microsoft.com/ExceptionFilterInjct
For debugging scenarios, the value of this tool is to make it easier to save a minidump of exceptions. Unfortunately, there is not yet a rich Visual Studio experience for managed minidumps, so you will need to use sos.dll (see http://msdn2.microsoft.com/en-us/magazine/cc164138.aspx).
I have been spending some time recently creating a custom code generator that outputs C++ and C# code from a custom XML format. This blog is about some of what I have learned while working on this.
Advice #1: Use '#line'
There is a really cheap way to get a debugging experience for your custom language: #line. With #line, you get to change the debug info that ilasm/csc/vbc/cl writes into the PDB so that the debug info points back to the original source file instead of the one that your code generator creates. This is quite handy if you custom language has at least parts where there is a strong connection between the generated code and the custom language. Notes:
- There is not a similar technique to allow you to re-associate variables, so I would advise using the original names of variables if possible.
- If a line of the original source expands to multiple lines of the generated code, just repeat the #line before each line of the generated code.
- There is an issue with this technique if your source language is XML-based in VS 2008 RTM. Visual Studio hopes to correct this problem in SP1.
- In C#, you can use this technique with #pragma checksum to set the checksum of the source file. This is useful if you expect to have a bunch of source files with the same name (ex: default.aspx) or if you are already grabbing the checksum for some other reason. Otherwise, it is probably overkill for a custom code generator. Note that the GUID with #pragma checksum identifies the hash algorithm (ex: MD5 is { 406ea660-64cf-4c82-b6f0-42d48172a799}).
Example:
Generated file:
static void Main(string[] args)
{
#line 1 "HelloWorld.ExampleLanguage"
Console.WriteLine("Hello World");
#line default
}
HelloWorld.ExampleLanguage (custom language source file):
Hello World
Advise #2: Use XML
XML is a great way to do a custom language these days because at no cost, you get a lexer, a parser, syntax validation, a language sevice, and your compiler gets to work with deserialized classes instead of with text. Here is the procedure that I would recommend:
- Figure out what you would like your language to look like by writing a bunch of examples.
- Run xsd.exe over your examples to create the start of a schema.
- Open up the generated schema and start making changes. You might need to make changes because the generated schema wasn’t specific enough (ex: you have an enumerator that it has represented as a string), or because your examples had bugs in it, or because you want to add documentation.
- Once you have the schema vaguely correct, it is time to use xsd.exe to create class files from your schema. This allows you to use XmlSerializer.Deserialize to create classes from the input XML with little work. You want to create a build step to do this as you will be changing your schema often.
- Hookup the schema file so that you can run it as part of compilation. This provides you with a pretty good set of validation without effort. I did this by embedding the schema as a resource in my compiler, but obviously you could also leave it as a file that your compiler opens.
- (Optional) Use sgen.exe to generate the serialization assembly -- XmlSerializer depends on a generated assembly to perform the serialization/deserialization. By default, this assembly is generated dynamically, but you can also use sgen.exe to generate this assembly up front.
- When you edit your language, edit in Visual Studio and make sure that the XML editor has your schema open. The XML editor will pick up schema items that are in your project, and it will also pick up schemas that are in the Visual Studio 'schema directory'.
Example target for running xsd.exe:
<!--Generate Example.cs using xsd.exe -->
<Target Name="GenerateXSDClasses"
Inputs="Example.xsd"
Outputs="$(IntermediateOutputPath)\Example.cs">
<Exec Command="$(RunManagedToolPath) xsd.exe Example.xsd /classes /fields /namespace:ExampleCompiler /out:$(IntermediateOutputPath)"/>
</Target>
<ItemGroup>
<Compile Include="$(IntermediateOutputPath)\Example.cs"/>
</ItemGroup>
You would also need to wire the target into a property group that runs before compiling
Example of embedding the schema as a resource:
<EmbeddedResource Include="Example.xsd" />
Example of using the schema:
public static void InitializeSchema()
{
if (s_schemaSet != null)
{
throw new InvalidOperationException();
}
System.Reflection.Assembly ThisAssembly = typeof(MyType).Assembly;
Stream stream = ThisAssembly.GetManifestResourceStream("ExampleCompiler.Example.xsd");
XmlReader schemaDocument = XmlReader.Create(stream);
s_schemaSet = new System.Xml.Schema.XmlSchemaSet();
s_schemaSet.Add("http://schemas.microsoft.com/vstudio/Example/2008", schemaDocument);
s_schemaSet.Compile();
}
XmlReaderSettings settings = new XmlReaderSettings();
settings.ConformanceLevel = ConformanceLevel.Document;
settings.Schemas = s_schemaSet;
settings.ValidationEventHandler += MyValidationEventHandler;
settings.ValidationType = ValidationType.Schema;
using (XmlReader reader = XmlReader.Create(filename, settings))
Advise #3: Check in the custom compiler output
If you are writing a very high quality compiler that you are trying to productize then, when people use your compiler, you would have them wire it into their build process such that they would input your custom language and get back a dll or exe.
But this is not necessarily the correct bar for a custom code generator. In my case, I am creating a truly custom compiler. Very few people are going to author the input language, and it doesn’t make sense to expend valuable QA resources directly testing the compiler (rather they would test the generated code). So rather than taking the output of my compiler and directly building these files, I instead checkin the compiler output as a baseline, and the build process runs the custom compiler and compares the output to the baseline. If they differ, it issues a build error.
There are a number of valuable properties that I get out of this 'baseline' approach:
- If I edit the compiler, testing the compiler becomes very simple. I just do a build and see if I got the same result that I expected.
- If I edit the input, I get confirmation that the compiler did what I expected. I need to diff the baseline output against the new output and validate that I got expected changes.
- Compiler bugs turn into build breaks. If I had a bug in my custom compiler where the output was incorrect, rather than going silently unnoticed instead the bug turns into a build break.
I recently stumbled upon a blog that explained how to use the Source Server technology to enable the debugger to automatically download source files from TFS. This is useful when you work on a software project that is large enough that you debug against modules that are built by your build lab instead of being built locally.
Here are instructions for setting up Team Build: http://jelle.druyts.net/2007/12/09/SettingUpSourceServerForTFSBuilds.aspx
After doing this, you will also need to configure the debugger. John Robins has instructions for doing that in MSDN Magazine. Go down to the 'Debugging in Visual Studio 2005' section.
The Visual Studio Debugger team is expanding, and is looking for candidates. If you are interested in applying for a position, we have openings in Dev, Test and PM.
Some great things about working on the debugger:
- You are helping to improve a tool which millions of developers use every day. There are few software development tools which impact more people.
- You get to use what you sell. There aren’t too many places where you can work on software that you also get to use. But you will have this experience on the debugger team.
- You get to play with lots of cool technology. People are always inventing new platforms, and they are always going to need debugged.
- Great team -- you will get to work with a team of talented and fun people. You will also get to work with me :)
One of the new features that we added in Visual Studio 2008 was debugger enhancement for Windows Communication Foundation (WCF) services.
There are four features that we support:
- You can step from a service client into the implementation. Just step into ('F11') a web service proxy and the debugger will automatically go and attach to the server process and stop when execution reaches the service implementation.
- You can step out of the implementation of a service back into the client.
- You can get a logical call stack which shows the client and server (see below).
- The debugger can automatically attach to services in your solution whenever the debugger sees a service request going to the service. This allows you to hit breakpoints in your service without having to go and explicitly stepping into it.
There is more information about these features in our documentation: http://msdn2.microsoft.com/en-us/library/bb514135.aspx. The first three of these features also exist for ASP.NET web services and have been included in Visual Studio for several releases. The fourth feature is new.
There is one troubleshooting item that I wanted to mention because it is not included in our documentation. If you are finding that none of these features seem to be working, it is possible that something has gone wrong with the installation and that the necessary machine.config changes did not happen. To resolve this, we shipped a tool to manually install/uninstall the necessary changes:
- Start a command prompt (Start->Run, cmd.exe). Note that this will get you a 64-bit command prompt on a 64-bit OS, which is what you want. If you are running on Vista, you need to start this command prompt as an Administrator.
- Change directory to the root of your VS installation (if you installed to the default location: cd /d "%ProgramFiles%\Microsoft Visual Studio 9.0\Common7\IDE")
- Run: vsdiag_regwcf.exe -i
You could also use the '-u' flag if you some reason you want to disable this integration.
Just-In-Time debugging is trickier on Windows Vista than on previous operating systems. Today I am going to talk about the various knobs that you can use to make this better.
Hint #1: Use Visual Studio 2008 Beta2. I expect Beta2 to be out soon. If you are developing on Windows Vista, I definitely recommend trying Beta2 over previous versions of Visual Studio. While it is still a beta, Visual Studio 2008 greatly improves the Vista experience (at least for the debugger). As far as Just-In-Time debugging goes, the fact that you can't debug elevated processes was a big problem for Visual Studio 2005.
Hint #2: If you get a Windows Error reporting dialog for your application and there is no 'Debug' button, then click 'Send Report' and then you will get a second dialog with a debug button.
Hint #3: Windows Error reporting has several configuration settings. If you are finding that Windows Error Reporting is preventing you from debugging your applications, there are a few settings that you can tweak.
First, you can define the 'AeDebug\Auto' string:
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug]
"Auto"="1"
Note that this is a String value, NOT a DWORD.
If this doesn't work, Error reporting can also be disabled at various levels through control panel:
- Start->Control Panel
- Click 'System and Maintenance'
- Click 'Problem Reports and Solutions'
- Click the 'Change Settings' link on the left side
- Click the 'Advanced settings' link
- Configure at will
Hope this helps.
[Updated on 5/23/2008] In Vista SP1 and Windows Server 2008, there is now a registry key to save the generated .dmp file: http://msdn.microsoft.com/en-us/library/bb787181(VS.85).aspx
If you’re a software developer, chances are that you have written an application, and this application has crashed. When this happened, it probably put up a dialog that looks something like this:
How do you figure out what went wrong?
Strategy #1: Extract the minidump from Windows Error Reporting
Windows Error reporting has already created a minidump of the crash. So one way to find out what went wrong is just to look at the minidump. This works really well if your application is written in native code and there is no debugger on the machine where the crash occurs. You can use it for managed code too, but you will need to use sos.dll to analyze the dump (see MSDN). Open a command prompt (Start->Run, cmd.exe), and switch to your temp directory:
C:\Documents and Settings\greggm>cd /d %tmp%
Look for the dump file that Windows Error Reporting produced. It will have a '.dmp' or '.mdmp' extension and the date should be shortly after the crash happened:
C:\DOCUME~1\greggm\LOCALS~1\Temp>dir *.*dmp
Volume in drive C has no label.
Volume Serial Number is 70E3-6676
Directory of C:\DOCUME~1\greggm\LOCALS~1\Temp
05/24/2007 08:54 AM 109,570 4A88835.dmp
1 File(s) 109,570 bytes
0 Dir(s) 25,379,823,616 bytes free
Mark the file as read-only. This will prevent Windows Error Reporting from deleting the file after you dismiss the dialog:
C:\DOCUME~1\greggm\LOCALS~1\Temp>attrib +r 4A88835.dmp
Now dismiss windows error reporting and copy the dump file wherever you want. You can analyze the dump in Visual Studio by opening the dump file as a project (File->Open Project), and start debugging (F5).
Strategy #2: Access the dump from Online Crash Analysis
Microsoft has a program to allow ISVs to access the crashes in their applications that have been submitted by users. If the person experiencing the crash is a customer, this is a great way to find out what happened. See the winqual site for more information.
Strategy #3: Debug the crash through Just-In-Time Debugging
Probably everyone already knows about this, so I won’t spend much time discussing it. However, as long as the application is crashing on a computer that has a debugger installed, this is the easiest option. The only thing tricky that I will mention is that in Windows Vista, depending on your computers Windows Error Reporting settings, you might actually need to click the 'Send information' button before Windows Vista will present you with a second dialog that allows you to debug.
Last week I had this displeasure of tracking down a fairly unpleasant reference counting bug. I wound up having to solve the problem by brute force:
- Set a breakpoint in the constructor of leaking object
- Set a data breakpoint on the object's reference count variable
- Change the data breakpoint to a tracepoint that prints the callstack (see dialog)
- Match up the callstacks.
However, my tracepoint was hit so often that matching up callstacks by hand proved to be impossible. To help, I wrote a little tool that I decided to post. The tool does some basic analysis of the tracepoint output - compute a database of all callstacks with their hit count, and compute a calltree with hit count. While matching up the callstacks was still not easy, it did help.
Basically, the tool gave me two things:
- In places where an AddRef and Release where matched with the same callstack, I could bucket all of these callstacks together and eliminate them quickly
- If CFoo::CFoo called AddRef and CFoo::~CFoo called release, I could easily filter out the callstacks that contained neither CFoo::CFoo or CFoo::~CFoo. If the resulting callstacks balanced out (just as many CFoo::CFoo as CFoo::~CFoo) then I could throw out all these callstacks quickly.
The code for my tool is in the attached ‘TracepointAnalysis.cs’ file. Here is some example code to call it:
static void Main(string[] args)
{
List<Callstack> stacks;
List<CallTreeNode> callTreeRoots;
CallstackFilter filter = delegate(CallstackKey key) {
return key.Contains("CFoo::CFoo") || key.Contains("CFoo::~CFoo");
};
TracepointAnalysis.ReadFile(@"c:\log.txt", filter, out stacks,
out callTreeRoots);
DumpStacks(stacks);
DumpCallTree(callTreeRoots);
}
private static void DumpCallTree(List<CallTreeNode> roots)
{
foreach (CallTreeNode root in roots)
{
root.Dump(0);
}
}
private static void DumpStacks(List<Callstack> list)
{
// Sort the stacks by their hit count
Comparison<Callstack> comparison = delegate(Callstack a, Callstack b)
{
return a.HitCount - b.HitCount;
};
list.Sort(comparison);
foreach (Callstack item in list)
{
item.Dump();
}
}
In native code, its sometimes useful to set a breakpoint condition so that a breakpoint will only stop for a particular instance of an object. To do this, I simply use the address of the object:
this == (CMyObject*)0x10fc10
In managed code, the garbage collector moves objects around, so even if you were writing in a language that had great support for pointers, setting a conditional breakpoint this way is probably useless. Instead, you can do this using object ids.
Step #1: Make an Object ID. To do this, get the object that you want to use in your condition into the watch window. Then right click on this object and invoke the ‘Make Object ID’ context menu. The watch window output will now include ‘{1#}’:
+ this {cswin.Form1, Text: Form1} {1#} cswin.Form1
The ‘1#’ indicates that this is the first object ID. Each object ID created will get a new object ID (2#, 3#, etc). There is more documentation under 'Object Identity' on MSDN.
Step #2: Set conditional breakpoint. Right click on your breakpoint and click ‘Condition…’, and set it to what you want. For example:
this == 1#
Happy debugging.
I have been spending some time looking at native memory leaks recently, and I decided to blog about some of the techniques which worked well for me.
First, find out what objects are leaking. If your code doesn’t already have some sort of leak detection scheme, you can take advantage of the fact that the operating system’s heap does. KB 268343 explains how to use UMDH to get a text file of all the leaking callstacks. Keep in mind that to get useful output, _NT_SYMBOL_PATH must point to symbols for both the operating system and your application.
So now you know which objects are leaking, but for ref-counted objects, an allocation callstack probably doesn’t get you to a solution. You need to find the missing ‘Release’ call.
Here is what I did that worked pretty well:
Step #1: Get a good repro. Without a repro, you don’t have a chance for leak bugs.
Step #2: Look at all the reported leaks and find a leaf object that is leaked. Often times leaks chain where one object is leaked, but that object contains a reference to another object. So the other object must also leak. Before spending ages looking at all the AddRef/Release calls to a leaking object, make sure that you are looking at a leaf object. For an example, if all 'apple' objects contain references to their 'tree' objects and trees and apples are leaking, look at the apple leaks instead of the tree leaks.
Step #3: Find an instance of a particular object that leaks. Set a tracepoint on the destructor and contructor of the leaking class. On the constructor, I would recommend a When Hit message like ‘New Object: {(void*)this} $CALLSTACK’. On the destructor, try a When Hit message of ‘Delete Object: {(void*)this}’. Run your scenario and do some time text processing to find out which callstacks leaked and didn’t. In batch script:
for /f "tokens=3" %d in ('findstr /c:"New Object" debug.txt') do findstr %d debug.txt
Where debug.txt is a text file that I saved the content of the output window to. Now I can look at all the allocation callstacks for the object – both those that leaked and those that didn’t. You will need to find some pattern that will allow you to predict one leaking object instance. If your lucky, the pattern is something simple like ‘the first instance created’.
Step #4: Trace all the AddRef/Release calls. For this I would recommend setting a data tracepoint on the field that holds the ref count. To do this: Debug->New Breakpoint->New Databreakpoint and set the location to the address where the ref count is stored (ex: 0x10E89934). Then change the ‘When Hit…’ property to output a callstack and the current ref count -- ‘{*((DWORD*)0x10E89934) } $CALLSTACK’.
When your done, hopefully you have a reasonable list of AddRef/Release calls to your object. So its easy enough to match them up and see what went wrong.
