Creating NGEN PDBs for Profiling Reports

Creating NGEN PDBs for Profiling Reports

  • Comments 22

Update: Visual Studio 2013 will generate PDBs automatically when using the CPU Sampling or the CPU Usage tool in the Performance and Diagnostics hub. See this post for more details!

When profiling managed applications on Windows 8 any samples in an NGEN’d module will appear in the report as [NI module name] instead of showing the function name, which obviously makes it difficult to understand the report.

For example in the screenshot below you see [mscorlib.ni.dll], and I can see in the output window that I failed to load symbols (PDB files) for mscorlib.ni.dll even though I enabled Microsoft Symbol Servers in my Visual Studio symbol settings (I can tell that the symbol server is working, because immediately below the “failed to load” for mscorlib.ni.dll I can see that I did load symbols for twinapi.dll)

image

Unfortunately without symbols, you cannot see function names in native modules so all I can tell in the above screenshot is that I’m calling something in mscorlib.ni.dll but I have no idea which function I am calling; where if I’m able to get the symbol file for mscorlib.ni.dll I’ll be able to understand what I’m calling in mscorlib that is expensive. So in this blog post I will show you how to resolve this issue in four steps, but first let’s understand why this is happening.

Background

On Windows 8 the profiler uses a different underlying technology than what it does on previous versions of Windows, which is why the behavior is different on Windows 8. With the new technology, the profiler needs the symbol file (PDB) to know what function is currently executing inside NGEN’d images.

NGEN modules are native modules that are generated on the local machine from the original IL module, so there is no PDB file on the Microsoft symbol servers for framework modules. Furthermore, during NGEN on the local machine there is no PDB file automatically generated for you. Fortunately it’s possible to generate the PDB file yourself and point the profiler to it, so let’s look at the four steps necessary to do that.

Step 1: Determine where the NI module is on disk

  1. Go to the “Modules” view in the profiler report.     
    image
  2. Right click on any column header
  3. Choose “Add/Remove Columns…” from the context menu    
    image
  4. In the dialog box that pops up, choose “Module Path”    
    image
  5. Click OK

The module path column now tells you where on disk the module was during collection.

Here is a screenshot from my local example:

image

Step 2: Create the PDB

To create the PDB, you’re going to use the Native Image Generator (ngen.exe) that ships with the .NET framework. Since NGEN’d images are native, it’s important you use the copy of ngen.exe that matches the architecture of the application you are profiling (x86/x64/ARM). For example if the application is running 64 bit on Windows 8 RTM then you would need to reference the copy of ngen.exe in “C:\Windows\Microsoft.NET\Framework64\v4.0.30319”

  1. Open a command prompt on the machine where you ran the application during profiling (if you remote profiled a Windows Store App, you have to do this on the machine you were running the app on while you were profiling. It will not work if you do it on the machine you are viewing the report on)
  2. Decide if you just need to see the names of the functions (use this for .NET libraries*), or if need to be able to get back to the original source file including the ability to see which lines of source code were executing most frequently in the Function Details view (only recommended for modules that are part of your project):
    If you don’t need to get back to source then the process is very simple: In a command prompt type “[Microsoft.NET Path]\ngen.exe createPDB [NI module including the full path from step 1] [directory to store PDB]”, and the PDB for the NGEN’d module will be placed in the directory you specified to store the PDB
    This is an example of generating an NGEN PDB for 32bit mscorlib.ni.dll
    clip_image012


If you do need to get back to source
then you will need the PDB for the original pre-NGEN’d module. Once you’ve located that type “ngen.exe createPDB [NI module including the full path from step 1] [directory to store PDB] /lines [directory containing the original PDB]”
NOTE: ngen.exe executes within the security context of the module you are generating the NGEN PDB for. So if you are generating an NGEN PDB for a module that is part of a Windows Store App (this does not mean a framework module such as mscorlib.dll referenced by a Store App), then the output path for the NGEN PDB will need to be in a folder the app has access to (placing it next to the NI module is easiest). Similarly, if you are using the /lines flag, you will need to ensure that that the original PDB is also in a location accessible to the Windows Store app (again next to the NI is easiest). Once the PDB has been generated, you can move this to any location you like.

Below is an example of generating an NGEN PDB with the /lines flag for a module in my Windows Store App.
clip_image014

 

*PDBs from the Microsoft Symbol Servers do not support the /lines flag, so will not work for generating line level information for .NET modules

Step 3: Add the PDB to your symbol path

In the Symbols page, add the folders containing the NGEN PDBs you created in step 2. This is what that looks like on my machine:

image

Step 4: Open the profiler report

Close the report showing the [NI modules], and open it again; assuming you did steps 1-3 correctly you will now see method names instead of [NI module]. For example, using the PDBs I created in Step 2 and modifying my symbol path per Step 3, when I reopen the report that generated my original screen shot I see the following:

image

NOTE: The reason the “Exclusive Samples %” changed from the first screenshot, is the profiler is now able to distinguish between the individual functions inside mscorlib.ni.dll, where in the first screenshot you only see the total percentage for the entire module

In Closing

Once you’ve created an NGEN PDB for a module, all future reports will reuse the PDBs you have already created. However, if the module is updated or replaced (e.g. it is for your Window Store App and you rebuild it, or there is an update to the .NET framework), then you will need to generate a new NGEN PDB to match the new version of the module.

Hope that helps and I'd love to hear any questions/feedback that you may have so please leave a comment below.

Leave a Comment
  • Please add 1 and 1 and type the answer here:
  • Post
  • If you are having problems with getting this to work make sure you set the "Cache symbols directory" to C:\Symbols. It didn't work for me without.

  • I'm facing problems with the 'C:\Windows\assembly\NativeImages_v4.0.30319_32\WindowsBase\...\WindowsBase.ni.dll' too.

    My environment: Win7 x64, VS 2013.

    I do generate the pdb with ngen (32-Bit) w/o problems. When I put my symbols path into VS Debug settings, they aren't be loaded automatically.

    A dialog appears to select them manually. After selecting the pdb, a message is given that VS could not find a matching pdb file.

    After closing the dialog, suddenly the modules view states "Symbols loaded". Unfortunately my stack trace does not change (no method names are given for this .ni.dll).

    The only wired thing is: The GUID in the path of the Pdb file is one digit longer than in the GAC. The numbers are equal, but the Pdb folder has additionally the number '1' at the end.

    I did everything on my local machine, this means the SW has been build with my local compiler, so it seems to be a general problem.

    Any hints?

    Thank you and best regards,

    Marc

  • Thanks! This was helpful!

  • I need to run the profiler on a production environment and I can only use the standalone profiler. Do I need to run NGEN on the client machine?

  • Hi there David, you need to run ngen on the same machine that you are collecting the data(i.e. your production machine).

  • How could I create PDB file for an assembly that targets .NET framework v2.0 x86 platform? ngen.exe v4.0 complains that the assembly is an invalid image while the ngen.exe v2.0 does not have the createPDB command.

  • Hi Bowen,

    Unfortunately, creating a PDB for NGEN assemblies targeting .NET v2.0 is not supported today. You'll need to target at least .NET v4.0.

Page 2 of 2 (22 items) 12