Code Coverage tool in Visual Studio 11 instruments native and managed binaries (DLLs/EXEs) whenever they are loaded at runtime, if they meet some criteria. The code coverage information is collected for these binaries

At the end of the Code Coverage run you see code coverage within the Code Coverage results window. The total code coverage as well as code coverage for each binary are reported.

 

However in some cases you may end up with a .coverage file which shows 0% coverage in the Code Coverage Results window with an error similar to “Empty results generated: No binaries were instrumented …”

This means code coverage results were not obtained for any binary.

 

In this article we will attempt to list down the most common reasons for such a problem and provide you with resolutions for the same.

  1. No tests were executed
  2. PDBs (symbol files) are unavailable/missing
  3. Using an instrumented or optimized binary
  4. Code executed is not managed (.NET) or native (cpp) code
  5. A custom .runsettings file with exclusions is being used

 

Reason

No tests were executed

Analysis

Check your output window – Select “Tests” in the “Show Output from:” dropdown to see if there are any warnings or errors logged.

Explanation

The Dev11 Code Coverage engine is dynamic in nature. What this means is that the instrumentation of binaries occurs only for those binaries that are loaded into memory.

However if none of the tests are executed then there is nothing that is available for the Code Coverage to report.

Resolution

Verify if that the tests run fine, without Code Coverage turned on, by clicking on “Run All”. Fix any issues you find here before using “Analyze Code Coverage”

 

Reason

PDBs (symbol files) are unavailable/missing

Analysis

Check that all the modules within the solution have their associated PDBs available alongside the binary

Explanation

The Dev11 Code Coverage engine requires that every module have its associated PDB available and accessible during execution. If the PDBs are unavailable we will skip the module and not provide any data for the same.

Note: PDBs have a direct association with a DLL via build. The DLLs and PDBs should be from the same build for it to be recognized as a valid PDB.

 

If PDB are not installed alongside the binary, but are present in some share, configure code coverage settings specifying the path to pick the the PDB from. See “Customizing Code Coverage in Visual Studio 11

Resolution

Install the PDB alongside the binary; Or customize code coverage settings by specifying the path to pick the PDB from.

 

Reason

Using an instrumented or optimized  binary

Analysis

Check if the binary has undergone any form of advanced optimization (like the BBT optimization, Profile Guided Optimization, etc.) or has been instrumented by a profiling tool like vsinstr.exe.

Explanation

If a binary has already been instrumented or optimized by another profiling tool eg. vsinstr.exe we skip the binary and do not include it as a part of the coverage results.

 

Code coverage cannot be obtained for such binaries.

 

Resolution

Use the non-instrumented / non-optimized version of the binary.

 

In fact, it is recommended to turn off all optimizations, and run code coverage with non-optimized CHK build to get best results.

 

Reason

Code executed is not managed (.NET) or native (cpp) code

Analysis

Verify if your project has any .net or cpp UTs being run

Explanation

Code Coverage in Visual Studio 11 is available only for .net and native (C++) code. If you are working with any language that does not fall into either of these categories Code Coverage will not be available for code in these unsupported languages.

 

This can typically happen if you are developing tests with a third party unit test adapter extension.

Resolution

None available

  

Reason

A custom .runsettings file with exclusions is being used

Analysis

Verify if you are using a .runsettings file and have specified any exclusion rules that leave out your dlls from being instrumented

Explanation

You can run your UTs with a custom .runsettings file with Code Coverage configuration specified in the DataCollectors node. In the Code Coverage configuration we allow exclusions of dlls based on name, company name, public key token etc.You might have, by mistake, excluded all your DLLs, or missed including any of your DLLs here

Resolution

Fix your .runsettings to have the minimal set of exclusions that do not exclude your DLLs OR explicitly include the DLLs you want coverage for.

 

 

Further Analysis

To know why a specific DLL was not included by code coverage, you can analyze the .coverage file generated further using our command line utility, CodeCoverage.exe

Cd “<VSInstallDir>\Team Tools\Dynamic Code Coverage Tools”

CodeCoverage.exe analyze /include_skipped_modules my.coverage > analysis.xml

my.coverage is the .coverage file generated for your test run found in the TestResults folder.

 

Here /include_skipped_modules will provide information regarding each and every module that was inspected by the Code Coverage engine and provide a reason why it was skipped. These reason codes are to be interpreted as below:

 

Reason

Resolution

no_symbols

See PDBs (symbol files) are unavailable/missing

public_key_token_is_excluded

path_is_excluded

company_name_is_excluded

 

See A custom .runsettings file with exclusions

 

optimized_or_instrumented

See Using an instrumented or optimized binary

nothing_instrumented

No code in the dll could be instrumented.

See Code executed is not managed (.NET) or native (cpp) code

instrumentation_failure

The engine itself failed due to some internal error. Please try again.

 

If the output file is empty, see “No tests were executed