.NET Blog

Update 4.0.2 for the Microsoft .NET Framework 4

Brandon Bray — Thu, 27 Oct 2011 22:24:22 GMT

With .NET 4, we’ve focused on bringing certain features to market through updates without having to wait for the .NET 4.5 release. This post covers what the latest platform update has to offer. Vivek Mishra from the .NET Servicing team authored the following. -- Brandon

The .NET Framework team is excited to announce that Update 4.0.2 for the Microsoft .NET Framework 4 is now available. This is a platform update that offers a set of additional functionality on top of the .NET Framework 4. In this release, we have added support for AlwaysOn and LocalDB in SqlClient. These features were introduced in SQL Server Code Name "Denali." This post briefly describes what is included in the release and links to the download packages.

You can find further information about how to use this release, setup instructions, and links to documentation in the .NET Framework Developer Center.

What are the features in 4.0.2?

4.0.2 introduces new features for supporting AlwaysOn and LocalDB with SqlClient. 4.0.2 also includes the features that were released as part of 4.0.1. Here's a complete list of features included in this update:

AlwaysOn support in SqlClient

New 4.0.2 feature

SQL Server Express Local Database Runtime support in SqlClient

New 4.0.2 feature

StateMachine (including designer support)

Existing 4.0.1 feature

SqlWorkflowInstanceStore on SQL Azure

Existing 4.0.1 feature

Compensation for custom hosts

Existing 4.0.1 feature

Note that we did not make any changes to the features that were shipped in Update 4.0.1.

Where can I get 4.0.2?

If you want to set up your development environment for building .NET Framework 4.0.2 applications in Visual Studio 2010 Service Pack 1 (SP1), download and install the Design-time Update (KB2544525); for more information, see the Knowledge Base article. You must install Visual Studio 2010 SP1 first, before you can use this package.
If you want to deploy your .NET Framework 4.0.2 applications, download and install the Run-time Update (KB2544514); for more information, see the Knowledge Base article. You must install the .NET Framework 4 or .NET Framework 4 Client Profile before installing the Run-time update.
If you want to set up your development environment for building .NET Framework 4.0.2 applications in Visual Studio 11 Developer Preview or later, download the Multi-Targeting Pack (KB2544526) standalone installer for Visual Studio 11 Developer Preview or later; for more information, see the Knowledge Base article.

Target framework changes in Visual Studio

Based on recent customer feedback, 4.0.2 exposes the full .NET Framework version number in the format Major.Minor,Revision (.NET Framework 4.0.2) in the Target framework dropdown in Visual Studio. Note that this update does not change the file versioning system for the .NET Framework files -- it only exposes the underlying target framework that you can use.

If you discover any issues while working with this update, please send them our way via Microsoft Connect.

Large Object Heap Improvements in .NET 4.5

Brandon Bray — Tue, 04 Oct 2011 01:57:00 GMT

Garbage collection is one of premiere features of the .NET managed coding platform. As the platform has become more capable, we’re seeing developers allocate more and more large objects. Since large objects are managed differently than small objects, we’ve heard a lot of feedback requesting improvement. Today’s post is by Surupa Biswas and Maoni Stephens from the garbage collection feature team. -- Brandon

The CLR manages two different heaps for allocation, the small object heap (SOH) and the large object heap (LOH). Any allocation greater than or equal to 85,000 bytes goes on the LOH. Copying large objects has a performance penalty, so the LOH is not compacted unlike the SOH. Another defining characteristic is that the LOH is only collected during a generation 2 collection. Together, these have the built-in assumption that large object allocations are infrequent.

Because the LOH is not compacted, memory management is more like a traditional allocator. The CLR keeps a free list of available blocks of memory. When allocating a large object, the runtime first looks at the free list to see if it will satisfy the allocation request. When the GC discovers adjacent objects that died, it combines the space they used into one free block which can be used for allocation. Because a lot of interaction with the free list takes place at the time of allocation, there are tradeoffs between speed and optimal placement of memory blocks.

A condition known as fragmentation can occur when nothing on the free list can be used. This can result in an out-of-memory exception despite the fact that collectively there is enough free memory. For developers who work with a lot of large objects, this error condition may be familiar. We’ve received a lot of feedback requesting for a solution to LOH fragmentation.

A Better LOH Allocator

In .NET 4.5, we made two improvements to the large object heap. First, we significantly improved the way the runtime manages the free list, thereby making more effective use of fragments. Now the memory allocator will revisit the memory fragments that earlier allocation couldn’t use. Second, when in server GC mode, the runtime balances LOH allocations between each heap. Prior to .NET 4.5, we only balanced the SOH. We’ve observed substantial improvements in some of our LOH allocation benchmarks as a result of both changes.

We’re also starting to collect telemetry about how the LOH is used. We’re tracking how often out-of-memory conditions in managed applications are due to LOH fragmentation. We’ll use this data to measure and improve memory management of real-world applications.

We still recommend some traditional techniques are for getting the best performance from the LOH. Many large objects are quite similar in nature, which creates the opportunity for object pooling. Frequently, types allocated on the LOH are byte-buffers that are filled by third-party libraries or devices. Rather than allocating and freeing the buffer, an object pool would let you reuse a previously-allocated buffer. Since fewer allocations and collections take place on the LOH, fragmentation is less likely to occur and the program’s performance is likely to improve.

Compatibility of .NET Framework 4.5

Brandon Bray — Tue, 27 Sep 2011 02:36:00 GMT

Fundamentals were a big part of our focus while building .NET 4.5. We divided fundamentals into seven areas called “tenets”. One of these tenets is compatibility. Today’s post is by Manish Agnihotri, a program manager who is driving compatibility across the .NET Framework. -- Brandon

.NET Framework 4.5 is an in-place update that replaces .NET Framework 4 (rather than a side-by-side installation). Our goal is for .NET 4.5 to be fully backward compatible with applications built for .NET 4 (.NET 3.5 and .NET 4.5 will be side-by-side). We’ll talk about the compatibility story for .NET 3.5 in a later post. One of the first things you’ll notice about .NET 4.5 is the version number (4.0.30319) is the same as .NET 4; this is the practice used by other in-place updates.

Our primary concern is guaranteeing applications you use do not break after you install .NET 4.5. We accomplish this by running hundreds of application in our compatibility lab to find issues as soon as they’re introduced. While designing new features or changing existing code, we keep compatibility in mind. And a small group of us, the Developer Division Compatibility Council (DDCC), monitor changes made by developers. We review potential breaking changes, and help teams understand and assess the compatibility impact of new features and bug fixes. For .NET 4.5, members of DDCC reviewed every proposed breaking change, every new feature, and a majority of the bug fixes for the release.

We’ve put a lot of effort into maintaining a consistently high bar for compatibility across the product, yet we know some issues may get past us. Many applications will exercise the .NET Framework in ways that we did not expect or we lack test coverage for. Still we care about knowing every issue, even those that may seem like corner cases. Once you install .NET 4.5 Developer Preview on a machine that previously had .NET 4, any compatibility issues can be sent to the Connect feedback site.

Types of compatibility issues

There are three kinds of version compatibility testing we do: (1) binary compatibility, (2) source compatibility, and (3) serialization compatibility. You may also find these approaches useful in your testing, and should an issue arise this may help you narrow down the root cause. Having a wide range of scenarios within each kind of tests is also critical to ensuring good compatibility coverage.

Binary compatibility uses binaries built targeting .NET 4 and then are run on .NET 4.5. Essentially, we’re testing that the behavior of newer .NET libraries is equivalent to previous versions. This can range from making sure the return value of a function is the same or that the same exceptions are raised. The hardest issues are multi-threading behaviors – sometimes performance improvements can be a breaking change.

Source compatibility takes an application that builds and runs successfully against .NET 4 then does the same with .NET 4.5. An interesting case is a machine where Visual Studio 2010 runs on a machine with .NET 4.5 installed – since reference assemblies are used for targeting versions of the framework, building an app in VS2010 and then running it is actually binary compatibility testing. To do source compatibility testing, we build against the .NET 4.5 targeting pack (which is part of the Visual Studio 11 Developer Preview).

Serialization compatibility deals with data and has several permutations. First, we let an app running on .NET 4 serialize data (e.g. save to disk) and then an app running on .NET 4.5 de-serializes (e.g. read from disk). Second, we do this in reverse by serializing from .NET 4.5 apps and de-serialize in .NET 4 apps. Third, we take a web service running on .NET 4.5 reading serialized data created by a .NET 4 based client. Fourth and last, we do the inverse with a .NET 4 web service and a .NET 4.5 client. A serialization problem will often show up as corrupted data or some kind of serialization exception.

Feedback welcome

Compatibility is an important part of migrating apps forward, though in rare cases it is necessary for a framework library to make breaking changes. We try to avoid this, using it as a last resort. MSDN documents the known breaking changes in .NET 4.5. We also have a migration guide with advice for testing your app.

If you discover any compatibility issues while working with previews and betas, we want to hear about them via Connect. Hopefully, all our work to make this a highly compatible release makes it easy for you to upgrade to .NET 4.5.

Introducing .NET Framework 4.5

Brandon Bray — Thu, 22 Sep 2011 04:08:00 GMT

Last week at the BUILD conference, we had the pleasure of announcing the next version of the .NET Framework and releasing a developer preview at the same time. We have so many new things in .NET 4.5 to discuss with our developer community – we’re excited to have this opportunity to begin a discussion about each of them.

At the BUILD conference, many talks provided a glimpse of what .NET 4.5 has to offer. These are some of our favorite talks:

The .NET platform is crucial to so many developers’ day to day lives. We’ve been working for the past two years to build the best platform yet. We started the last release by listening to what customer’s wanted, and worked hard to make improvements where we could while supporting important new Microsoft releases like Windows 8. Over this release, a few themes drove a lot of our decisions:

First class support for writing Metro style applications with .NET
Improve performance with little to no effort from application developers
Write code more easily for data access and web services
Make it possible to write portable libraries across platforms, and support a new processor
Address many top developer requests for .NET APIs and tools

We want to dive into each of these areas. Blogging allows us to have a two-way discussion about the data and considerations that went into the design-decisions. We’re very excited to talk about what we’ve been building.

Who are we

Together, Layla Driscoll and I will be your hosts on this blog series as we tour what you can expect from .NET 4.5. We’ll invite others from across all the .NET teams to give an inside look at the features and scenarios now available in the preview. While we both are on the CLR team, together we have driven a great deal of the planning and experiences for this release of .NET.

It doesn’t take long before developers see how much .NET has to offer. That’s because so many teams from all over Microsoft contribute to the framework. We have much to share, and we’ll do our best to bring the work of many teams into the discussion. All of the posts will be written by engineers on the .NET team.

Starting a new discussion

Keeping a line of discussion open with the developer community is a critical factor in how we make decisions for the .NET Framework. That’s one of the reasons why we keep the Connect site open for suggestions and bug submissions year round. Then we recently joined UserVoice to give the community a way to vote on requests and wishes important to you. Lastly, the MSDN Forums give the community a place to ask questions and provide answers.

Here on the .NET blog, we hope to spark new discussions about specific work. We want to share with you the context and decisions that went into the features and scenarios we built. We hope that the community will share their opinions and participate in helping us make this an outstanding developer product. Admittedly, we’ll make mistakes and we’ll recognize when we do. We’ll work hard to make the conversation constructive and learn from what you all have to say.

We do want comments, and many of us on the .NET teams will be watching. Like other Microsoft blogs, we hope to see in comments:

On topic discussions about .NET and the posts on this site
Respect for everyone in the conversation
Focus on the content of the post; do ask for more details

And please refrain from comments that would be considered abusive (according to the community standard), misrepresentation, repeatedly posting the same comment, or blog-spam.

With that, I want to say thank you for reading and I look forward to joining us in the discussion about .NET.

--Brandon Bray

Changes coming to ClickOnce applications running in the Internet Zone

Abhishek_Mondal — Tue, 07 Jun 2011 00:02:40 GMT

We wanted to let you know that we’ll be changing the signature verification process for ClickOnce applications and WPF XAML Browser Applications (XBAPs) in an upcoming update. This change will help users recognize when they’re running untrusted applications from the Internet Zone, but may require you to make changes to your applications. Currently, applications configured to run “online-only” will not display notifications requesting that the user make a decision whether to run the application or not. After the update, users will see an Application Run dialog commonly encountered when accessing applications over the Internet using Internet Explorer.

In some situations you may want to retain the current application launch behavior. There are two ways to achieve this:

1. Add the application publisher to the Trusted Publisher list. (http://msdn.microsoft.com/en-us/library/ms172241.aspx)

2. Add the application URL to the Trusted Sites list (http://windows.microsoft.com/en-US/windows-vista/Security-zones-adding-or-removing-websites)

Hardening Server Applications

Abhishek_Mondal — Fri, 01 Apr 2011 16:09:14 GMT

The Base Class Libraries team, the wider CLR team and Microsoft Research worked hard on this new technology and we are very proud to be able to share the news. Read the full article for details.

Is That the Right Version of Mscoreei.dll?

Abhishek_Mondal — Mon, 13 Sep 2010 22:39:10 GMT

After installing .NET 4.0 or later you may notice something a little unusual about your .NET processes. Here is a partial list of the loaded modules of a simple “Hello World” executable compiled with the .NET 2.0 compiler.

start end module name

60f00000 61491000 mscorwks C:\Windows\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll

6c650000 6c6b6000 mscoreei C:\Windows\Microsoft.NET\Framework\v4.0.30319\mscoreei.dll

6d420000 6d46a000 MSCOREE C:\Windows\SYSTEM32\MSCOREE.DLL

75a80000 75aca000 KERNELBASE C:\Windows\system32\KERNELBASE.dll

Something here looks out of place – the mscoreei.dll file is loaded from the v4.0.30319 folder. What is it doing there next to the main clr dll (mscorwks.dll) from v2.0.50727? Actually, that’s the expected behavior. We call mscoreei.dll the “shim implementation”, or “shim impl” for short, and it’s new for .NET 4.0. The third dll in the list above, previously known as the “shim”, is now more accurately known as the “shell shim”. The two are tightly paired to do the main job that was previously done by mscoree alone – providing the interfaces for loading the runtime. Generally speaking, the shell shim now consists of thin wrapper functions, each of which delegates its functionality to a corresponding function in the shim implementation.

Why the split? We’ve seen significant numbers of machine reboots as part of .NET redist installation, typically due to needing to update a file that is in use. The most common file in use has been mscoree.dll, which is loaded by every single .NET application, and even some services such as MSI. As a result, we undertook the mscoree “split” to avoid machine reboots. By moving the shim implementation into a version-specific file, we are able to deploy a new version of mscoreei.dll (say in the v5.0 .NET folder, as part of .NET 5.0 installation), without touching the machine-wide mscoree.dll file. The next time a managed app is run, mscoree will dynamically find the new mscoreei and defer each of its function calls to it. That way we can deploy new versions of the framework on a machine running existing managed applications without requiring a restart.

Note that mscoree always uses the newest mscoreei it can find, but the actual runtime that is loaded is a completely different question. Therefore it shouldn’t be a surprise to see a newer version of mscoreei loaded with older versions of runtime dlls in the same process. In fact, CLR versions have been running with the latest shim for many years now. If I examine file versions when running a .NET 1.1 app on a machine that has both .NET 1.1 and .NET 2.0 installed I can see that I am using a shim newer than the runtime.

start end module name

79000000 79045000 mscoree C:\WINDOWS\system32\mscoree.dll

File version: 2.0.50727.42

791b0000 79412000 mscorwks C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorwks.dll

File version: 1.1.4322.573

The difference now is that the split of the shim makes it more obvious that the shim version doesn’t always match the runtime version because the version number is right there in the path of the shim impl.

This brings to mind an aphorism from David Wheeler: All problems in computer science can be solved by another level of indirection.

In-Proc SxS and Migration Quick Start

Abhishek_Mondal — Wed, 23 Jun 2010 21:41:06 GMT

This post is meant to help you understand what runtime in-process side-by-side is, how to think about it, how to use it, and how it affects application and component migration to the .NET 4 Runtime. This post is relevant to you if you use native runtime activation APIs, depend on specific runtime activation behaviors, or use mixed mode assemblies built with Visual Studio Managed Extensions for C++ v8 or v9, or if you’re just interested in how we handle pre-.NET 4 and .NET 4 code coexisting on a machine.

The problem

Prior to the .NET 4 Runtime, any given process was limited to loading only one runtime version, and was bound to that runtime for remainder of that process lifetime. In environments where independently authored components built and tested against different .NET Runtime versions can be loaded into a single process, this sometimes caused compatibility problems for the components that did not target the runtime that was loaded into the process.

Our Solution

A new feature in the .NET 4 Runtime, “in-process runtime side-by-side” (or “in-proc SxS” for short) describes the ability to load more than one .NET Runtime version into the same process and have them run “side by side”. This ability to load multiple runtimes into the same process gives us with the ability to provide the highest level of compatibility possible for environments in which multiple independent managed components are loaded by, and communicate with, a native layer. COM is the biggest example of a native layer through which managed code must communicate, and this means that in an environment in which multiple independently-authored managed COM components are activated, each may be loaded into the runtime for which it was built and tested, which maximizes compatibility. The upcoming release of Visual Studio Tools for Office (VSTO) will take advantage of this by enabling all managed office addins to be loaded within their targeted runtime.

Our Solution Is Not…

In-proc SxS does not affect managed assembly loading, such as Assembly.Load or loads due to assembly references – these scenarios continue to load the target assembly in the runtime where the load request was made. To take advantage of in-proc SxS, a managed component must be activated by a native host and interact with its environment through a native interop layer such as COM interop and P/Invoke.

This may be easier to understand if you consider that that two runtimes loaded into a process operate completely independently; i.e., runtime X has no more knowledge about runtime Y than it does about any other native DLL loaded into a process. Each runtime has its own GC; each runtime creates, owns, and manages its own Application Domains (never the other way around), including Shared and Default domains for each; and each runtime interacts with unmanaged DLLs through interop layers.

How to Think About In-Proc SxS

Thinking about multiple runtimes in the same process without additional context isn’t all that helpful; in fact, it just leads to more questions: “How and when does this happen?”; and most importantly, “Does this affect me?”

Does In-Proc SxS Affect Me?

In-proc SxS was designed so that existing applications, components and libraries should be completely unaffected by an installation of the .NET 4 Framework - applications that were running against a pre-v4 framework will remain blissfully unaware of the install.

In addition, migrating a product to .NET 4 is unlikely to be affected by in-proc SxS if:

· The product is a pure managed application;

· The product is a pure managed library; or

· The product is a pure managed COM component.

However, a product may be affected by in-proc SxS during migration if:

· The product makes use of one or more deprecated APIs; or

· The product contains, or takes a dependency on, a mixed mode assembly built with Visual Studio Managed Extensions for C++ version 8 or 9.

See the migration section below for more information.

Runtime Activation

For the small number of you who may be affected by in-proc SxS and need to gain a better understanding of it, the easiest way to understand how and where in-proc SxS fits into the managed world is to think about it as a fundamental change to our runtime activation model. Runtime activation describes the process by which the most appropriate runtime version is found, located, and loaded. A runtime request consists of a set of inputs, which can include (among others) an assembly (often the application’s EXE), an application configuration file (often found next to the application’s EXE), a version string, or any combination thereof, and runtime activation uses these inputs to determine the most suitable match from among the set of installed runtimes. Common examples are: managed EXE launch, CoCreateInstanceInstance of COM-visible managed types, and mixed-mode assemblies loaded from native code.

Old Activation Model

In prior releases, when in-proc SxS did not exist, our runtime activation policy had to take into consideration the reality that only one runtime could ever be loaded into a process. For self-contained managed applications the obvious choice was invariably the right one: activate the runtime that the application targets (derived either from metadata in the managed executable itself or from an application configuration file if one exists). And since virtually all managed application installers out there list the application’s target runtime as a pre-requisite, the application could count on the target runtime being installed. Unfortunately, runtime activation was not always as clear cut in other important scenarios, the most prevalent of which is the activation of managed COM components. In such scenarios, the chosen policy has always been to activate in the latest installed runtime when no runtime has yet been loaded – a policy we call “bind to latest.”

Consider the following example to help understand why this policy was chosen for managed COM class activation. Let’s say that a native application X exposes an extensibility mechanism through COM, and two authors separately write managed extensions A and B that target .NET Frameworks v1.1 and v2.0 respectively, and that application X and both extensions are installed on machine Z with both .NET Frameworks v1.1 and v2.0 installed.

Say that, in the course of normal execution, application X CoCreateInstances component A. In this case it may be tempting to use the same activation policy as that of managed executables and pick the runtime that the component targets (v1.1). However, this choice would cause a later attempt to CoCreateInstance component B to fail, because the .NET Runtime v1.1 (to which the process has been locked) cannot load a v2.0 assembly.

To avoid this problem, the policy for managed COM component activation has always been to activate and load the component into the latest installed runtime, if a runtime has not yet been loaded. However, this policy created an impossibly high compatibility burden: we would need to provide 100% release-to-release compatibility, and to do so we would need to test every possible usage scenario for every managed application ever built on any previous .NET Runtime! So, while the entire .NET team will always set very high release-to-release compatibility requirements, we can all sleep a bit easier knowing that in-process side-by-side enables us to provide a much better compatibility model for our customers, insulating them from the small but inevitable accidental breaking changes that occur in a major product release.

New Activation Model

I’m sure you’ve guessed the fundamental change to our activation policy by now: starting with version 4, the .NET Runtime activation policy may now load the most appropriate runtime version for a given activation request, without regard for the set of runtime version(s) that may already be loaded into the process. The activation policy will no longer target the latest runtime on the machine as it did in previous versions (more details on this below). Again, there are some restrictions with when dealing with pre-v4 runtimes, but the above statement will hold true, without restriction, for all .NET Runtimes version 4 and higher.

Using the previous example, but with the small change that component A has been updated to target .NET 4, runtime activation policy is now free to activate component A in the v4 runtime and later activate component B in the v2 runtime. This provides the greatest compatibility by allowing components to run within their target runtime, without regard for the process environment into which it is loaded.

Now that we’ve covered the basics of the new in-proc SxS feature and activation model, let’s look at the necessary considerations when migrating an application, component or library to .NET 4 from an earlier runtime.

Migration

In the vast majority of cases it is simple to migrate an application to run on .NET Framework 4: if you are migrating an application at the source level, import your solution into Visual Studio 2010 and choose “.NET Framework 4” for the Target Framework in each project’s properties page, then recompile; if you are migrating an application without recompiling, create or update the application configuration file to target .NET 4.For example, the following is the recommended <startup> section to use when migrating an application that targets .NET 3.5 to also target .NET 4:

<?xml version ="1.0"?>

</startup>

</configuration>

Note that starting with .NET Framework 4, only a two-part version string is required, though three-part version strings are still accepted.

A migration decision tree is provided later in this section.

Limitations and Exceptions

When transitioning code originally targeting an in-proc-SxS-unaware runtime to an in-proc-SxS-aware runtime, some additional steps will need to be taken if your application contains one or more of the following:

· calls to any of the hosting global static functions, from native and/or managed code;

· calls to any of the Strong Naming Global Static Functions, from native and/or managed code;

· native code that calls CoCreateInstance using any of the hosting CoClasses (CLSID_CorRuntimeHost, CLSID_CLRRuntimeHost, CLSID_TypeNameFactory, or ComCallUnmarshal), or is managed code that uses COM interop to access interfaces implemented by any of the same CLSIDs;

· calls to CoCreateInstance, from native code, using any of the metadata CoClasses (CLSID_CorMetaDataDispenser, or CLSID_CorMetaDataDispenserRuntime), or managed code that uses COM interop to access interfaces implemented by any of the same CLSIDs;

· calls to CoCreateInstance, from native code, using the CoClass of any COM-registered .NET Framework type;

· a managed COM component (in the binary migration case only);

· an assembly produced with the Visual Studio Managed Extensions for C++ compiler (otherwise known as a managed C++ assembly), or a dependency, either direct or indirect, on such an assembly.

Figure 1

The functionality in the above list was created for prior releases, and each lacks the ability to specify the target framework that is needed for in-proc SxS, and could not be retrofitted to do so. For the most part, all of these chose the same “bind to latest” activation policy as managed COM component activation.

Non-Impactful Install and “Latest Runtime” Activation Model

Why am I even talking about our “bind to latest” activation policy? Well, there’s a very good reason: one of the primary compatibility-related goals of the .NET 4 Framework is to ensure that its installation is non-impactful to pre-existing managed application or component. To achieve this, we have had to modify the “bind to latest” activation policy’s semantics in the .NET 4 release.

The “bind to latest” runtime activation model kicks in when no target runtime version information is provided with the activation request, such as through a managed assembly’s built-in version, a configuration file’s list of <supportedRuntime> entries, and/or an explicit version string passed into any of the native hosting APIs that accept them. In these cases, in previous versions of the runtime, “bind to latest” activation would pick the very latest .NET Runtime installed on the machine. This is also the activation model used for managed COM classes built against pre-v4 runtimes, for precisely the reasons stated in the earlier example.

Starting with the .NET 4 Runtime, this “bind to latest” activation policy has been modified so that, by default, it will only consider pre-v4 runtimes. By making this the default behavior, all pre-existing applications and components using this activation model will be unaffected by the installation of .NET 4.

Source Migration

If you intend to retarget your application or component to .NET 4 and rebuild, and it uses any of the functionality listed above, then you need to migrate to the equivalent functionality provided in the new .NET 4 Framework Hosting interfaces:

· for calls to any of the hosting global static functions, use the corresponding methods provided by the new ICLRMetaHost, ICLRMetaHostPolicy and ICLRRuntimeInfo interfaces;

· for calls to any of the Strong Naming Global Static Functions, use the corresponding methods provided by the new ICLRStrongName interface;

· for instance creation of any of the hosting CoClasses, metadata CoClasses, or COM-visible Framework types, use ICLRRuntimeInfo::GetInterface.

Figure 2

Each ICLRRuntimeInfo and ICLRStrongName interface instance targets a specific runtime, and each method exposes functionality specific to, or provided by, that targeted runtime.

Binary Migration

If the preferred option of source migration and re-compilation is not possible, and the application or component also depends on any of the functionalities listed above, an application configuration extension is available to make migration possible in many cases. Adding this extension to the application’s configuration file will modify the “bind to latest” policy to cause it to bind to the same runtime as the application itself. The extension takes the form of an attribute named useLegacyV2RuntimeActivationPolicy, to be placed in the <startup> element of an application configuration file. The attribute’s valid values are “true” and “false”, of which “false” is the default. When set to “true”, this attribute instructs “bind to latest” runtime activation policy to consider all <supportedRuntime> entries in the application configuration file; when set to false, the policy considers only those <supportedRuntime> entries with a major version number of 2 or less. When a configuration file does not contain any <supportedRuntime> entries (or a deprecated <requiredRuntime> entry), then the version number contained in the executable image, which represents the runtime version against which the executable was built, is treated as an implicit <supportedRuntime> entry (roughly speaking).

Thus, if you were migrating a .NET 3.5 application to .NET 4, your configuration file might look like this:

Figure 3

Managed COM Component

To accommodate managed COM component authors (who in most cases cannot modify application configuration files), a separate mechanism is available: a semicolon delimited list of supported runtimes may be added to the component’s InprocServer32 registration. Specifically, add the string value “SupportedRuntimeVersions” to “HKCR\CLSID\<COM component clsid>\InprocServer32\<COM component version>”.

It is important to note that if a managed COM component depends on any of the items in Figure 1, then it must be migrated at the source level. This is because the COM component would need functionality equivalent to the new useLegacyV2RuntimeActivationPolicy configuration extension to work, which is a process-wide policy decision that must be made by the application author rather than by extensibility components.

If a managed COM component is activated through a reg-free manifest (which cannot be extended to contain a SupportedRuntimes entry), runtime activation will look for a configuration file next to the assembly containing the COM visible class. Note that a useLegacyV2RuntimeActivationPolicy entry will be ignored in this scenario.

Assemblies Built with Managed Extensions for Visual C++ Compiler

There are two additional limitations with managed C++ assemblies.

First, any dependency on a mixed-mode assembly built with a previous release of Visual C++ must be migrated using the useLegacyV2RuntimeActivationPolicy configuration file extension. This is because all such assemblies depend implicitly on the managed CRT, which in turn contains dependencies on functionality listed in Figure 1.This dependency has been removed in the upcoming release.

Second, all mixed-mode assemblies (including those built using this release of Visual C++) and all assemblies built with a previous release of Visual C++ may be loaded into at most one runtime per process, because they may contain process-global static image data that cannot be virtualized across runtimes, and/or because they depend on a prior release of the managed CRT. This limitation is virtually identical in nature to the current limitation restricting these types of assemblies to no more than one application domain at once.

Migration Decision Tree

Use the following decision tree to decide the correct migration path for your component in the most common scenarios.

Binary Migration Examples

Application: Target v2 and v4

Create an application configuration file with the following contents, where the highlighted portion is required only if the application depends on functionality from Figure 1:

<configuration>

<startup useLegacyV2RuntimeActivationPolicy="true">

<supportedRuntime version="v4.0"/>

<supportedRuntime version="v2.0.50727"/>

</startup>

</configuration>

As always, supportedRuntime entries are considered to be listed in decreasing preference; if the .NET 2 Runtime is preferred, the entries should be reversed.

Application: Target v4 Only

Use the same application configuration file as above, but remove the v2-specific supportedRuntime entry.

Managed COM Component: Target v2 and v4

If you authored a managed COM component built against the .NET 2 Runtime, with a CLSID of {DFEEABC2-17FF-4ce6-B25B-A35F7A156C68} and a version of 6.2.0.0, and you wanted the component to run within either the .NET 2 Runtime or the .NET 4 Runtime (in that order of preference), the registry update (in RegEdit syntax) would be:

[HKCR\CLSID\{DFEEABC2-17FF-4ce6-B25B-A35F7A156C68}\InprocServer32\6.2.0.0]

"SupportedRuntimeVersions"="v2.0.50727;v4.0"

If your component was registered without the assembly version portion of the registry key name, you would remove “\6.2.0.0” from the key name.

Or, if your managed COM component is activated through registry-free COM, you would place an application configuration file (should we coin the term component configuration file?) next to the managed COM assembly with contents similar to the example above (without the highlighted portion, and with the supportedRuntime entries reversed).

Troubleshooting

Instantiating Managed COM Components From Managed Code

If your application instantiates and uses COM components through the .NET Runtime’s managed interop layer, and one of those COM components happens to be managed, then there is a small chance that after migrating your application you might run into an error similar to this:

Error: Unable to cast COM object of type 'System.__ComObject' to interface type 'X'. This operation failed because the QueryInterface call on the COM component for the interface with IID '{12345678-1234-1234-1234-123456789012}' failed due to the following error: No such interface supported (Exception from HRESULT: 0x80004002 (E_NOINTERFACE)).

This error can occur as a result of the following two combined conditions:

1. The managed COM component is registered against a different .NET Runtime version from that of the instantiating caller, causing it to be activated in another runtime than that of the instantiating caller.

2. The interface is not marked with System.Runtime.InteropServices.ComVisibleAttribute, with a value of true.

This scenario worked in the past because of a runtime optimization: when the .NET Runtime notices that a COM type it instantiates has a managed implementation, and that it was instantiated into the same Application Domain as the instantiating caller, it bypasses the managed interop layer and returns to the caller the managed object itself. In prior .NET Framework releases (where in-proc SxS was not available), all managed COM types and their managed clients were commonly loaded into the same Application Domain, allowing the implementing managed object to be returned, and in turn allowing the interface cast succeed without being marked as ComVisible.

Migrating an application can easily create condition 1; if the application also relies on condition 2 (which is much less common), then the above error will be encountered. There are a couple of solutions available:

1. Get an updated version of the managed COM type that properly declares the interface as ComVisible.

2. Update the application configuration file to use the useLegacyV2RuntimeActivationPolicy extension.

Loading Managed C++ Assemblies Built Against Previous .NET Runtime Versions

Attempting to load a managed C++ assembly that was built with an earlier compiler into the .NET 4 Runtime when the application has not been configured with the useLegacyV2RuntimeActivationPolicy configuration file extension will result in the following error:

System.IO.FileLoadException: Mixed mode assembly is built against version 'v2.0.50727' of the runtime and cannot be loaded in the 4.0 runtime without additional configuration information.

Note: the error is not 100% correct, as it specifies that only mixed mode assemblies are subject to this problem; it should say “Managed C++” instead of “Mixed mode”, as managed C++ assemblies compiled with /clr:pure will also encounter this error.

It is possible to correct this issue with either of the following:

1. Recompile the assembly with the .NET 4 managed C++ compiler.

2. Update the application to use the useLegacyV2RuntimeActivationPolicy configuration file extension.

NGen: Walk-through Series

Abhishek_Mondal — Tue, 04 May 2010 01:22:51 GMT

The CLR Code Generation team blog has a series of new posts featuring hands-on style content around how to use the NGen technology and how to measure performance benefits from it. You can find the series here.

Connect

Abhishek_Mondal — Tue, 13 Apr 2010 23:42:00 GMT

As hopefully most of you know, Microsoft has a site called Connect where customers can log bugs and suggestions (http://connect.microsoft.com/). Issues for .NET can be found in the Visual Studio and .NET Framework feedback category (http://connect.microsoft.com/VisualStudio/Feedback) and can be logged from the main Visual Studio Connect page (http://connect.microsoft.com/VisualStudio/).

One of the complaints we often hear from customers is that they don’t know what happens to a Connect issue after they file it. So I wanted to take this chance to tell you a little about how the process works on the CLR team.

Once the issue is opened, there’s a first level of screening just to figure out which team it should be sent to. It usually ends up assigned to someone in the product within a few days. If the initial assignment was incorrect, though, it could take a couple more days to get it to the right person.

If the issue is a code defect, we treat it like any other product bug. We make sure it’s reproducible and not by design and something we can fix for the current release depending on where we are in the product cycle. We don’t necessarily do this right away, though. There are times in our product cycle when we’re more focused on bugs than others, and the bar gets higher toward the end of the release when all changes are riskier. Any change can introduce regressions in compatibility, performance, or behavior, and taking changes toward the end gives us less time for stabilization. We might not be as good about delivering status updates on Connect as we could be, but we do try to comment on each issue.

When deciding whether to fix a bug, there are several criteria we look at. One of them is whether anyone might have taken a hard dependency on the existing behavior. We don’t want to solve one problem but end up with a bunch of customers whose applications no longer work after upgrading to the newest version or applying the latest patch. There are occasionally security reasons to make breaking changes, but we try to avoid them when we can. We also look at things like how hard it is to work around the bug and how many votes there are on the issue. The severity of the issue also affects the priority: Is this a bug that will crash your machine, or is it just a minor annoyance?

Another thing we look at when deciding to fix a bug is how many people are likely impacted. If you find a new bug in an older product, like .NET 2.0 that’s been out for 5 years and used by hundreds of thousands of developers, you might be one of only a few people who is impacted by that bug. But if you file a bug on a newer product, like .NET 4, that’s has just been released, there’s a good chance that hundreds or even thousands of people are impacted. Because of this, we take bugs on products in Beta, or that have just been released, particularly seriously.

Once we do decide to fix a bug, we have to decide where to fix it. Our default answer is generally to fix it in the next major or service pack release (which is usually the one we’re working on at the time). Only fixes for particularly severe bugs might be included in an update to an existing release. If you are blocked on a bug for an existing release that’s been out for longer, e.g. .NET 2.0, and you need a fix, Connect is not the place to file a bug. You should be contacting Microsoft customer support to request a QFE.

We tend to get a high number of suggestions, both from Connect and other sources, and we don’t always get a chance to look at them and respond right away. But we do look at all the suggestions that come in on Connect. When suggestions come in, we consider whether it’s something we might do in this release or a future release. We also check to see if it’s a duplicate of an existing suggestion

Aside: please search before you file. One suggestion with 20 votes has more impact than 20 suggestions with 1 vote. You can sort by number of votes on the site, and that helps us focus our attention when it comes to suggestions. For example, we recently fixed a SerialPort issue and a serialization issue that had a high number of votes.

We try to close issues if there isn’t a chance we’d ever do them, but we leave ones we might consider open so that the community has a chance to vote and comment on them. Again, these can go for a while without comment, until we get to a point where we can act on them. However, we do have a mechanism to be alerted when someone comments on an issue. It’s opt-in, so it may not be universally used, but it may get someone’s attention. If suggestions come in while we’re working on a release and already have plans, it’s less likely we’ll be able to implement anything large for that release.

One thing that really helps us with suggestions is when the poster or commenter includes details on what problem this would help them solve. With a lot of suggestions, people jump straight to the suggested solution. You may think this is a shortcut for us, but it’s not always. We see those and realize that maybe we could do that, but don’t necessarily know why we should. What would you be able to do if you had this that you can’t do now? Often there are several ways to solve a problem (at least). If we know what you’re trying to accomplish, it helps us measure whether our solution really met the goal. Or if we see lots of customers with similar problems, we might be able to come up with a slightly different solution that meets lots of needs. And it’s a lot easier to reason about that from the problem descriptions than from the often-diverse solution descriptions.

If you’re wondering about the kinds of suggestions we’ve acted on, in .NET 4 we took suggestions ranging from new numeric types to adding single APIs or overloads that helped customers out.

For those of you wondering when we’ll get to these suggestions, the answer is now. With work on .NET Framework 4 coming to an end, we’re starting to look at the open suggestions in more detail. Through May we’ll be trying to close ones we don’t think we’ll be able to do in our next cycle. We’ll also be trying to work through any bugs that might still be open. The fact that a suggestion is still open in June doesn’t mean you’ll necessarily be seeing it any time soon, but only that we’ll take the time to look into it more closely.

If you’ve been thinking about voting or commenting on a suggestion, or opening a new one, now would be a good time. If we close one that you feel passionately is more important than some of that other stuff we’ve left open, let us know that, too.

Announcing a new CLR Team Codeplex Project: CLR Runtime Activation

Abhishek_Mondal — Tue, 16 Feb 2010 21:04:23 GMT

This Codeplex site will feature demos, samples, and helper libraries related to the selection and activation of the CLR: http://clractivation.codeplex.com/

With .Net Framework 4 and VS 2010 the new ability to load multiple versions of the runtime has changed the way a version is chosen for any particular piece of managed code and gives developers much more control over the runtime their code will end up running against. For a detailed introduction to this new functionality you can see this MSDN magazine article. This CodePlex project will focus on the various ways to start up the runtime and how to influence these choices using the new runtime activation APIs introduced in .Net Framework 4. Our first two projects include a helper library to ease migration from the old to the new APIs and a managed wrapper of our new native activation APIs.

Sharing Silverlight Assemblies with .NET Apps

Abhishek_Mondal — Tue, 01 Dec 2009 23:08:00 GMT

At the recent PDC, Scott Guthrie announced in his Silverlight 4 keynote that we had implemented a new feature, to enable developers to share certain assemblies between Silverlight and .NET. There are many differences between Silverlight and full .NET including WPF, and this new feature doesn’t solve those differences – in those cases, you’ll still need to compile your code twice. But in some cases, developers will write code that only uses features whose behavior is identical between Silverlight and full .NET, and in those cases, we want to enable that code to be shared. This post provides more detail on that sharing, and explains how developers can target it, and what the restrictions are.

Developer Scenario

Today, many developers write code that is intended to run on both Silverlight and .NET. A good example would be validation code: when writing a client-server application, you want to validate the data at the client (to make sure the user gets quick feedback) and then re-validate it again at the server (to make sure a rogue client implementation can’t send back bad data.) Today, in order to get that scenario to work, developers need to compile their source for Silverlight and also for .NET. In addition to dual-compilation, developers need to manage and deploy those assemblies separately, ensuring that they end up being deployed to the right places and consumed by the right platform (Silverlight or .NET). Today’s model is certainly workable, but it is not ideal.

We’ve heard from many developers that you want something better. We’ve also read quite a number of blog and forum posts stating the same. We’ve even seen some interesting workarounds to try and make the situation better for themselves and other developers. Cool! Specifically, developers want to write and compile their code once and deploy it as part of both their Silverlight and .NET apps, without having to dual-compile or worry about paying attention to the compilation target. This ability has the obvious benefit of avoiding duplication of effort for a number of the steps in your development and deployment processes.

Portability Explained

We named this new feature “assembly portability”, given that the feature allows your code to be “ported” between both Silverlight and .NET. Portability provides the ability to compile your source with the Silverlight tools, and run your built assemblies on both the Silverlight and .NET runtimes. This feature doesn’t change the underlying implementation of either the Silverlight or .NET runtimes; instead, if you write code that only uses APIs that have the same behavior across Silverlight and .NET, it allows you to use one set of binaries to target both. But how do you know if the APIs you are using are compatible? We have identified five key assemblies that are compatible between Silverlight and .NET. (The Silverlight UI layer is of course not one of these – there are some important differences between Silverlight UI and WPF.)

Like most features we build, we had a lot of choice about how to design the feature, and which scenarios we would enable. At the heart of the feature, a major design choice was to enable portability from Silverlight to .NET. The motivation for this choice was that Silverlight exposes an API subset of .NET, and so assemblies built with Silverlight should “just work” on .NET, whereas the reverse direction (.NET assemblies running on Silverlight) would have proven more technically challenging (for us all).

Another important design choice was identifying the set of Silverlight/.NET assemblies that developers could safely use, while still maintaining binary comparability. We looked at the scenarios that would both most benefit from portability, and would be straightforward for developers to use. We looked at a bunch of scenarios, including: the most low-level, typical business logic, networking and also UI. We decided we would start with the most fundamental and requested scenarios for this release. As a result, for .NET 4 and SL4, we have enabled portability for a significant set of low-level assemblies that we believe will enable a variety of interesting scenarios.

For SL 4 and .NET 4, we have made the following assemblies portable:

Mscorlib
System
System.Core
System.ComponentModel.Composition
Microsoft.VisualBasic

Note again that the Silverlight surface area for these assemblies is what we made portable. There are a great number of types and members in the .NET surface area that cannot run on Silverlight. As a result, you need to write code that targets the Silverlight versions of these assemblies in order to get this scenario to work.

Also note that there may be some behaviors that are not strictly identical across both platforms. We’ve worked to avoid those behaviors, however, those differences are sometimes very subtle and difficult to identify. Please do contact us if you do see behavioral differences between Silverlight and .NET, for portable APIs, which are bothering you.

Visual Studio Experience

Like all .NET features, it is important that we provide good tooling support in Visual Studio. This feature can be used in Visual Studio, in just the way that one might imagine. Developers should write their portable logic in Silverlight Class Library projects, and then are free to reference such projects from both Silverlight and .NET application projects. To clarify, on the .NET side, you can reference such a library from any kind of .NET project (Ex: WPF, winforms, WCF, WF, ASP.NET, …).

The only caveat is that the support in Visual Studio was implemented in such a way that you need to click a few extra times to get a reference to a portable project correct setup. For VS 2010, you must rely on browsing to the binary – AKA “binary reference” – that is built from the Silverlight class library to setup a reference. You cannot just reference the project itself – AKA “project to project reference”. Note that this restriction only exists for .NET apps, and not Silverlight ones.

The following are the basic steps to follow to enable the use of portable code in Visual Studio 2010, followed by a set of screen caps that hopefully make it super clear what to do.

Steps:

Setup projects
1. Create .NET application
2. Create or add existing Silverlight class library project
3. Build Silverlight project
Establish portable code reference
1. Reference Silverlight class library binary from the .NET project
Code
Run App!

Screen Caps:

Step 1 – Setup projects

Step 2 -- Establish portable code reference to the other project (required going up and back down the directory structure to the Silverlight library project).

Step 3 – Code

Step 4 – Run App!

Note: My app is a WPF app that is intended to display how many days there are until the Vancouver 2010 Olympics start, from today. The Silverlight library is the one that does the actual countdown calculation, and is (naturally) usable in both .NET and Silverlight apps. Cool!

Looking Forward and Feedback

It is great to see so much energy around both .NET and Silverlight, and particularly around the maximal sharing of code across the platforms. I encourage you to use this new scenario to its fullest extent, and to communicate back to us where you’d like to see the feature expanded in the future.

We are aware of some scenarios that are not currently enabled to be portable, but that developers will probably run up against pretty quickly. The most obvious ones are XML, networking, calling WCF web-services and UI code. Another interesting one is portable XAML. It would be really useful to hear from you on the specific scenarios that you would like to implement in portable code, but cannot implement due to the limitations that I’ve mentioned. I’d also like to hear why you saw being able to implement that scenario in portable code as being such a benefit. Your feedback will help guide us with future changes in the portable code space.

How to Make the Most of Your .NET Server Code

Abhishek_Mondal — Fri, 13 Nov 2009 18:32:00 GMT

One of our team’s field engineers recently sent a link to a Channel 9 video: Steve Michelotti of e.magination on High Performance Web Solutions. This company built a 64-bit web server that handles over 3 billion transactions a day and guarantees a 250 ms response time. And it goes without saying that they built it on top of .NET.

The team optimized their code to avoid implicit allocations, pool and reuse objects and allocate large collections directly on the Large Object Heap. But they still saw some latencies that were greater than their contract allowed. So they worked with their field engineer and the GC team to get a new feature implemented in .NET 3.5 SP1 that let them work around the problem.

The problem comes from the size of their managed heaps. When you try to manage memory on a 16 GB server it can take a few seconds to run through the heaps. There’s not much you can do on a single machine besides keeping your heaps reasonably sized. Having millions of objects in memory is expensive. But their architecture is such that they can direct requests from the web server to any of a set of identical application servers. This means that they can take any individual server offline for the few seconds it takes to do a full collection. All they needed is to know when a GC is going to happen and when it’s completed. Full GC notifications is a new feature in 3.5 SP1 that gave e.magination necessary insight into the GC’s behavior that lets them redirect requests away from servers that are doing full GCs.

This video is only about 20 minutes long but it’s packed with great insights into how to make the most of your .NET server code. It shows the team at e.magination doing a fantastic job of analyzing and tuning their system for optimum performance. I found it interesting and inspiring. I hope you do too.

Automatically Capturing a Dump When a Process Crashes

Abhishek_Mondal — Fri, 16 Oct 2009 03:35:00 GMT

I recently received the following question from a customer:

“During our test runs (which might run for hours), if a process crashes, we’d like to create full memory dumps for later diagnosis. Can I configure the machine to do this automatically?”

I’ve actually gotten this question numerous times over the past couple of years. It comes in various forms but the general scenario is: A user wants to be able to diagnose application crashes and the environment does not lend itself to live debugging. The latter criterion could be for a number of reasons, but the most common I see are:

· We have an intermittent failure in production and want to gather a dump to debug the issue offline.

· I’m running a bunch of tests and when one crashes I don’t want to interrupt the whole run to diagnose the issue at failure time. Let’s just gather some information for triaging.

· Our issue isn’t reproducible under a debugger. E.g. a stress bug.

These essentially reduce to: You want to get as much data as you can, while minimizing the impact to the environment. Given these requirements, the solution I find meeting most people’s needs is to configure a just-in-time (JIT) debugger to launch, grab a dump, and exit whenever a process crashes.

Support for just-in-time debugging has been in the CLR since V1x and in the OS for as long as I can remember. The basic idea behind a JIT debugger is: when a process crashes, launch and attach a debugger so you can figure out why.

There are registry keys which provide this general ability for both managed and native code (support for the latter might actually be in Win.ini for 9x/Me). If your application is written in managed code (which is the case I’m often presented) you might ask, “My application is managed code, why do I care about native code?” Given that even the most simple managed applications run native code (e.g. the runtime itself), if your requirement is to gather data for any crash, you’ll need to set the keys for both types of code. In CLR V4 we’ve actually unified the key which controls the managed JIT debugger with the native one. However, that change does not change my guidance since, for the time being, we’ll be living in a world where V2 managed code exists alongside V4.

How do I configure the debugger?

1. Download and install the latest “Debugging Tools for Windows.”

a. If you’re running a 64-bit OS, you’ll want both the 32- and 64-bit versions*

b. You can either install the entire set of tools on the machine (it’s a quick, small install) or you can install to one machine and copy “cdb.exe” from the install directory to any number of target machines.

*Note: My sample .reg files below assume you install the 32-bit debugger to c:\debuggers\x86\ and the 64-bit version to c:\debuggers\x64\.

2. Create/set the following registry keys and values (If you’re working on a 64-bit version of Windows, you’ll need to set these keys under the Wow6432node as well.†):

a. Key: HKLM\Software\Microsoft\Windows NT\Current Version\AeDebug:

i. Value: “Debugger”

1. Type: String

2. Value data: <path to cdb> -pv -p %ld -c “.dump /u /ma <dump file path\name.dmp>;.kill;qd"

ii. Value: “Auto”

1. Type: String

2. Value data: “1”

b. Key: HKLM\Software\Microsoft\.NETFramework

i. Value: “DbgManagedDebugger

1. Type: String

2. Value data: <path to cdb> -pv -p %ld -c ".dump /u /ma <dump file path\name.dmp>;.kill;qd"

ii. Value: DbgJITDebugLaunchSetting

1. Type: DWORD (32-bit)

2. Value data: 2

†Note: You should set the keys to point to the appropriate “bitness” debugger. I.e. you want the OS/CLR to launch the 64-bit debugger for 64-bit process crashes and the 32-bit version for 32-bit crashes. Make sure your debugger paths are set accordingly.

The following sample .reg file will set cdb.exe to auto-launch and generate a crash dump for every process crash on the machine. Note the assumptions the file makes about debugger paths and the dump-file-placement path.

Windows Registry Editor Version 5.00

;This reg file installs just-in-time debuggers to capture a dump of all process
;crashes for the machine.

;

;Assumes 32-bit debugger is cdb.exe and is installed to c:\debuggers\x86\.

;Assumes 64-bit debugger is cdb.exe and is installed to c:\debuggers\x64\.

;

;Assumes crash dumps can be written to c:\crash_dumps\.

;Make sure all users have write access to this directory.

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework]

"DbgManagedDebugger"="\"c:\\debuggers\\x64\\cdb.exe\" -pv -p %ld -c \".dump /u /ma c:\\crash_dumps\\crash.dmp;.kill;qd\""

"DbgJITDebugLaunchSetting"=dword:00000002

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug]

"Debugger"="\"c:\\debuggers\\x64\\cdb.exe\" -pv -p %ld -c \".dump /u /ma c:\\crash_dumps\\crash.dmp;.kill;qd\""

"Auto"="1"

;The following keys are only used on 64-bit versions of Windows (note Wow6432Node).

;They can be safely created with no side-effects on 32-bit versions of Windows.

;Alternatively, you can delete the remainder of this file if you’re running a

;32-bit version of Windows.

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\AeDebug]

"Debugger"="\"c:\\debuggers\\x86\\cdb.exe\" -pv -p %ld -c \".dump /u /ma c:\\crash_dumps\\crash.dmp;.kill;qd\""

"Auto"="1"

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework]

"DbgManagedDebugger"="\"c:\\debuggers\\x86\\cdb.exe\" -pv -p %ld -c \".dump /u /ma c:\\crash_dumps\\crash.dmp;.kill;qd\""

"DbgJITDebugLaunchSetting"=dword:00000002

What do these keys do?

The “Debugger” and “DbgManagedDebugger” value data are basically command lines (printf format strings) that are run when a process crashes. The OS or CLR will substitute values for the format specifiers (e.g. in the case above it substitutes the crashing process ID for the “%ld”) and run the command in the user context of the crashing process. The command line I’ve supplied:

· launches cdb.exe, the debugger.

o Obviously, you must specify the correct path to the debugger.

· “-pv %ld” : Attaches non-invasively (just suspends the threads) to the crashing process (the OS or the CLR will actually fill in the PID for you).

· “.dump /u /ma <dump file path\name.dmp>”: Takes a full memory dump with a unique name (appends the date, time and process ID) and stores it in the path defined.

o The path and file name can be whatever you want them to be. Since the debugger is launched in the context of the crashing process, make sure the path points to a location all accounts can write to.

· “.kill”: Kills the target process (you’ve gotten the data you need).

· “qd”: Quits the debugger.

The “Auto” and “DbgJITDebugLaunchSetting” values set the policy for when to launch the debugger. As I wrote above, we want to get our data as quickly as possible and move on, so we don’t want to require any user intervention. For example, in the server scenario, there may be no one logged into the machine to click some “OK” button. The settings I described will automatically launch the registered debugger for all processes on the machine, without prompting (see Enabling JIT-attach Debugging for more details on the settings). Note that when these settings are in place, the debugger will be launched for crashes in all processes running on the machine and you will not have the option of submitting the crash through “Windows Error Reporting.” In future posts I’ll discuss ways of enabling both.

I don’t care about most processes on a machine. Can I just target specific ones?

The answer depends on the version of the OS and the version of CLR. Here are the rules:

· For native code: your OS must be Vista/Server 2008 or higher.

· For managed code: Your version of the CLR must be V4 (or higher, if you’re reading this from the future).

And here is how you configure:

1. Set the debugger keys (AeDebug\Debugger and .NETFramework\DbgManagedDebugger) just as you did in 2.a.i and 2.b.i, above.

2. Ensure AeDebug\Auto and .NETFramework\DbgJITDebugLaunchSetting are not set to auto-launch (again, see Enabling JIT-attach Debugging for more details on the settings).

a. Or you can delete them.

3. Create the following registry keys and values:

a. HKLM\Software\Microsoft\Windows\Windows Error Reporting\DebugApplications

i. Value: <Name of application executable> (e.g. “myapp.exe”)

1. Type: DWORD (32-bit)

2. Value data: 1

b. Repeat this for each application you want the debugger to be auto-launched.

You can set the DebugApplications key and values in HKCU if you prefer per-user control. When these settings are in effect the debugger will be launched for only the processes you specify and normal error handling will occur for all other processes (e.g. for most default configurations, a prompt to submit an error report to Microsoft).

The following sample .reg file will set cdb.exe to be auto-launched only for HelloWorld.exe. Replace HelloWorld.exe with the name(s) of the application(s) for which you’re interested in generating crash dumps.

Windows Registry Editor Version 5.00

;This reg file installs just-in-time debuggers to capture a dump of only the
;processes listed under the [DebugApplications] key, below.

;

;Assumes 32-bit debugger is cdb.exe and is installed to c:\debuggers\x86\.

;Assumes 64-bit debugger is cdb.exe and is installed to c:\debuggers\x64\.

;

;Assumes crash dumps can be written to c:\crash_dumps\.

;Make sure all users have write access to this directory.

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework]

"DbgManagedDebugger"="\"c:\\debuggers\\x64\\cdb.exe\" -pv -p %ld -c \".dump /u /ma c:\\crash_dumps\\crash.dmp;.kill;qd\""

"DbgJITDebugLaunchSetting"=dword:00000000

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug]

"Debugger"="\"c:\\debuggers\\x64\\cdb.exe\" -pv -p %ld -c \".dump /u /ma c:\\crash_dumps\\crash.dmp;.kill;qd\""

"Auto"="0"

;The following keys are only used on 64-bit versions of Windows (note Wow6432Node).

;They can be safely created with no side-effects on 32-bit versions of Windows.

;Alternatively, you can delete the remainder of this file if you’re running a

;32-bit version of Windows.

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\AeDebug]

"Debugger"="\"c:\\debuggers\\x86\\cdb.exe\" -pv -p %ld -c \".dump /u /ma c:\\crash_dumps\\crash.dmp;.kill;qd\""

"Auto"="0"

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework]

"DbgManagedDebugger"="\"c:\\debuggers\\x86\\cdb.exe\" -pv -p %ld -c \".dump /u /ma c:\\crash_dumps\\crash.dmp;.kill;qd\""

"DbgJITDebugLaunchSetting"=dword:00000000

;For each application you want the debugger to be auto-launched, add a row below

;similar to “HelloWorld.exe"=dword:00000001 but replacing HelloWorld.exe with

;your application .exe name.

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\DebugApplications]

"HelloWorld.exe"=dword:00000001

Can I do more than just capture a dump?

More information on the command-line switches for the Windows Debuggers can be found in the documentation included with the Debugging Tools for Windows. You’re not limited just to dumps. You can perform quite a bit of debugger automation if you choose.

So what happened?

Now that you have your dump, it’s time to figure out why your application crashed. For those of you familiar with dump debugging, at this point you’re about to fire up WinDbg + SOS (the managed debugging extension) and dive in. But wait! If your application runs on V4 of the CLR (.Net Framework 4.0) you can debug the dump in Visual Studio 2010. Our desire for the experience in VS 2010 is that it feels very similar to stopped-state live debugging (like you’ve hit a breakpoint). The fact that you’re often debugging optimized code can make this a bit trickier, but that’s a topic for another day.

For more information about how to debug crashes in managed code, a search for “managed dump debugging” yields quite a few helpful results. A great place to start is Tess Ferrandez’s blog (a support engineer at Microsoft). She has a number of great posts on the topic, including a post on dump debugging in VS 2010, and some labs/walkthroughs as well.

Jon Langdon,

PM, CLR

Five Myths about Managed Code

Abhishek_Mondal — Sat, 03 Oct 2009 03:41:40 GMT

My name is Immo Landwerth and I was a Program Manager intern this year in the CLR team. In this blog post I am not going to showcase any of the fantastic features that will ship with .NET 4.0 – my colleagues in the CLR team know them much better and already did a fabulous job discussing them here, over there and on Channel 9.

Instead I want to discuss the following five myths about managed code and in particular about the CLR:

· Managed code is always JIT compiled

· Generic co- and contra variance are new in .NET 4.0

· Everything is an object

· .NET only supports statically typed languages

· Microsoft is not using Managed Code

Myth Five – Managed code is always JIT compiled

Having a JIT compiler has many advantages because a lot of things are becoming much easier when a JIT compiler is available:

1. On-the-fly code generation (System.Reflection.Emit) is much easier because you only have to target one virtual machine architecture (IL) instead all the processor architectures the runtime supports (such as x86 and x64).

2. To some degree it solves the fragile base class library problem. That means we can share class definitions across modules without having the problem that changes such as adding fields or adding virtual methods crashes dependent code.

3. The working set can be improved because the JIT only compiles methods that are actually executed.

4. Theoretically, you could take situational facts into consideration, such as which processor-architecture is actually used (e.g. is it SSE2 capable), the application usage patterns etc. and optimize differently for them.

However, JIT compilation also has downsides such as:

1. It takes time. That means JIT compilation always has to trade-off time vs. code quality.

2. The code is stored on private pages so the compiled code is not shared across processes.

Therefore we created a tool called NGEN that allows you to pre-create native images during the setup. You could call this ahead-of-time compilation (as opposed to just-in-time). Certain special conditions left aside (such as some hosting scenarios or profiling), the runtime will now pick up the native images instead of JIT-compiling the code.

Why did we not allow you to pre-create the native images during build time and let you ship the native images directly? Well, because we then run into the fragile base class library problem mentioned above. In that case, your native images would get invalid every time the .NET Framework is updated. Today we solve this problem by re-running NGEN on the customer’s machine when the framework is serviced. In .NET Framework 4 we ship a new feature called targeted patching, that allows us for method-body only changes to minimize or to even to fully avoid recompilation. For more details about NGEN in general see here and for more details about NGEN in .NET Framework 4 see here.

Even if you are not using NGEN for you application code: for desktop CLR applications all the assemblies that are part of the .NET Framework itself are not JIT compiled – instead the runtime will bind to the native images. So even in these cases only your application code will be JIT compiled and therefore both ahead-of-time as well as just-in-time technologies are used simultaneously. Thus, stating that all code is JITted is simply wrong.

Myth Four – Generic co- and contra variance are new in .NET 4.0

The short answer is ‘no’. The longer answer is ‘well, sort of’.

But I am getting ahead of myself. Let’s first see what co- and contravariance actually means. Generic covariance allows you to call a method that takes an IEnumerable<Shape> with an IEnumerable<Circle> (if Circle is derived from Shape). This is useful if Shape contains, e.g. a method that allows you to compute the area. This way you can write a method that computes the area for any collection of shapes. Contravariance on the other hand allows you to call a method that takes an IComparer<Circle> with an IComparer<Shape>. This is handy if someone wants to compare circles and you already have created a general comparer for any shape (this works because if your comparer knows how to compare two instances of Shape it certainly is also able to compare two instances of Circle).

The support for co- and contra variance has always been in the CLR since generics came up in the .NET Framework 2.0. However, as Rick Byers pointed out you would have to use ILASM for creating covariant and contravariant type definitions:

In IL, covariant type parameters are indicated by a ‘+’, and contravariant type parameters are indicated by a ‘-‘ (non-variant type parameters are the default, and can be used anywhere).

What has been added in the .NET 4.0 release is language support for C# and Visual Basic. For example, the following uses the C# syntax (in and out modifiers for the generic type declaration) to create some covariant and contra variant types:

// Covariant parameters can be used as result types

interface IEnumerator<out T> {

T Current { get; }

bool MoveNext();

}

// Covariant parameters can be used in covariant result types

interface IEnumerable<out T> {

IEnumerator<T> GetEnumerator();

}

// Contravariant parameters can be used as argument types

interface IComparer<in T> {

bool Compare(T x, T y);

}

Myth Three – Everything is an object

“Wait a minute – this is the number one programming promise everyone was making about .NET!” you might say now. Yes, and yet it is false. Many .NET or C# books make this mistake in one form or the other. “Everything is an object”. Although we believe there is a lot of value in simplifying things for didactic reasons (and hence many authors just claim it that way) we would like to take this opportunity to tell you “sorry, it is not completely true”.

Before we discuss this issue we should first define what the sentence “everything is an object” is supposed to mean. The interpretation we will use here is this:

Every type is derived from a single root (System.Object). This means, that every value can be implicitly casted to System.Object. More precisely, this means that every value is representable as an instance of System.Object.

So why is this not true for the CLR? The counter example is a whole class of types that are not derived from System.Object: pointers (such as int*). So you cannot pass a pointer to a method that takes an object. In addition you cannot call the ToString or GetHashCode methods on a pointer.

We could also use a different interpretation of “everything is an object” such as:

Every type is derived from a single root (System.Object). This means, that every value is an object at all times.

Why is this different? Simple values (i.e.. values that have types derived from System.ValueType) are not objects by the definition of an object (they lack identity). But every value can be casted implicitly to System.Object (because System.ValueType is derived from System.Object). However, in that case an object instance that contains the value is created. This process is called boxing. The resulting object instance (the “box”, not to be confused with Don Box) has indeed a notion of identity (which is in particular also true for Don Box).

As you can see, the CLR uses the first interpretation and yet it is still not completely true as pointers do not derive from System.Object.

Myth Two – .NET only supports statically typed languages

It is true that the CLR uses a static type system. But this does not necessarily mean that it is only suited for programming languages that use a static type system. At the end, the programming language is implemented using the CLR but it is not identical with the CLR. So do not be fooled by the fact that the type system and mechanics of C# almost map directly to first class CLR-concepts. In fact, there are many concepts in C# that the CLR is not aware of:

1. Namespaces. As far as the CLR is concerned namespaces do not even exist. They are just implemented as type prefixes separated by dots (so instead of saying ‘the class Console is contained in the namespace System’ the CLR would just say ‘there is a class called System.Console’).

2. Iterators. The CLR does not provide any support for it. All the magic is done by the compiler (if you want to know, the compiler turns your method into a new type that internally uses a state-machine to track the current point of execution. Details can be found here).

3. Lambdas. They are just syntactic sugar. For the runtime these are just delegates, which in turn can also be considered syntactic sugar. In fact, a delegate is nothing more than a class derived from System.MulticastDelegate that provides an Invoke, BeginInvoke and EndInvoke method with the appropriate parameters.

Please note that this list is not complete. Instead it is only used to show you that even C# has to implement itself on top of the CLR and hence it is not a 1:1 mapping of the concepts the runtime provides. What does this have to do with static typing vs. dynamic typing? The answer is simply: you can implement a dynamically typed system on top of a statically typed system.

If you know see a huge business opportunity here, we have to disappoint you. Some smart people already had the same idea. This effort is called the Dynamic Language Runtime, or DLR for short. If you are like me then you immediately think of native code when someone mentions the term ‘runtime’. However, the DLR is completely implemented in C# and is just a class library that can be used by programming languages to implement dynamic systems on top of the CLR. The DLR shares the fundamental design principle of the CLR, i.e. it provides a platform for more than one language. That means you can share IronPython objects with IronRuby objects because they are implemented with the same underlying runtime (the DLR).

With .NET 4.0 the DLR ships as part of the box. So while .NET has first-class support for statically typed languages through the CLR it also provides first-class support for dynamically typed languages through the DLR.

Myth One – Microsoft is not using Managed Code

We often hear this (“Office and Windows are still not built on top of managed code!”) when customers ask about performance and future investments of Microsoft in managed code. The reasoning goes like this:

Since Microsoft is not implementing Windows and Office in managed code that means that it must be significantly flawed/runs much slower than native code and therefore their long term strategy will still be C++. This in turn means that we should not use managed code either.

In fact Microsoft has a huge investment in managed code (although it is still true that Office and Windows are not implemented in managed code). However, there are a bunch of products that are significantly (if not completely) implemented in managed code:

1. Windows components, such as

a. PowerShell

b. System Center

2. Office components, such as

a. Exchange

b. SharePoint/Office Server

3. Developer Tools, such as

a. Visual Studio and Visual Studio Team System

b. Expression

4. Dynamics

This list if by far not complete but it should be large enough to convince you that we are in fact ‘eating our own dog food’.

The reason that not all products are written in managed is not only related to performance. Sometimes the wins of re-implementing working native code in managed code do not outweigh its costs. On the other hand, there are still scenarios in which managed code simply cannot be used today (such as building the CLR itself or the debugger).

However, we will not deny that there are scenarios in which we cannot compete with the performance of native code today. But this does not mean that we have given up on this. In fact, projects like Singularity should show you that we are really very ambitious about redefining the limits of the managed world.

The last thing to keep in mind is that manually optimized assembler code is also faster than plain C-code. But this does not mean that all operating systems are completely written in assembler.

Thus our vision is more like this: native code where it makes sense, managed code where it makes sense with the bigger portion being managed.

IL Stub Diagnostic Tool

Abhishek_Mondal — Thu, 27 Aug 2009 09:08:50 GMT

The IL Stub Diagnostic Tool enables real-time inspection of the contents of IL stubs. Developers now have a powerful tool to troubleshoot issues in interop marshalling,

Introduction

Jesse posted a great blog talking about the concept, history, and improvements of Intermediate Language (IL) stubs for CLR v4. He mentions

The remainder of this post continues the discussion that Jesse started in the “Easier Debugging” section of his blog.

Get the tool

You are encouraged to download the tool from CodePlex.

Usage of the IL Stubs Diagnostics Tool

To begin using the tool, follow these basic steps:

1) Launch the IL Stub Diagnostic tool. Note: you need to be an administrator of the machine because the tool utilizes the Event Tracing for Windows (ETW) feature to receive ETW events.

There are three key features of the tool:

· Starting and Controlling the IL stubs monitoring session

The upper-left button, Start, is the most important aspect of starting a session. After clicking the Start button, the tool begins monitoring all the stubs generated by CLR on the machine. It will keep monitoring the IL stubs until you click it again (i.e., Start button will turn to Stop after it is clicked).

· Filtering to find a specific IL Stub

The IL Stub List section shows all the IL stubs collected by the tool. Sometimes you will receive a huge number of stubs. In this case, you can set several filters using the Filters feature to find the exact stub.

· Navigating and Inspecting the contents of an IL Stub

Once you find the IL Stub, the contents of it (the IL code) will be shown in the IL Code section. We provide some features like navigation to help you read and understand the IL code.

2) Start monitoring IL stubs by clicking the Start button.

3) Run a program involving interop. For example, the sample program sample\pinvoke.exe, which is included with the tool on CodePlex, generates a p/invoke from managed code to native code. Running this sample from the command line will generate several entries in the IL Stub List section. We will use this for illustration purposes below.

In this example, each item represents an IL Stub. Select an entry and notice its corresponding IL code shown in the IL Code Section.

Features which Enable Inspection and Navigation of IL Stubs

IL Stubs are generated on the fly by the CLR for Interop, and bridge the gap between managed and native methods. Since they are dynamically generated, it is difficult for developers to investigate or troubleshoot issues because the stubs are further JIT-compiled into machine code. The IL Stub Diagnostic tool eases the interop debugging experience by enabling the inspection of every stub. Let’s take a look at some of the key features which enable the inspection and navigation of IL stubs.

Detailed information of IL Stub

By default, the tool shows the following categories for each IL stub: MethodName, CreateTime, ProcessName, Category, ManagedSignature and NativeSignature. These categories are important for developers to identify an IL stub. In special cases (e.g., debugging with windbg), developers may also be interested in information like ModuleId and ManagedMethodToken. These are available by right clicking the table header of the IL Stub list.

IL Stub Filters

You may find that the CLR generates a lot of IL Stubs during the execution of a managed program. To quickly locate the stub we’re seeking, filters can be set to reduce the number of IL stub entries in the IL Stub list. Using the pinvoke.exe sample (included with the tool on CodePlex), you can use the following filter:

Once the filter has been entered, click the Refresh button to obtain the filtered list.

To remove a filter, you can right click on the row header of the first filter condition and select Delete Filter. It will take effect after clicking Refresh button.

Navigation buttons for IL Stub code

The code may not be well-formatted for first time display. To make the code more reader-friendly, the tool provides navigation buttons that allow you to view the code block by block. As you can see, each IL stub contains the following color-coded blocks: Initialize, Marshal, the calling method, and Unmarshal blocks. In the IL code, StubHelpers are used to do some complex work. Most of the functionality of each stub helper can be figured out by its name. The tool also provides a Stub API button to look up its usage.

Please note that stub helpers are internal to the runtime and their names/signatures/description should only be used for informational purposes. We may change any of these in the next version of the CLR so no one should establish any dependencies on them.

Troubleshooting marshalling issues with IL Stub Diagnostic tool

As mentioned earlier, developers can use the tool to investigate and debug marshalling issues. Here is an example.

Case description

There is an issue with the following p/invoke scenario. The signature of the native method is:

void DumpString(WCHAR *pStr)

{

printf("%ws", pStr);

}

In managed code, we declare it as:

[DllImport("testNative.dll")]

public static extern void DumpString(string str);

When invoking from managed side with the following code, it shows nothing on the console.

DumpString("Hello World");

Our job is to discover what happens during the p/invoke call.

Troubleshooting

Since the native code simply dumps the string, the problem must be occuring during the marshaling phase.

First, locate the stub by following these steps:

1. Launch IL Stub Diagnostic tool.

2. Click Start Button.

3. Run the program again.

4. Set the following filter as shown below and click the Refresh button.

Next, navigate to the block of code which handles the marshaling by following these steps:

1. Click the “IL Code” button to get a bigger view.

2. Click the “Next Block” and navigate to the Marshal block.

3. Usually, StubHelpers APIs are the most informational code here. Move the mouse over it to see the explanation of the method (shown in the green box below):

4. The description tells us that the StubHelpers method is used to convert an Ansi string. However, our native method expects a WCHAR string, which results in the marshaling issue. The fix is to specify Unicode, as the highlighted code below.

[DllImport("testNative.dll", CharSet=CharSet.Unicode)]

public static extern void DumpString(string str);

Summary

As you can see, the IL Stubs Diagnostic Tool provides a window into what used to be a black box, and allows you to look into the very core of interop marshaling – the IL Stub itself.

Acknowledgements

We would like to specially thank Yifeng Fu for being one of the key developers of IL Stub Diagnostic tool during his internship with us.

See Also

The MSDN topic “MSIL Stub Customization” contains detailed reference and usage information.

Yongtai Zhu,

Developer, CLR.

The good and the bad of exception filters

Abhishek_Mondal — Wed, 26 Aug 2009 03:04:00 GMT

Every so often we get asked questions about the CLR’s support for exception filters. Why do some languages support them and others do not? Is it a good idea to add them to my new language? When should I use a filter vs. catch/rethrow? Etc. I’ll try to answer some of these questions for you here, and while I won’t go into all of them hopefully you’ll walk away with enough info to form your own opinion on the rest. Like so many things there’s good things and bad things about exception filters…

So what’s a filter?

The CLR provides a number of exception handling primitives that higher level languages can build upon. Some are fairly obvious, and map readily to language constructs that most of us know and love: try/catch and try/finally, for instance. I’d hazard to guess that everyone knows what those do, but just in case, let’s consider a quick example in C#:

try

{

try

{

Console.Write(“1”);

if (P) throw new ArgumentException();

}

finally

{

Console.Write(“2”);

}

catch (ArgumentException e)

{

Console.Write(“3”);

}

Console.Write(“4”);

If P is true, then that will print out “1234”, of course. If P is false, then it will print “124”. Groovy.

But the CLR also provides two more EH primitives: fault and filter. A fault clause is much like a finally clause; it runs when an exception escapes its associated try block. The difference is that a finally clause also runs when control leaves the try block normally, whereas the fault clause will only run when control leaves the try block due to an exception. In the case above, if we replaced the “finally” with “fault” (there’s no C# syntax for that, but suspend your disbelief for a moment) then it would print “1234” if P is true, and “14” is P is false. See the difference? Most languages don’t expose this as a first-class language construct, but we do have a few that use fault clauses under the covers for specific scenarios.

So that leaves us with filters. I suppose the simplest definition of a filter is that it is a construct that allows one to build a conditional catch clause. In fact, that’s exactly what VB uses filters for. Let’s consider a more complicated example in VB:

Function Foo() As Boolean

Console.Write("3")

Return True

End Function

Sub Main()

Dim P As Boolean = True

Try

Console.Write("1")

If (P) Then

Throw New ArgumentNullException()

End If

Console.Write(“P was False!”)

Finally

Console.Write("2")

End Try

Catch ex As ArgumentNullException When Foo()

Console.Write("4")

Catch ex As ArgumentException

Console.Write("5")

End Try

Console.Write("6")

End Sub

Here you’ll note the “Catch ex As ArgumentNullException When Foo()” line is a conditional catch statement. The catch handler will only execute and print “4” when the exception is an ArgumentNullException and when Foo() returns true. If Foo() returns false, then the catch clause doesn’t execute, and the system continues to search for a catch clause that can handle the exception. In this case, the very next clause would handle the exception, and print “5”.

So, what do you think this program prints? Don’t cheat by attempting to compile and run it! Using what you know about exception handling and looking at the program structure and syntax, what would you imagine this program prints? I suspect most people would guess “12346”. I even gave you a clue with the numbering.

I think most of us would look at the example above and conclude that the result should be “12346” because when we look at the syntax above we, quite rightly, see lexically scoped language constructs. We expect that when the code in the inner finally clause starts executing that no more code anywhere in the associated try block will execute. For instance, in the example above, if P is true then when we enter the Finally we know that no more code will execute in the try block, and that we’ll never print “P was False!”. Likewise, when we evaluate one of the catch clauses, we expect that all the code in the associated try block is done executing.

And here comes the bad…

It turns out that program actually prints “13246”. My clue with the numbering was an evil ruse. After the throw, Foo() is executed first as part of evaluating the first catch clause, and then the finally within the associated try block executes. And that’s just freaky… what happened to our lexically scoped language constructs?!

This is a surprising result for most. It breaks our intuitive reasoning about the language based on the lexically scope exception handling constructs provided. Here, when we evaluate the conditional catch clause, all the code in the associated try block has not, in fact, finished executing.

Why does this happen?

The reason we see “3” before “2” is subtle, and founded in the CLR’s implementation of exception handling. The CLR’s exception handling model is actually a “two pass” model. When an exception is thrown, the runtime searches the call stack for a handler for the exception. The goal of the first pass is to simply determine if a handler for the exception is present on the stack. If it sees finally (or fault) clauses, it ignores them for a moment.

The handler may be in the form of a typed catch clause, i.e., “Catch ex as ArgumentException)”. When the runtime sees a typed catch clause, it can determine itself if this clause will handle the exception by performing a simple check to determine if the exception object is of a type that inherits from (or is) the type in the clause.

But when the runtime sees a filter, it must execute the filter in order to determine if the associated handler will handle the exception. If the filter evaluates to true, then a handler has been found. If it evaluates to false, then the runtime will keep searching for a handler.

Once a handler has been found, the first pass is over and the second pass begins. On the second pass, the runtime again runs the call stack from the point of throw, but this time it executes all finally (or fault) clauses it finds on the way to the handler it identified on the first pass. When the handler is reached, it is executed, and the exception has finally been handled.

But why is that bad?

“Okay”, you say, “I get it. I understand that filters run during the first pass. I can deal with that… what’s the big deal?” Well, let’s first consider what a finally clause is for. We typically use finally clauses to ensure that our program state remains consistent when exiting a function even in the face of an exception. We put back temporarily broken invariants. Consider that the C# “using” statement is built using try/finally, and then consider all the things one might do with that.

But when your filter runs, none of those finally clauses have executed. If you called into a library within the associated try block, you may have not actually completed the call when your filter executes. Can you call back into the same library in that case? I don’t know. It might work. Or it might yield an assert, or an exception, or, well, your guess is as good as mine. The point is that you can’t tell.

Using filters wisely (or, “the good”)

But the notion of a conditional catch clause really is quite appealing, and there are ways to use these without getting caught by the problems of when the filter actually executes. The key is to only read information from either the exception object itself, or from immutable global state, and to not change any global state. If you limit your actions in a filter to just those, then it doesn’t matter when the filter runs, and no one will be able to tell that the filter ran out of order.

For instance, if you have a fairly general exception, like COMException, you typically only want to catch that when it represents a certain HRESULT. For instance, you want to let it go unhanded when it represents E_FAIL, but you want to catch it when it represents E_ACCESSDEINED because you have an alternative for that case. Here, this is a perfectly reasonable conditional catch clause:

Catch ex As System.Runtime.InteropServices.COMException When ex.ErrorCode() = &H80070005

The alternative is to place the condition within the catch block, and rethrow the exception if it doesn’t meet your criteria. For instance:

Catch ex As System.Runtime.InteropServices.COMException

If (ex.ErrorCode != &H80070005) Then Throw

Logically, this “catch/rethrow” pattern does the same thing as the filter did, but there is a subtle and important difference. If the exception goes unhandled, then the program state is quite different between the two. In the catch/rethrow case, the unhandled exception will appear to come from the Throw statement within the catch block. There will be no call stack beyond that, and any finally blocks up to the catch clause will have been executed. Both make debugging more difficult. In the filter case, the exception goes unhandled from the point of the original throw, and no program state has been changed by finally clauses.

The problem is that we rely on programmer discipline in order to use filters correctly, but it’s easy to use them incorrectly and end up with infrequently executed code (exceptions, after all, are for exceptional circumstances) that has subtle and hard to diagnose bugs due to inconsistent program state that should have been cleaned up by finally clauses further down the stack.

Why does the CLR use a two-pass exception handling model?

The CLR implements a two-pass exception handling system in order to better interoperate with unmanaged exception handling systems, like Win32 Structured Exception Handling (SEH) or C++ Exception Handling. We must run finally (and fault) clauses on the second pass so they run in order with unmanaged equivalents. Likewise, we must not execute filters later (say, on the second pass) because one of those unmanaged systems may have remembered that it was supposed to be responsible for handling the exception. If we were to run a filter late on the second pass and decide that a managed clause really should catch the exception after previously having not declared that on the first pass, then we would violate our contract with those unmanaged mechanisms with unpredictable results.

So, in short, it’s for interop, and like many things involving interop, we have a compatibility burden that we can’t ignore.

Would a one-pass model be better?

Many have wondered over the years if perhaps the two-pass model in general is bad, and if a one-pass model would be better. Like so many things in the world, it’s just not that clear. A one-pass model would simplify the exception handling implementation, and it would make more sense in the cases shown above. However, there are advantages to the two-pass model that can’t be ignored. Perhaps the most important one is that if the search for a handler fails on the first-pass the exception goes unhandled and in general no program state has changed, even though filters are run since filters tend not to change things. The call stack is still intact, and all values that lead to the exception are still present on the stack and on the heap (assuming no race conditions.) This is frequently essential when debugging an unhandled exception. In a one-pass model, all of the finally clauses would have been run before the exception goes unhandled.

Wrapping up

Of the languages that MS ships, only VB and F# support filters, and both do so via conditional catch clauses. In F#, you have to really go out of your way to attempt to inspect mutable global state, or to actually have a side effect, so you’re fairly safe there. In VB, though, you can call a function from the “when” clause of their catch statement, and in there you have free reign to do whatever you please. You can, without a doubt, get yourself into trouble attempting to do too much complicated work within such a filter. To keep your world safe and simple try to limit yourself to expressions that only access the exception object, and don’t modify anything. If you go beyond that, you need to consider carefully all the code executed in the try block, and if your actions in the filter will work if the backout code below has not finished executing yet.

Mike Magruder,

Developer, CLR.

Improvements to Interop Marshaling in V4: IL Stubs Everywhere

Abhishek_Mondal — Mon, 10 Aug 2009 23:27:15 GMT

When the CLR needs to transition between managed and native code – usually because of P\Invoke or COM interop – we need to generate marshaling stubs (little chunks of code) to handle that specific call and transform the data from managed to native format and back again. . These stubs are little pieces of code that are usually generated at runtime to do their work behind the scenes and, hopefully, without making developers or users aware of their presence. In .Net 4 we’ve improved our marshaling infrastructure substantially with a feature we call “IL Stubs Everywhere.” For those who have a deep interest in interop marshaling we get into the details of this feature below. For everyone else, here are a few key benefits of this feature:

· Faster interop marshaling: the more complex the signature the greater the speed-up

· x86 and x64 behavior matches: we’ve updated the x64 marshaling to behave exactly as x86 always has and mostly without impact to x64

· Better debugging: when something goes wrong in marshaling we now give you the ability, and specialized tools, to find the problem

History

The 1.0 and 1.1 versions of the CLR had several different techniques for creating and executing these stubs that were each designed for marshaling different types of signatures. These techniques ranged from directly generated x86 assembly instructions for simple signatures to generating specialized ML (an internal marshaling language)* and running them through an internal interpreter for the most complicated signatures. This system worked well enough – although not without difficulties – in 1.0 and 1.1 but presented us with a serious maintenance problem when 2.0, and its support for multiple processor architectures, came around.

We realized early in the process of adding 64 bit support to 2.0 that this approach was not sustainable across multiple architectures. Had we continued with the same strategy we would have had to create parallel marshaling infrastructures for each new architecture we supported (remember in 2.0 we introduced support for both x64 and IA64) which would, in addition to the initial cost, at least triple the cost of every new marshaling feature or bug fix. We needed one marshaling stub technology that would work on multiple processor architectures and could be efficiently executed on each one: enter IL stubs.

With x64 and IA64 bit support in v2 we introduced IL stubs to perform the marshaling on these platforms. With IL stubs we generate actual IL rather than assembly code or the specialized language in our ML stubs. On these platforms we only used the IL stubs and were thus able to have a single implementation, for all stubs, across both processor architectures. These IL stubs were much faster than the ML stubs but were also much slower than x86 stubs. Because of the performance regress and our resource constraints, we left the x86 platform with its original marshaling stubs implementation.

New in V4: IL Stubs Everywhere

One of the largest features the interop team worked on in the v4 product was the “IL Stubs Everywhere” feature. With this work we moved to uniformly using the same IL stubs infrastructure for all marshaling on all platforms. Sometimes when we spend a lot of time developing, or reworking, a feature there is a concern that it will be expensive for developers to start taking advantage of it: the good news with this one is that this will change will occur automatically and the majority of the benefits will happen just by running your app on v4.0.

A few of the benefits you will see with this feature are:

Increased performance:

The original x86 stubs for the simplest signatures were pretty fast – often a simple copy operation -- but as soon as you needed marshaling for more complex signatures you were forced onto the slow path with the ML interpreter. We heavily optimized our IL stub infrastructure and now IL stubs are faster across the board (even compared to x86 stubs) and are orders of magnitude faster than the interpreted stubs you used to get for the more interesting signatures. Additionally, for those that use NGEN you’ll find that we include most of your IL stubs in the NGEN images to improve this scenario even further.

Consistent behavior:

One of the common problems people ran into in interop in the v2 days is that with different marshaling technologies used for 32 and 64 bit the behaviors didn’t line up exactly the same. There were a few places where the same code behaved differently based on platform. With every platform running off of IL stubs we now have the same behavior across all platforms.

Note: when we did this we had a choice between giving the 4.0 IL stubs the behavior of the v2.0 64 bit IL stubs or the v2.0 32 bit stubs. We chose to use the 2.0 32 bit behavior for the 4.0 because of the much larger percentage of interop apps that have been built on 32bit to date.

This means that you should not see any behavioral differences moving a 32 bit app to the v4.0 NetFX but you may see some for 64 bit interop apps. Because of the nature of these changes we expect the impact of them to 64 bit apps to be very minimal: most of the changes involve relaxing behavior that was stricter in 64 bit than 32 bit. The biggest change in reverting to 32 bit behavior was to stop clearing ‘out’ parameters during certain error conditions and should be completely non-breaking to 64 bit applications.

Easier Debugging:

One of the other problems with the old 32 bit marshaling infrastructure was that it worked essentially as a black box that was very difficult to debug if something went wrong. It was very difficult to figure out how/what the runtime was trying to marshal when something went wrong. By moving to IL stubs it is actually possible to see the IL instructions that we’re executing and determine where your problem is.

Since Visual Studio does not support debugging IL, the move to IL stubs alone wouldn’t help debugging very much if that was all we did. Instead we updated our IL stubs infrastructure to fire an ETW event detailing each stub as they get generated and built a tool, which we recently released on codeplex, which lets you easily see all the stubs being generated and quickly search for the ones for the methods you are most interested in. In an upcoming post we will have more details and a walkthrough of the tool, but in the meantime you can find it here: http://clrinterop.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=29745

*ML Stubs: it is a little known fact that up until v4 the CLR, at runtime, converted complex signatures into an internal CLR language called ML (marshaling language) which it then ran through the CLR’s internal ML interpreter each time the method was invoked. This little interpreter is almost another separate runtime just for interop marshaling. You can imagine how replacing this with IL and thus getting all the advantages of the perf work that goes into NGEN and the JIT allowed us to massively speed up these scenarios. There was much rejoicing on our team when we removed this interpreter in v4.

Jesse Kaplan,

PM, CLR

CLR 4: Making the AssemblyResolve event more useful

Abhishek_Mondal — Thu, 16 Jul 2009 01:04:00 GMT

In the introductory post on CLR Binder (‘Understanding the Binder – Part 1’), we listed the set of steps that the CLR Binder follows, in order to locate an assembly and bind to it. On reading this, an obvious question comes to mind. What happens when all of these steps fail to locate the assembly? Does the binder simply quit looking?

It eventually does, but not before firing the AssemblyResolve event. The user can register an event handler for the AssemblyResolve event and then load the assembly that was intended to be loaded in the first place (or execute some other code appropriately).

The AssemblyResolve event itself has been around for a while now. So what’s changed in CLR 4? Prior to CLR 4, if an assembly A has a reference to another assembly B, and an AssemblyResolve event occurs for the referenced assembly (in this case, B), there is no means to know the identity of the parent assembly (or the referencing assembly, or A).

Why is this problematic? Let’s take this example. Let’s assume that an assembly FirstParent.dll references Child.dll. Let’s also assume SecondParent.dll also references Child.dll. On failing to load Child.dll, AssemblyResolve event is fired.

Now, while loading FirstParent.dll and SecondParent.dll using LoadFile(), an AssemblyResolve event is fired for Child.dll. Looking at the AssemblyResolve event, it is unclear as to which parent assembly actually triggered loading Child.dll. This is not helpful if the user wants to execute different code as a part of the ResolveEventHandler, depending on the parent assembly that caused attempting to load Child.dll.

In CLR 4, the AssemblyResolve event provides richer information that includes the parent assembly’s identity. System.ResolveEventArgs now has a property called RequestingAssembly, which is the assembly which requested the (unsuccessful) load of the child assembly, triggering an AssemblyResolve event.

With this new property, when AssemblyResolve event is fired, apart from the current assembly (which could not be located by the Binder), the parent assembly is also provided.

Thus, using this property,

(a) The user now knows which assembly caused the load event

(b) The event handler can now make use of the parent assembly that is passed.

You can read more about this new property here.

Enabling CAS Policy Compatibility Mode for a Project

Abhishek_Mondal — Wed, 24 Jun 2009 21:48:00 GMT

If you’re migrating a VS project over to VS 2010 Beta1 and you are calling code that is passing Evidence objects to the runtime expecting security policy resolution, or if you’re using a permission Deny, you may run into the following exception:

NotSupportedException: This method uses CAS policy, which has been obsoleted by the .NET Framework. In order to enable CAS policy for compatibility reasons, please use the legacyCasPolicy configuration switch. Please see http://go2.microsoft.com/fwlink/?LinkId=131738 for more information.

What does this mean? In earlier versions of the .NET Framework, Code Access Security (CAS) Policy was the primary mechanism for defining whether or not certain classes of managed applications were able to perform security sensitive operations or not. While powerful, CAS Policy was extremely difficult to use and manage – developers and administrators would have to set up and analyze complicated policy trees to determine what their code would be granted. Furthermore, these restrictions could be circumvented by simply using unmanaged code!

We’ve provided cleaner, more appropriate mechanisms for managed code security in .NET Framework 4; those will be described in detail in future posts. CAS Policy has been rendered obsolete as a result of this, hence the exception messages. It is, however, still there for those migrating apps and projects that still require CAS policy in the meantime.

Note that by enabling CAS for your project, you’re subjecting yourself to the CAS security policy decisions of the runtime. If you made any changes (via caspol.exe, etc) to CAS policy in previous versions of the runtime, these will not be reflected in .NET Framework 4 even with CAS policy turned on. Your custom policy will have to be reconfigured with the .NET Framework 4 caspol.exe.

If you’re using CAS policy implicitly (for example, an Assembly.Load with Evidence) or explicitly (SecurityManager.PolicyHiearchy), check out Shawn Farkas’ blog for guidance on the new, supported mechanisms for sandboxing and running partial trust code.

Follow these steps to enable CAS Policy for your project:

1. Go to “Project” in the menu bar, then “Add New Item…”

2. Select “Application Configuration File” from the list of items available.

3. Enable CAS Policy via the following switch:

</runtime>

</configuration>

4. CAS Policy is now enabled and enforced for this project.

If you’re simply trying to get loads from remote sources (i.e. Assembly.LoadFrom(@"http://...")) to work, you don’t need to enable CAS Policy to do this – replace legacyCasPolicy with loadFromRemoteSources in the example above and you’ll be able to load from any location. Please do note that these assemblies will be loaded as fully trusted! You can find more information on this switch here.

Getting more information than the exception class provides

Abhishek_Mondal — Sat, 20 Jun 2009 07:59:00 GMT

We recently had a question about how to get more information than an exception’s type provides. The developer was trying to copy a file and didn’t know why the copy was failing. File copies can fail for many reasons, almost all of them what Eric Lippert calls “exogenous conditions”. The developer was catching System.IOException and parsing the error message. It worked on his machine, but failed during test passes.

Unfortunately, the .NET Framework doesn’t always directly provide the reason that the file copy failed. If an error results from an IO operation the .NET Framework will call GetLastError to find out why the operation failed, then throw an IOException with a message explaining the problem: sharing violation, path too long, etc. The error message is formatted in such a way that you can expose it directly to your user: it clearly states the problem and gets translated into the user’s OS language.

This is a little annoying if you’re trying to programmatically respond to the error. For example, if there’s a sharing violation you might want to wait for a few seconds then try again. How do you tell that this IOException comes from a sharing violation as opposed to a disconnected USB drive or a network failure?

You do NOT want to parse the exception message because exception messages get translated into the user’s OS language. You can’t programmatically parse a localized string without doing a lot of unnecessary work. Moreover, the exception message could change slightly between different versions of the .NET Framework. While we don’t rewrite exception messages just for fun, one day someone could complain that the error is slightly misleading and we’d fix it, breaking your parsing code. So what can you do if you want a more robust way to get more information than the exception code gives you?

The answer is pretty simple: call Marshal.GetLastWin32Error yourself in your catch block and look up the system error code on MSDN. Reading the error doesn’t clear it so you can call GetLastWin32Error even after the Framwork API does.

Here’s some sample code that illustrates how to use GetLastWin32Error. It relies on the fact that the default file handle on Windows is an exclusive handle so you can’t call File.Open twice on the same file without having closed it between the two calls.

using System;

using System.IO;

using System.Runtime.InteropServices;

class Program

{

static void Main()

{

File.Open("foo.txt", FileMode.OpenOrCreate);

try

{

// This call will fail due to a sharing violation.

File.Open("foo.txt", FileMode.Open);

}

catch (IOException e)

{

// A P/Invoke here would overwrite last the error.

int error = Marshal.GetLastWin32Error();

Console.WriteLine("Exception message is: " + e.Message);

Console.WriteLine("Win32 system error code is " + error);

}

This code prints out two pieces of information: first, that the process cannot access the file “foo.txt” because it is being used by another process, and second, that the Win32 error code is 32. Looking it up online, 32 means ERROR_SHARING_VIOLATION. If you’re passing this information on to your end user, the exception error message is already formatted for you. If you need to know the error programmatically, you can switch on the error code. (Note to the nitpickers: yes, the file’s being used by the same process, not another process as the error code says. The BCL team believes that you’re smart enough not to try to open the same file twice in your own code so they’re giving you the benefit of the doubt and blaming some other process. But this is a fantastic example of why you shouldn’t parse error codes: maybe we’ll fix this one to say that “this process or another process” has the file open already.)

This is a simple answer, but it’s also a bit too simple. GetLastWin32Error returns the error code of the last native function executed on the thread. (Technically, the last native function that sets the error code, but let’s assume they all do.) If the call to File.Open was the most recent native function call on this thread, then everything is cool. If you P/Invoke—or if a library function that you call P/Invokes—then you won’t get the error code from the File.Open call.

The code I showed above will work fine because it is so simple. But because exceptions can propagate far out of the context from where they were raised you can easily run into problems. For example, say that instead of calling File.Open yourself, you call a function that will open the file’s directory before opening the file—call it OpenFileInDirectory. When the exception is raised, OpenFileInDirectory’s finally clause executes and closes the directory. Now the finally clause has reset the Win32 error code so when you call GetLastWin32Error in your catch clause it returns success—the error value of closing the directory.

So what should you do to get more information from an exception than its type tells you? Parsing the exception message is definitely the wrong thing to do. We’ve had suggestions on Connect that we provide exception subtypes that add more information to an exception. Essentially, when we call GetLastError, we’d pass that error on to the programmer through something like Exception.SubType or Exception.Win32Error. This is a great suggestion, and it's something the BCL team is considering for a future release.

If you find an exception code that doesn’t give you all the information you need, enter a suggestion on Connect. If there’s one there already, vote it up. The BCL team reads all Connect issues and tries hard to get the top-rated suggestions into future releases. There's always more work to do than there is time available so we try to prioritize the things that matter most. If this issue matters to you, let the team know!

Andrew Pardoe

Program Manager, CLR EH team

Tail Call Improvements in .Net Framework 4

Abhishek_Mondal — Sat, 20 Jun 2009 00:42:00 GMT

Grant Richins has a post on the CLR Code Generation team blog that covers the work done to improve the x64 JIT to honor the "tail." instruction prefix, thereby making functional languages like F# more viable. You can find the full post here.

What's New for .Net Framework Security

Abhishek_Mondal — Thu, 11 Jun 2009 22:16:00 GMT

Shawn Farkas, CLR security developer, has started a cool series of posts on what’s new in .NET Framework 4 security. If you’re doing any work with or have scenarios involving partially trusted code, you’ll want to take a look. Topics include sandboxing, Code Access Security (CAS) Policy, and a nifty Channel 9 video. Check it out!

New Security Model: Moving to a Better Sandbox

Abhishek_Mondal — Thu, 11 Jun 2009 04:56:00 GMT

For .Net Framework 4, we decided to remove the dependency on caspol and the policy levels and make things simpler.

With this change, the default grant-set for assemblies is now FullTrust unless the host (such as InternetExplorer) decides to load them in a sanbox. We also made CodeAccessPermission.Deny obsolete. This MSDN article describes the changes we made in the security namespace for .Net Framework 4 in detail.

For most people, this change in policy will be unnoticeable, their program probably was already running in full-trust and the world is the same. For some others, things will be a lot easier: launching their tool from the companies’ intranet share is now possible without the need to change the .Net Framework policy. For an even smaller set of users, there will be some issues that they will encounter, they were probably expecting assemblies to be loaded as partial trust and now they are full-trust.

In this post, we will cover how programs can be migrated to the newer security model and still work in the way they were intended to work.

In versions of .Net Framework before v4, we had many ways to restrict the permissions of an assembly or even certain code path within the assembly:

1. Stack-walk modifiers: Deny, PermitOnly

2. Assembly-level requests: RequestOptional, RequestRefuse, RequestMinimum

3. Policy changes: caspol, and AppDomain.SetPolicyLevel

4. Loading an assembly with a Zone other than MyComputer

In the past, these APIs have been a source of confusion for host and application writers. In .Net Framework 4, these methods of restricting permissions are marked obsolete and we hope to remove them at a point in the future. The .Net Framework 4 throws NotSupportedException when encountering calls to functions allowing any of these sandboxing methods. Applications that used these sandboxing APIs will now see an exception similar to this:

System.NotSupportedException: The Deny stack modifier has been obsoleted by the .NET Framework. Please see http://go2.microsoft.com/fwlink/?LinkId=131738 for more information.

Here is what you can do, after identifying that your code uses one of the previously described sandboxing methods:

1. Execute the partial trust code inside a partial-trust AppDomains. This approach might appear difficult because it asks you to figure out what trust levels your application needs. Having part of your application running in another AppDomain also requires some consideration about how to do the communication with the objects residing in the new AppDomain. This model might be more complex than just a command that changes the machine policy, but we think it is a better one. This article provides more information about why this sandboxing strategy is better.

Here are the steps for creating a new sandboxing AppDomain:

1.1. Remove the Deny, assembly level requests or the caspol command from your application.

1.2. Create a new partial-trust AppDomain with a partial-trust grant set. This is done by calling the override for AppDomain.CreateDomain that receives a grant-set and a full-trust StrongName list:

public static AppDomain CreateDomain(string friendlyName, Evidence securityInfo, AppDomainSetup info, PermissionSet grantSet, params StrongName[] fullTrustAssemblies)

The MSDN article talking about this specific override is located here

This would create a new AppDomain which would give assemblies by default the PermissionSet given as a parameter. Assemblies that have their StrongName present in the fullTrustAssemblies parameter, will receive FullTrust as the grant-set.

1.3. Create an instance of one of your classes inside the new AppDomain. Your class would have to inherit from MarshalByRefObject. The API to use is this:

public object CreateInstanceAndUnwrap(string assemblyName, string typeName)

The MSDN article talking about this specific override is located here.

This API will return you a reference of type object to an instance of a class inside the new AppDomain. You would have to convert that instance to your specific type, so you would be able to call functions present in your class on it.

1.4. Call a function on your newly created instance. This function call would be marshaled across AppDomain boundaries and will be executing in the new AppDomain, thus in partial trust.

Example:

PermissionSet ps = new PermissionSet(PermissionState.None);

ps.AddPermission(new SecurityPermission(SecurityPermissionFlag.Execution));

//Create a new sandboxed domain

AppDomainSetup setup = AppDomain.CurrentDomain.SetupInformation;

AppDomain newDomain = AppDomain.CreateDomain("test domain", null, setup, ps);

//Create an instance in the new domain.

//The class has to derive from MarshalByRefObject. We consider

//PartialTrustTest to be such a class

PartialTrustTest remoteTest = newDomain.CreateInstanceAndUnwrap(

typeof(PartialTrustTest).Assembly.FullName,

typeof(PartialTrustTest).FullName) as PartialTrustTest;

remoteTest.EntryPoint();

This will get slightly more complicated if you would have 2 assemblies, where one needs to be run in FullTrust and one in PartialTrust. The way to do this is to sign with a key (obtained by caling sn –k) the FullTrust assembly, and pass it’s strong name as the last parameter to AppDomain.CreateDomain.

Example:

…

//We consider FullTrustAssm to be the name of the assembly that needs to be

//executed as a full-trust assembly

AppDomain newDomain = AppDomain.CreateDomain("test domain", null, setup, ps,

Assembly.Load("FullTrustAssm").Evidence.GetHostEvidence<StrongName>());

…

2. Sandboxing AppDomains might seem complicated and daunting, especially if all you need is to launch an entire executable in PartialTrust and you don’t want to know all the details of AppDomain creation and communication.

If this is the case, we recommend the PTRunner tool. If you are going to take this route, you still need to remove the Deny, assembly level requests or the caspol command. In order to be able to use this tool, your application has to be launched from the command prompt. The tool is present in the CLR Security codeplex site at project PTRunner.

So let’s say your test is located in one assembly called “partialTrustAssembly.exe” that used to have Deny. You remove the deny and call PTRunner. Under the covers the PTRunner tool sets a sandbox AppDomain for you and launches your application in it.

Example:

PTRunner partialTrustAssembly.exe

Now let’s say, you want one of the assemblies your executable references to be full trust. You would do something like this:

PTRunner –af fullTrustAssembly.dll partialTrustAssembly.exe

You could also do something like this.

PTRunner –af FullTrustAssembly.exe FullTrustAssembly.exe .

This is rather fancy but it runs your assembly as full trust in a partial trust AppDomain. This would allow you to do operations allowed only to full-trust code, like assert, but at the same time run in a partiual-trust AppDomain so demands would still fail.

Or perhaps you don’t like the minimum execution permission that your assembly is run under. You could do something like this:

PTRunner –ps Internet partialTrustAssembly.exe

Or perhaps you want your very own permission set that is non-standard. You would write the permission in XML and call PTRunner like this:

PTRunner –xml Permission.xml partialTrustAssembly.exe

The xml file contains the serialization to XML of a permission set.

3. If you don’t have the luxury to launch a runner that, in turn, launches your test, another way to launch code in partial-trust is to use the SandboxActivator. This is located in the CLR Security codeplex site in the Security 1.1 library and will help you set up a new AppDomain in an easier way. This is basically a wrap around ApDomain.CreateDomain and AppDomain. CreateInstanceAndUnwrap from the example above about sandboxed domains and it returns you just an instance of your type in the new AppDomain.

You still need to remove the Deny, assembly level requests or the caspol command from your application and make one of the existing classes derive from MarshalByRefObject. Then you call the SandboxActivator.GetPartialTrustInstance which will return a new instance of your type in a partial-trust AppDomain. Calling a method on this new instance will be executed in a partial-trust sandbox.

Example:

//This will create a sandboxed AppDomain with the Execution permission set

MainClass m = SandboxActivator.GetPartialTrustInstance<MainClass>();

m.DoPartialTrustCall();

These are the public APIs available through the SandboxActivator interface:

public static AppDomain CreateSandboxedDomain(PermissionSet ps, params Assembly[] fullTrustList)

public static T GetPartialTrustInstance<T>(PermissionSet ps, params Assembly[] fullTrustList) where T : MarshalByRefObject

public static T GetPartialTrustInstance<T>(params Assembly[] fullTrustList) where T : MarshalByRefObject

4. The last method to make your application work is not typically recommended by us. We have a way to set your process to run in legacy mode that is according to the pre-v4.0 security model. We prefer users of.Net Framework to fully migrate to the new security model; we introduced this only for exception situations. This legacy mode will be available only in v4 and in future versions would be removed. We also believe the new security model does a better job at setting up a sandbox for your application to run.

So there, you have been warned to not use this method. To follow this migration path, you keep the code as it is, and add a config file for your executable that will look like this:

<NetFx40_LegacySecurityPolicy enabled="true" />

</runtime>

</configuration>

Cristian Eigel,

SDET, CLR

In-Process Side by Side Part 2 - Common in-proc SxS scenarios

Abhishek_Mondal — Sun, 07 Jun 2009 20:36:00 GMT

Update: The information in this blog post applies to pre beta-1 behavior and is not applicable to beta 2 and RTM. We will have more posts and documentation on this subject as we get closer to RTM.

This section covers the most common in-process side by side scenarios. The scenarios cover which CLR is used to run the specific code, given a particular machine configuration.

Native Application calling Managed COM Components

Machine Configuration: Both CLR 2.0 (.NET v2.0, v3.5 or v3.5) and CLR 4 installed

In this scenario, all Legacy Managed COM Components will execute against CLR 2.0 and any call to the Global Hosting APIs, in the whole process (in the application or any assembly activated through COM or legacy API), will return CLR 2.0 results. Also, any activated Managed COM Components compiled against CLR 4 will execute against CLR 4, in-process side by side with others, compiled against CLR 2.0 (executed against CLR 2.0).

Native Application calling the legacy Global Hosting APIs

Machine Configuration: Both CLR 2.0 (.NET v2.0, v3.5 or v3.5) and CLR 4 installed

In this scenario, any call to these functions, in the whole process (in the application or any assembly activated through COM or legacy API), will return CLR 2.0 results.

Managed Application compiled against v4 calling Managed COM components

Machine Configuration: Both CLR 2.0 (.NET v2.0, v3.5 or v3.5) and CLR 4 installed

In this scenario, a managed application compiled against CLR 4 calls a managed COM component compiled against CLR v1.x, CLR v2.0 or CLR 4. In this situation, any call to the Global Hosting APIs, in the whole process (in the application or any assembly activated through COM or legacy API), will return CLR 4 results. Also, any activated legacy Managed COM Components will execute against CLR 4.

Managed Application compiled against v4 using P/Invokes to call the legacy Global Hosting APIs

Machine Configuration: Both CLR 2.0 (.NET v2.0, v3 or v3.5) and CLR 4 installed

In this scenario, a managed application compiled against v4 uses P/Invoke to call the existing legacy Global Hosting APIs. In this situation, any call to these functions, in the whole process (in the application or any assembly activated through COM or legacy API), will return CLR 4 results.

Managed Application compiled against v2 calling Managed COM components compiled against CLR 4

Machine Configuration: Both CLR 2.0 (.NET v2.0, v3.0 or v3.5) and CLR 4 installed

In this scenario, a managed application compiled against CLR 2.0 calls a managed COM component compiled against CLR 4. In this scenario, any activated Managed COM Components will execute against CLR 4 in-process side by side with CLR 2.0.

What are the most common non-default scenarios?

This section illustrates some common scenarios and the corresponding configuration file entries in order to get the right results.

Native Application rolling forward all its Managed COM Components to CLR 4

Machine Configuration: CLR 4 installed (other CLRs can be installed also, but in this case, this is irrelevant)

In this scenario, a native application needs to activate all its managed COM components (independently of the CLR version they were compiled against) using CLR 4. In this situation, an APP.CONFIG file needs to be included in the same directory of the native application (EXE) with the following entry:

</process>

</startup>

This configuration will force all Managed COM Components to execute against CLR 4.

Native Application using legacy Global Hosting APIs to return CLR 4 results

Machine Configuration: .NET 4 installed (other CLRs can be installed also, but in this case, this is irrelevant)

In this scenario, a native application uses the existing Global Hosting APIs to return CLR 4 results. To accomplish that, an APP.CONFIG file needs to be included in the same directory of the native application with the following entry:

</process>

</startup>

This configuration will force all Managed COM Components to execute against CLR 4. It's important to note that some Global Hosting APIs may not be affected by the APP.CONFIG file above.

Managed Application compiled against v2 rolling forward to execute against CLR 4

Machine Configuration: .NET 4 installed (other CLRs can be installed also, but in this case, this is irrelevant)

In this scenario, a managed application compiled against v2.0 needs to execute against CLR 4. In this situation, an APP.CONFIG file needs to be included in the same directory of the managed application (EXE) with the following entry.

</startup>

Where XXXXX is the CLR build number. In this situation, any Managed COM Component activated by this application will also roll forward to CLR 4. Also, any Global Hosting API call in the whole process (in the application or in any of the Managed COM components) will return CLR 4 results (for example, CorBindToRuntime(pwszVersion=NULL) or GetCORSystemDirectory).

Managed Application compiled against v4 turning on In-process Side By Side

Machine Configuration: Both CLR 2.0 (.NET v2.0, v3.0 or v3.5) and CLR 4 installed

In this scenario, a managed application compiled against CLR 4 needs to activate Managed COM Components compiled against CLR 4 side by side with Managed COM Components compiled against CLR 2.0 (or previous version). In this situation, an APP.CONFIG file needs to be included in the same directory of the managed application (EXE) with the following entry.

</process>

</startup>

Managed Application compiled against v2 running against v4 and turning on In-process Side By Side

Machine Configuration: Both CLR 2.0 (.NET v2.0, v3.5 or v3.5) and CLR 4 installed

In this scenario, a managed application compiled against CLR 2.0 needs to run against CLR 4 and also needs to activate Managed COM Components compiled against CLR 4 side by side with Managed COM Components compiled against CLR 2.0 (or previous version). In this situation, an APP.CONFIG file needs to be included in the same directory of the managed application (EXE) with the following entry.

</process>

</startup>

Luiz Fernando Santos,

PM, CLR

AlwaysOn support in SqlClient	New 4.0.2 feature
SQL Server Express Local Database Runtime support in SqlClient	New 4.0.2 feature
StateMachine (including designer support)	Existing 4.0.1 feature
SqlWorkflowInstanceStore on SQL Azure	Existing 4.0.1 feature
Compensation for custom hosts	Existing 4.0.1 feature