BCL Team Blog

Code Contracts at PDC 2009

BCLTeam — Tue, 17 Nov 2009 19:22:00 GMT

If you’re at PDC this year and interested in Code Contracts, go watch Mike Barnett and Nikolai Tillmann talk about Code Contracts and Pex (another MSR tool). Their session is called Code Contracts and Pex: Power Charge Your Assertions and Unit Tests. It's Tuesday at 4:30 in 408A. They’ll give you details on some of the latest developments in the Code Contract tools.

There will also be a table in the exhibition area where Manuel Fahndrich will have a demo and be able to answer questions.

What's New in the BCL in .NET 4 Beta 2 [Justin Van Patten]

BCLTeam — Wed, 21 Oct 2009 22:58:00 GMT

Visual Studio 2010 and .NET Framework 4 Beta 2 are now available to download. .NET 4 Beta 2 contains several new BCL features and enhancements in addition to what was included in .NET 4 Beta 1. Many of these improvements were added in large part due to specific feedback and suggestions reported by customers through Microsoft Connect.

Summary of New BCL Functionality in .NET 4 Beta 2

Complex Number
Location
IObservable<T>
Stream.CopyTo
Guid.TryParse, Version.TryParse, and Enum.TryParse<T>
Enum.HasFlag
String.Concat and String.Join overloads that take IEnumerable<T>
String.IsNullOrWhiteSpace
Environment.SpecialFolder additions
Environment.Is64BitProcess and Environment.Is64BitOperatingSystem
Path.Combine params support
TimeSpan Globalized Formatting and Parsing
Stopwatch.Restart
StringBuilder.Clear
IntPtr and UIntPtr Addition and Subtraction operators
ServiceInstaller.DelayedAutoStart
ObservableCollection<T> moved to System.dll

Complex Number

System.Numerics.Complex represents a complex number comprised of a real number and imaginary number. Complex supports both rectangular coordinates (real and imaginary) and polar coordinates (magnitude and phase measured in radians) and supports arithmetic and trigonometric operations. Complex numbers are used in a number of areas including high-performance computing, graphing and charting, electrical engineering (e.g. Fourier transforms), and other numerical applications.

Location

System.Device.Location (which can be found in System.Device.dll) enables .NET applications running on Windows 7 to determine the current geo-location (e.g. latitude/longitude) of the device. Windows 7 supports a number of different location sensors, including GPS devices and WWAN radios, and automatically handles transitioning between different sensors — if multiple sensors are available — to provide the most accurate data for the current situation. .NET applications will be able to use this new API to access the following data (if available): latitude, longitude, altitude, horizontal and vertical accuracy, course, speed, and civic address (i.e. country/region, state/province, city, postal code, street, building, floor level). Note that we are planning to make some changes to the shape of the location API between Beta 2 and RC, so please keep this in mind as you evaluate and use the Beta 2 location API in apps. A separate blog post will provide more details on the planned changes for RC.

IObservable<T>

System.IObservable<T> and System.IObserver<T> provide a general mechanism for push-based notifications and can be used to represent push-based, or observable, collections. Anyone familiar with design patterns will recognize these interfaces as representations of the well-known observer pattern. IObservable<T> opens up the doors to some exciting new paradigms for event-based and asynchronous programming on .NET. A great example of this is the Reactive Framework (RX), which is a library of extension methods (not included as part of .NET 4) that implement the LINQ Standard Query Operators and other useful stream transformation functions for IObservable<T>. F# first-class events also support IObservable<T> and F# includes a library of Observable operators similar to those available in RX. Imagine being able to do LINQ-style queries against events; this is just one of the kinds of things that IObservable<T> enables. To help grasp the power of IObservable<T>, I encourage you to watch the Channel 9 videos of Erik Meijer introducing RX and Wes Dyer and Kim Hamilton discussing IObservable<T> in the BCL.

You may be wondering if IObservable<T> is meant to replace events in .NET. The answer is largely “no,” but you certainly could use it in place of events if you wanted to. Events are a well-known mechanism for notification in .NET and are already familiar to .NET developers. As such, they will continue to be the primary mechanism for notification that we recommend for .NET APIs. The great thing about IObservable<T> is that it is fairly straightforward for callers to convert any existing event into an IObservable<T>. RX provides built-in methods that do this conversion and F# has support for this built-in to the language. Stay tuned to this blog for more news about RX in the future.

Stream.CopyTo

When working with Streams, how many times have you had to write the following boiler-plate code to read from one stream and write the contents to another stream?

Stream source = ...;
Stream destination = ...;
byte[] buffer = new byte[4096];
int read;
while ((read = source.Read(buffer, 0, buffer.Length)) != 0) {
    destination.Write(buffer, 0, read);
}

In .NET 4, you no longer have to write this code. We’ve added a new CopyTo method to Stream that allows you to simply write:

Stream source = ...;
Stream destination = ...;
source.CopyTo(destination);

Guid.TryParse, Version.TryParse, and Enum.TryParse<T>

We’ve added TryParse to System.Guid, System.Version, and System.Enum. Enum.TryParse is generic, a nice improvement over the existing non-generic Parse method, which leads to much cleaner code.

On previous versions of .NET, to parse an enum value, you’d write something like this:

try {
    string value = Console.ReadLine();
    FileOptions fo = (FileOptions)Enum.Parse(typeof(FileOptions), value);
    // Success
}
catch {
    // Error
}

On .NET 4, you can use the new generic TryParse method:

string value = Console.ReadLine();
FileOptions fo;
if (Enum.TryParse(value, out fo)) {
    // Success
}

Enum.HasFlag

We’ve added a new convenience method to System.Enum that makes it super easy to determine if a flag is set on a particular Flags enum, without having to remember how to use the bitwise operators. This also allows for more readable code.

In VB:

Dim cm As ConsoleModifiers = ConsoleModifiers.Alt Or ConsoleModifiers.Shift
Dim hasAlt1 As Boolean = (cm And ConsoleModifiers.Alt) = ConsoleModifiers.Alt ' using bitwise operators
Dim hasAlt2 As Boolean = cm.HasFlag(ConsoleModifiers.Alt) ' using HasFlag

In C#:

ConsoleModifiers cm = ConsoleModifiers.Alt | ConsoleModifiers.Shift;
bool hasAlt1 = (cm & ConsoleModifiers.Alt) == ConsoleModifiers.Alt; // using bitwise operators
bool hasAlt2 = cm.HasFlag(ConsoleModifiers.Alt); // using HasFlag

String.Concat and String.Join overloads that take IEnumerable<T>

We’ve added new overloads of String.Concat and String.Join that take IEnumerable<T>, adding to the existing overloads that take arrays. This means you can pass any collection that implements IEnumerable<T> to these APIs without having to first convert the collection into an array. We’ve also added “params” support to the existing array-based String.Join overload, which allows you to write more succinct code. In C#:

String.Join(", ", new string[] { "A", "B", "C", "D" }); // you used to have to write this
String.Join(", ", "A", "B", "C", "D"); // now you can write this

String.IsNullOrWhiteSpace

This new convenience method is similar to the existing IsNullOrEmpty method, but in addition to checking if the string is null or empty, it also checks to see if the string only consists of white-space characters, as defined by Char.IsWhiteSpace. It works great for Code Contract preconditions.

Environment.SpecialFolder additions

We’ve added a number of new values to the Environment.SpecialFolder enum:

AdminTools
CDBurning
CommonAdminTools
CommonDesktopDirectory
CommonDocuments
CommonMusic
CommonOemLinks
CommonPictures
CommonProgramFilesX86
CommonPrograms
CommonStartMenu
CommonStartup
CommonTemplates
CommonVideos
Fonts
LocalizedResources
MyVideos
NetworkShortcuts
PrinterShortcuts
ProgramFilesX86
Resources
SystemX86
Templates
UserProfile
Windows

We’ve also added a new overload to Environment.GetFolderPath that takes a new SpecialFolderOption enum, allowing you to specify some additional options, such as “Create” (to create the directory if it doesn’t exist) and “DoNotVerify” (to just return the path without checking if it exists, which can help reduce lag time if the directory is on a network share).

Environment.Is64BitProcess and Environment.Is64BitOperatingSystem

These new convenience APIs make it easy to determine the bitness of the current process and operating system. Environment.Is64BitProcess is equivalent to calling IntPtr.Size == 8. Environment.Is64BitOperatingSystem will tell you if the underlying OS is 64-bit. Note that it is possible for Environment.Is64BitProcess to return false and Environment.Is64BitOperatingSystem to return true if the process is a 32-bit process running on a 64-bit operating system under WOW64.

Path.Combine params support

Have you ever had to nest calls to Path.Combine like this:

Path.Combine("dir1", Path.Combine("dir2", Path.Combine("dir3", "dir4")));

With .NET 4, you can simply write:

Path.Combine("dir1", "dir2", "dir3", "dir4");

TimeSpan Globalized Formatting and Parsing

In previous versions of .NET, TimeSpan only supported a single culture-agnostic format for formatting and parsing. This meant if you called ToString on a French machine the output would not match culture settings, making the application appear not to be globalized. This is often noticeable in apps that display results from SQL databases that use the frequently used Time column type, because SQL’s Time maps to TimeSpan in .NET. For example:

Thread.CurrentThread.CurrentCulture = new CultureInfo("fr-FR");
Console.WriteLine(new TimeSpan(0, 1, 1, 1, 1));
Console.WriteLine(12.34);

Output:

01:01:01.0010000
12,34

The output from TimeSpan.ToString is a culture-agnostic format so it uses a '.' as the decimal separator, whereas Double uses ',' based on the current culture.

In .NET 4, System.TimeSpan now supports formatting and parsing based on culture settings via new overloads of ToString, Parse, and TryParse, and new ParseExact and TryParseExact methods. The default ToString overload still behaves the same for backwards compatibility, outputting a constant, culture-agnostic format, but the new overloads allow you to specify additional culture-sensitive formats. You can pass the “g” format (general format) to the new ToString overload to specify a culture-sensitive format:

Thread.CurrentThread.CurrentCulture = new CultureInfo("fr-FR");
Console.WriteLine(new TimeSpan(0, 1, 1, 1, 1).ToString("g"));
Console.WriteLine(12.34);

Output:

01:01:01,0010000
12,34

With this new format, the output from TimeSpan is in line with the user’s culture settings.

Stopwatch.Restart

This is a new convenience method that is equivalent to calling Stopwatch.Reset followed by Stopwatch.Start.

StringBuilder.Clear

In previous versions of .NET, you could clear StringBuilder by setting the Length property to 0. This isn’t very discoverable, so we’ve added StringBuilder.Clear as a more discoverable equivalent.

IntPtr and UIntPtr Addition and Subtraction operators

Adding an offset to an IntPtr or UIntPtr can be error prone and often introduces subtle bugs on 64-bit machines if not done correctly. .NET 4 includes addition and subtraction operators on these types, along with Add and Subtract methods, to help prevent these kinds of errors.

ServiceInstaller.DelayedAutoStart

.NET-based services that start automatically can now set a new DelayedAutoStart property to true (recommended), which will delay auto-starting the service until after other auto-start services are started plus a short delay to improve system boot performance when running on Vista or greater.

ObservableCollection<T> moved to System.dll

We’ve type-forward System.Collections.ObjectModel.ObservableCollection<T>, System.Collections.ObjectModel.ReadOnlyObservableCollection<T>, and System.Collections.Specialized.INotifyCollectionChanged from WindowsBase.dll (a WPF assembly) to System.dll. This allows you to use these collections without taking a dependency on a WPF assembly and enables a cleaner separation between model and view.

We hope you like what we’ve added to the BCL in .NET 4 Beta 2. Download Visual Studio 2010 and .NET 4 Beta 2 today. Be sure to let us know what you think and report any bugs you run into.

Update: Added links to the MSDN documentation for each feature throughout. We'd appreciate any feedback on the docs as well. Also updated ServiceInstaller with the correct name (fixed from ServiceProcessInstaller).

Code Contracts CLR Inside Out Article [Melitta Andersen]

BCLTeam — Fri, 21 Aug 2009 22:26:00 GMT

In the August 2009 issue of MSDN Magazine, we have a CLR Inside Out article on the BCL’s experience with adding code contracts to the BCL. It includes both an overview of the feature, as well as a few best practices we came up with as we went along. For regular readers of this blog who have read the previous posts on code contracts and followed all of the links, this article probably won’t provide much new information with the overview, but some of the tips will likely be useful if you plan on using code contracts.

As always, comments and questions are welcome here or on the Code Contracts forum.

Free LiveMeeting with the CLR Team on CLR 4 [Justin Van Patten]

BCLTeam — Tue, 14 Jul 2009 21:55:00 GMT

Join the CLR team for a free interactive LiveMeeting session / conference call on What's New in CLR 4 on Friday, July 17, 2009. We'll specifically be covering Garbage Collection, NGEN, and Performance.

Here are the details:

Date: Friday, July 17
Time: 10:00AM-11:00AM PST

LiveMeeting:
https://www.livemeeting.com/cc/0000000379_103/join?id=76P755&role=attend&pw=p%2C%2Fzk2K%7CM

Conf call:
Toll Free: 866-500-6738
Toll: 203-480-8000
Participant code: #198585

Building Tuple [Matt Ellis]

BCLTeam — Wed, 08 Jul 2009 04:47:00 GMT

For readers who are interested in the work that goes into designing a feature, I wrote an article for MSDN Magazine that appears in this month’s issue. Check out CLR Inside Out: Building Tuple which introduces the new Tuple type as well as discusses the design work we did behind it.

I’d love to hear feedback on what you think about the article and if you’d like to see more behind the scenes design articles in the future. I’d also love to answer any questions about the design or why we made the decisions we did.

Also, there is one change we are thinking about making between Beta 1 and Beta 2 around tuple. In Beta 1 we have a factory method, Tuple.Create, which builds tuples and has some nice type inference properties. In Beta 1 the overload of this method which takes eight arguments requires that the last element is a tuple and builds an extended tuple. For example:

Tuple.Create(1, 2, 3, 4, 5, 6, 7, Tuple.Create(8));

Will build an eight element tuple that looks like [1, 2, 3, 4, 5, 6, 7, 8].

Tuple.Create(1, 2, 3, 4, 5, 6, 7, Tuple.Create(8, 9));

Will build a nine element tuple that looks like [1, 2, 3, 4, 5, 6, 7, 8, 9]

For Beta 2 we hope to change this so that eight argument version of Tuple.Create always builds an eight element tuple. In this case:

Tuple.Create(1, 2, 3, 4, 5, 6, 7, 8);

Will build an eight element tuple that looks like [1, 2, 3, 4, 5, 6, 7, 8]

Tuple.Create(1, 2, 3, 4, 5, 6, 7, Tuple.Create(8));

Will build an eight element tuple that looks like [1, 2, 3, 4, 5, 6, 7, [8]].

Tuple.Create(1, 2, 3, 4, 5, 6, 7, Tuple.Create(8, 9));

Will build an element tuple that looks like [1, 2, 3, 4, 5, 6, 7, [8, 9]]

If you want to build tuples with more than eight elements and your language doesn’t have special tuple syntax, you’ll have to use the Tuple constructors directly. If we see lots of people doing this we’ll add more overloads to Tuple.Create.

Thanks for reading; I hope everyone enjoys the article and look forward to your questions and comments. Cheers!

What's New in Globalization in .NET 4 Beta 1 [Melitta Andersen]

BCLTeam — Mon, 08 Jun 2009 18:00:00 GMT

In Justin’s recent post outlining What’s New in the BCL in .NET 4 Beta 1, there were only a few lines about the globalization updates. That’s because Mohamed Elgazzar was working on a more extensive document which provides details about the extent of the changes and additions to that space. I’m happy to say that the document is now live. Please check it out here: Microsoft .NET Framework 4 (Beta 1): What is New in Globalization.

What's New in the BCL in .NET 4 Beta 1 [Justin Van Patten]

BCLTeam — Sat, 23 May 2009 04:31:00 GMT

Visual Studio 2010 and .NET Framework 4 Beta 1 are available for download. .NET 4 Beta 1 contains new functionality and improvements throughout the framework. In this post I’ll discuss the changes specific to the BCL.

Many of the features listed below were previously announced when we released the .NET Framework 4 CTP back in November of last year. I’ve included these features again below for completeness. New functionality in Beta 1 that wasn’t present in the CTP is denoted as such.

Code Contracts
System.Diagnostics.Contracts provides a language-agnostic way to express coding assumptions in .NET programs. The contracts take the form of pre-conditions, post-conditions, and object invariants. Contracts act as checked documentation of your external and internal APIs. The contracts are used to improve testing via runtime checking, enable static contract verification, and could be used in documentation generation. We partnered with Microsoft Research to deliver this feature in .NET 4. More information and links to the tools are available on the Code Contracts DevLabs site.
(new in Beta 1) The latest release of the tools works with Visual Studio 2010 Beta 1.
Parallel Extensions
The Parallel Computing Platform team worked with us to add the Task Parallel Library (TPL), Coordination Data Structures, and Parallel LINQ (PLINQ) to the BCL in .NET 4. This includes an improved ThreadPool scheduling algorithm for tasks, Task and Task<T>, the static Parallel class, concurrent collections in System.Collections.Concurrent, and other coordination data structures such as Lazy<T>, CountdownEvent, Barrier, SemaphoreSlim, SpinLock, SpinWait, and ManualResetEventSlim. More information is available over at the PFX team’s blog.
BigInteger
System.Numerics.BigInteger is an arbitrary-precision integer data type. BigInteger supports all the standard integer operations, including bit manipulation. It can be used from any .NET language, and some of the new .NET languages—such as F# and IronPython—have support built-in to the language. We partnered with the Microsoft Solver Foundation team to deliver this in .NET 4.
Variance annotations (new in Beta 1)
C# and VB now support safe co- and contra-variance for generic interface and delegate types. Co-variance means that a generic of a type, e.g. an IEnumerable<String>, can be treated as a generic of any supertype, e.g. an IEnumerable<Object>. Contra-variance means that a generic of a type, e.g. an Action<Object>, can be treated as a generic of a subtype, e.g. an Action<String>. In C#, co-variance is annotated with the “out” keyword and contra-variance is annotated with the “in” keyword. VB has similar “In” and “Out” keywords. We’ve annotated several interfaces and delegates in the BCL for variance, including: IEnumerable<T>, IEnumerator<T>, IComparer<T>, IEqualityComparer<T>, IComparable<T>, the Action and Func delegates, Predicate<T>, Comparison<T>, EventHandler<TEventArgs>, and Converter<TInput, TOutput>.
Tuples (new in Beta 1)
We’ve added common Tuple types in the BCL to facilitate language interoperability and to reduce duplication in the framework. A tuple is a simple generic data structure that holds an ordered set of items of heterogeneous types. Tuples are supported natively in languages such as F# and IronPython, but are also easy to use from any .NET language such as C# and VB.
SortedSet<T> (new in Beta 1)
We’ve added a SortedSet<T> collection along with a new ISet<T> interface. SortedSet<T> uses a self-balancing tree which maintains data in sorted order for performance guarantees with insertion, deletion, and searches. Both the new SortedSet<T> and the existing HashSet<T> implement ISet<T>.
File System Enumeration Improvements (new in Beta 1)
System.IO.Directory and System.IO.DirectoryInfo have new APIs for enumerating the file system that return IEnumerable<T>s instead of arrays. These new APIs are more efficient than the array-based APIs because they do not need to allocate a (potentially large) array and you can access the first results immediately instead of waiting for the entire array to be created and returned. We’ve also added new convenience APIs for efficiently reading, writing, and appending lines from/to a text file using IEnumerable<String>. These new APIs are useful in LINQ scenarios where you may want to quickly and efficiently query the contents of a text file and write out the results to a log file without allocating any arrays.
MemoryMappedFiles
System.IO.MemoryMappedFiles exposes the memory mapping functionality provided by Windows as first-class managed APIs. Memory mapped files can be used to efficiently edit very large files and can also be used to create shared memory for inter-process communication. Along with this feature, we’re introducing System.IO.UnmanagedMemoryAccessor, a new class that enables random access to unmanaged memory similar to how UnmanagedMemoryStream enables sequential access to such memory.
Registry Improvements (new in Beta 1)
We’ve added support to Microsoft.Win32.RegistryKey for accessing alternate registry views (32bit and 64bit views) on 64bit Windows. We’ve also added the ability to create volatile keys—keys that are stored in memory and are not persisted when the hive unloads (which typically happens on reboot). In addition, we’ve added new extensibility APIs that take and return a SafeRegistryHandle to make it easier to interoperate with Win32 registry APIs that do not have managed wrappers.
Globalization data updated to Unicode 5.1 (new in Beta 1)
Culture data in .NET 4 has been updated, including updating the character information to Unicode 5.1. We have also added support for new cultures, including new neutral cultures.
ResourceManager Improvements
The ResourceManager in System.Resources has been improved to respect the user’s preferred UI languages when looking for localized resources, instead of only using the CurrentUICulture’s parent chain. This means if the user has specified in Windows that she prefers French and Spanish, the ResourceManager will look for French and Spanish resources before falling back to the neutral resources. This change is present in Silverlight 2 as well as .NET 4.
Compression Improvements
The compression algorithms in System.IO.Compression have been improved in .NET 4. DeflateStream and GZipStream no longer inflate already compressed data. This means that in many cases you’ll see better compression ratios when using these streams on .NET 4. We’ve also removed the 4 GB size limit, so you can now compress streams over 4 GB in length.

You can learn more about what’s new in the BCL in .NET Framework 4 on Channel9: Inside .NET 4: Meet the BCL Team.

We hope you enjoy the new functionality and improvements in Beta 1. Be sure to let us know if you run into any issues or if you have any further suggestions. And look out for even more improvements and refinements in the next preview release of .NET Framework 4.

Preview of Code Contract Tools Now Available [Melitta Andersen]

BCLTeam — Tue, 24 Feb 2009 02:00:00 GMT

In my Introduction to Code Contracts post, I mentioned that the tools to enable runtime checking and static analysis were not included in the .NET Framework 4.0 CTP.

Today we are releasing a preview version of the Code Contracts tools on the DevLabs site. DevLabs is a site for getting preview releases of innovative new projects for developers out to the community. You can read more about it on Soma’s introductory blog post. The specific Code Contracts page on DevLabs can be found here.

The tools include a binary rewriter to enable runtime checking, a static checker, a new project pane for VS projects, and an updated copy of the Code Contract library. For more details on what these are, you can check out the Code Contracts page, which includes an overview, a video on how to get started, an FAQ, and other useful information. You can also check out the documentation (PDF), or get an overview in Soma’s recent post on Code Contracts.

This current release is targeted at Visual Studio 2008 Team System and can be used for commercial development. The academic release that can be used for non-commercial use on any Visual Studio 2008 edition except Express can still be found on the Contracts page on the Microsoft Research site.

This new release also comes with more ways to provide feedback and get answers. In my last post, I mentioned the Code Contracts thread on the Pex forum. Now Code Contracts has its own forum. So please go try out these new tools and tell us what you think.

In-depth: How .NET managed services interact with the ServiceControlManager (SCM) [Kim Hamilton]

BCLTeam — Fri, 20 Feb 2009 01:44:00 GMT

.NET managed services are based on NT services, and both need to know how to interact with the ServiceControlManager (SCM) to remain responsive. For managed services, many of the complexities of interacting with the SCM are handled for you by System.ServiceProcess.ServiceBase; however, there are some you still need to be aware of.

I’ll summarize the key interactions between the SCM and any service (managed or native). I’ll describe how these apply to managed services — which interactions you need to perform and which are handled for you. Lastly, I’ll walk through a representative SCM command to show the full sequence.

SCM Interaction 1

General Guidance

When SCM sends a request, acknowledge it by updating the service state (e.g. pending) and spawn a new thread to handle the request so that the SCM can communicate with other services

Background

The SCM only communicates with one service at a time. This guidance allows the SCM to be unblocked so it can communicate with other services. So the suggested behavior for services is: set the status to pending and kick off a new thread to do any necessary work (thereby unblocking the SCM). The SCM can then move on to interacting with other services, while the worker thread updates the SCM with progress and completion status.

Impact for managed services

This is only partially relevant for managed services because ServiceBase handles much of this for you. You only need to worry about this if you expect your service will require more time than the “unresponsive” limit, or to spawn a thread that will last the entire lifetime of the service. Here are the details:

You don’t need to start a new thread just to unblock the SCM. ServiceBase calls your OnStart/OnStop (we’ll refer to those as OnX) on a worker thread, so when you’re OnStart/OnStop is called, you’re not blocking the SCM
You don’t set the service state. The ServiceBase class sets the service state to X_PENDING before calling your OnX/OnX. Also, when your OnX method returns, ServiceBase performs the final status update, e.g. to STATUS_STOPPED. (In fact, there’s not even a managed API for setting these states.)
While your OnX thread isn’t blocking the SCM, you do need ensure it doesn’t exceed its time limit. You can help ensure you have enough time by calling RequestAdditionalTime, discussed in SCM Interaction 2.
You do need to spawn a new thread in OnStart for any processing that should happen during the entire lifetime of the service.

SCM Interaction 2

General Guidance

After the SCM issues a command, the SCM gives each service a default amount of time to complete its work. If the service requires more time it must tell the manager that it needs more time, otherwise the manager will assume it’s hung.

Background

As mentioned in the previous interaction, it’s important that a service doesn’t block the SCM, so that the SCM can respond to other services. However, the SCM also expects status updates from the service, to ensure the service is responsive. For example, after issuing a start command, the SCM expects the service to update its state to started within 30 seconds. After issuing a stop command, the SCM expects the service to update its state to stopped within 20 seconds.

Impact for managed services

This is where your managed service needs to pay attention to avoid the SCM flagging your service as unresponsive.

You don’t have to explicitly update the state; ServiceBase does this for you when your OnStart or OnStop method completes
You do have to call RequestAdditionalTime if you expect the OnX method to exceed the timeout.

SCM Interaction 3

General Guidance

To give SCM impression of progress, update the dwCurrentState or dwCheckPoint

Background

These state and checkpoints are intermediate “in progress” types of notification to let the SCM know your service isn’t hanging.

Impact for managed services

With managed services, these details are abstracted away. All you have to do is call RequestAdditionalTime() and those details are handled for you behind the scenes. ServiceBase will handle the state and checkpoint details in its message to the SCM.

Walkthrough: how ServiceBase handles Stop command

To demonstrate the above, let’s walk through how ServiceBase responds to a STOP command from the SCM. Note that Start is similar.

ServiceBase receives STOP command:

ServiceBase queues up an async call, to call your handler and update status
In the async call, ServiceBase does the following:
- Sets status to STATE_STOP_PENDING
- Calls your OnStop (note this is already in a separate thread)
- When your OnStop returns, sets state to STATE_STOPPED

What should your OnStop look like?

If you need more time, just call RequestAdditionalTime. We’ll send this value to the SCM and update the checkpoint for you
If you don’t need more time, you don’t have to do any additional steps. Your method returns and we update the state for you.

If your OnStop fails to call RequestAdditionalTime or blocks for longer than 20 (default; see below) seconds, SCM marks service as unresponsive.

To prevent a service from stopping shutdown, the SCM will only wait up to a limit for your service to stop itself. The default for this limit is 20 seconds (this value is in the registry key WaitToKillServiceTimeout() in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control)

Additional Reading

The following MSDN article describes writing responsive native services.

http://msdn.microsoft.com/en-us/magazine/cc164252.aspx

Working with the ResourceManager [Kim Hamilton]

BCLTeam — Tue, 17 Feb 2009 00:28:00 GMT

The essentials of resource fallback and how to debug failures

Resource loading failures can be tricky to debug. However, once you have a basic overview of how the ResourceManager works, as well as a few debugging tools, you’ll be able to easily get to the bottom of most problems.

This post will cover the key elements of resource lookup and debugging with a simple code sample, addressing the following:

Basics of resource generation and file structure
How to construct a ResourceManager to find your resources
How resource fallback works
How to debug resource failures

Code Sample

The code sample is posted in the BCL code gallery downloads as ResourceManagerSample.zip. The zip file contains:

.restext files in different languages: these are text files containing key, value pairs from which we’ll generate .resources files. Note that the extension restext isn’t important, it can be anything you want.
FallbackTest.cs: sample code that looks up resources in different cultures and prints the results
make.bat: builds main exe and satellite assemblies

Instructions

Extract the zip file and run make.bat, which does the following:
1. Converts the .restext files to .resources files (the binary format understood by the ResourceManager) using resgen.
2. Compiles the main exe and embeds the invariant resources, or the resources that are always expected to exist if other cultures are missing.
3. Generates satellite assemblies for the other cultures, placing them in culture-specific subfolders

After Step 1, the directory structure looks like the following:

FallbackTest.exe                 (main executable, contains FallbackStrings.resources)
de\FallbackTest.resources.dll    (de satellite assembly, contains FallbackStrings.de.resources)
fr\FallbackTest.resources.dll    (fr satellite assembly, contains FallbackStrings.fr.resources)
fr-CA\FallbackTest.resources.dll (fr-CA satellite assembly, contains FallbackStrings.fr-CA.resources)

Run FallbackTest.exe to see the results, discussed below

Before diving into the results, let’s look at the source of FallbackTest.cs to see what it’s doing.

Creating a ResourceManager

The ResourceManager can perform two types of resource lookup, depending on how you construct it.

File-based: resource files are directly on disk in the binary .resources files. Create this type of ResourceManager via ResourceManager.CreateFileBasedResourceManager.
Assembly-based: resources are embedded in an assembly or a satellite assembly. Create this type of ResourceManager with the standard constructors.

Assembly-based is by far the most commonly used; a satellite assembly has the same advantages as a regular assembly such as versioning and the ability to sign. That’s why we’re focusing on assembly-based lookup in this example.

The code sample creates a ResourceManager as follows:

ResourceManager rm = new ResourceManager("FallbackStrings",
                            Assembly.GetExecutingAssembly());

The first argument is baseName — the name of the resources files we’re searching without the culture and “.resources” extension. In step 1a above, resgen created files named, for example, FallbackStrings.fr.resources and FallbackStrings.fr-CA.resources. So baseName in this example would be FallbackStrings.

The second argument is the main assembly for the resources. Since we’re calling from that assembly, we can just say Assembly.GetExecutingAssembly(). Note that this is the common case.

Resource Fallback Model

Let’s move to the resource lookups and sample code output. First, notice these lines:

CultureInfo ci = new CultureInfo(culture);
Thread.CurrentThread.CurrentUICulture = ci;

This is done because the ResourceManager uses the current thread’s CurrentUICulture to search for resources by default. ResourceManager.GetString also has overloads that accept a CultureInfo. If provided, that culture will be searched instead.

After setting the UI Culture, fallback test searches for a couple of localized strings and prints them to the screen.

Current UI culture: fr-CA
Greeting: bonjour
End of week: la fin de semaine
------------------------------
Current UI culture: fr-FR
Greeting: bonjour
End of week: le week-end
------------------------------
Current UI culture: de-DE
Greeting: Guten Tag
End of week: weekend
------------------------------
Current UI culture: ja-JP
Greeting: hello
End of week: weekend
------------------------------

Let’s focus on fr-CA and fr-FR. fr-CA is the French Canadian locale, which has some different words than fr-FR, the French spoken in France.

In this code sample, we’ve included resources for fr-CA, but none for fr-FR. Instead, we’ve provided resources for fr, which is the neutral French culture. A neutral culture is not region-specific and is the parent of its corresponding region-specific cultures. For example, fr is the parent culture of both fr-CA and fr-FR.

A quick aside on this decision:

Resource divisions like this are fairly common in practice; the most common words for a language (across regions) are placed in the neutral culture’s resource file, and any deviations from that are placed in the specific culture’s resource file. This minimizes duplication in resource files. So in this example, the choice to put greeting in fr allowed us to share resources across French-speaking regions, but adding explicit resources for fr-CA allows French Canadian to differ from the most commonly-used French.

Now notice in the program output that fr-CA and fr-FR have the same greeting but different words for the end of the week. That’s because for greeting, fr-CA resources didn’t have a value, so it fell back to fr (same as fr-FR).

In general, a resource fallback chain looks like this.

Culture passed to GetString(, x) or ThreadUICulture (fr-CA) -> parent culture (fr) -> invariant culture (embedded in main assembly)

Compare the output for the other cultures above to see why the output makes sense.

The fallback chain can also vary as follows:

In .NET 4 (and Silverlight 2), on Windows Vista and above, we merge in user and system preferences according to the API GetThreadPreferredUILanguages (with flags MUI_MERGE_USER_FALLBACK | MUI_MERGE_SYSTEM_FALLBACK). These cultures are searched after the specified culture and its parents, but before the invariant culture
NeutralResourcesLanguageAttribute on the assembly: this allows performance improvements and allows ultimate fallback resources to be located in a satellite assembly (and in a different culture), instead of the main assembly. About the NeutralResourcesLanguageAttribute
Specify in the app’s config that only certain assembly locales should be probed. This also allows performance improvements. Note that this won’t add cultures to the fallback chain, it will only remove them. Performance tips for using resources

Debugging Resource-Related Issues

There are 3 common tools I use for debugging resource-related issues. These are:

Fusion: shows attempts to load satellite assemblies, which can help uncover assembly-loading problems
Reflector: view embedded .resources file
Resview: same as above but also has diagnostics to tell if there are problems

Let’s look at these in-depth by seeing what can go wrong. Comment out the call to GoodTest(), then uncomment the call to BadTest(), and rerun make.bat. This code has a bug, which is that the baseName is incorrect (“Fallback_Strings” instead of “FallbackStrings”). Running the code, we get the following exception:

System.Resources.MissingManifestResourceException: Could not find any resources appropriate for the specified culture or the neutral culture. Make sure "Fallback_Strings.resources" was correctly embedded or linked into assembly "FallbackTest" at compile time, or that all the satellite assemblies required are loadable and fully signed.

Let’s investigate this failure showing how the different tools are used:

Fusion logs

To enable fusion logs, launch the fusion log viewer, fuslogvw.exe, which is provided as a Framework sdk tool (you can find it under Framework\tools). Choose the option “Log all binds to disk”. I also select the “Enable custom log path” checkbox and specify a path such as C:\fuslog. (Since fusion logs are written as html files, I prefer to browse the files directly on disk than view the output in the fusion log viewer.)

Then run your program and look in C:\fuslog for the results.

In C:\fuslog\Default\FallbackTest.exe\, I see there were binding attempts for fr-CA and fr, because there are files named, e.g.:

“FallbackTest.resources, Version=0.0.0.0, Culture=fr-CA, PublicKeyToken=null.HTM”

Opening those files, you can see that the assembly did load correctly, so the problem isn’t a missing satellite assembly.

If the fusion log file did indicate a problem with the satellite assembly loading, then you know that either the satellite assembly is missing or has some other problem (version or signing problem), in which case you most likely have a build or install problem to correct.

Reflector/Resview

Both of these allow you to see the contents of a .resources file embedded in an assembly, but resview (part of the Framework sdk) has additional diagnostic tools. However, Reflector provides a nice interface for browsing the resource file, and I most often use that.

Open up the assembly of interest (either satellite or main) in Reflector. Expand the “Resources” section of the assembly to verify that the baseName we specified is actually present. As expected, we have a resource file named FallbackStrings.X.resources, but not Fallback_Strings.X.resources, and the solution is to remove the underscore from the baseName.

While this file name problem seems basic, it turns out to be a very common problem, so keep it in mind.

Another simple common mistake is if the key isn’t actually in the file. You can double-click on the .resources file in reflector to see the contents.

CLR Team Blog Now Live [Justin Van Patten]

BCLTeam — Wed, 07 Jan 2009 23:14:00 GMT

If you like this blog, you’ll want to check out and subscribe to the new CLR Team Blog that just launched last week. The new blog will serve as one consolidated site to find information on all aspects of the CLR, brought to you by CLR team members. It will aggregate posts from lots of existing CLR team member and feature team blogs. Since the BCL team is a part of the larger CLR team, when we post something on the BCL Team Blog we’ll also include a link to the post on the CLR Team Blog, or cross-post in some cases. Other CLR bloggers and features teams will do the same. The new blog will have its own original content too! Aarthi Ramamurthy’s introductory blog post describes in more detail what the new blog is all about. Check it out, subscribe, and stay tuned for upcoming posts!

Introduction to Code Contracts [Melitta Andersen]

BCLTeam — Wed, 12 Nov 2008 02:36:00 GMT

This blog post is to provide a bit more detail about the Code Contracts feature that was recently announced at the PDC and in Justin’s blog entry, and that can be found in the Visual Studio 2010 and .NET Framework 4.0 CTP. I’ll include some information on what can be found in the CTP, some of the history and design decisions we made while developing the feature, and some places to get more information. Both of the announcements mentioned above give details about why we added this feature, but I’ll summarize here. The main point of contracts is to reduce bugs by helping you not write them in the first place, or at least catch them sooner. Contracts let you express statements about the behavior of your code in a way that is accessible to tools for runtime and static analysis.

What’s not in the CTP

One thing you should note is that the tools to enable runtime checking and static analysis are not in the CTP. However, you can still write your contracts now and then use the tools when they become available. This works because the contracts are conditionally defined. They’ll only appear in your code when you have the CONTRACTS_FULL symbol defined. Until the tools are available and runtime checking is enabled, do not define this symbol. Full contract checking will not work without the tools. However, you also have the option of getting only preconditions with the CONTRACTS_PRECONDITIONS symbol. Preconditions happen to be in the correct place in the code, so your program will still run with this symbol defined. But the runtime checking tool provides many additional benefits, such as contract inheritance, so checking is still not recommended without the tools.

What is in the CTP

The CTP includes a CodeContracts class in the System.Diagnostics.Contracts namespace that allows you to write contracts in your code. All contracts are static methods that return void. They take a Boolean expression which encodes the condition that must be true. They also have an overload that takes a string parameter as a message for when the contract is false. Contracts are declarative and come at the beginning of your method. You can think of them as part of the signature. Here’s a rundown of what contracts are in the CTP:

Preconditions
Preconditions are statements about what must be true at method entry for successful execution of the method. It’s the responsibility of the caller to make sure these conditions are met. Often, preconditions are used for parameter validation. There are 3 possible ways to encode preconditions with code contracts:
- CodeContract.Requires
  This method simply states a precondition. Here’s an example:
  
  CodeContract.Requires(parameter >= 0);
- CodeContract.RequiresAlways
  This method is the one exception to the conditional compilation rule. It is always included, no matter which symbols are defined. Thus, you can use RequiresAlways for preconditions that you want in your released code. Here’s an example:
  
  CodeContract.RequiresAlways(parameter >= 0);
- CodeContract.EndContractBlock
  This method name might seem a bit odd to include in the precondition section, so let me explain. Much code already exists in the world that has some kind of parameter validation in the form of “if [condition] then throw [exception]”, e.g.
  
      if (parameter < 0)
          throw new ArgumentOutOfRangeException();
  
  We don’t expect everyone to go back and change their existing validation code. However, we wanted a way to let the tools recognize these legacy contracts. So we added the CodeContract.EndContractBlock method, which tells the tools that all if-checks of that form before the call can be considered preconditions.
  
      if (parameter < 0)
          throw new ArgumentOutOfRangeException();
      CodeContract.EndContractBlock();
  
  This method is needed only if you have no other contracts in your method, as all CodeContract preconditions and postconditions have this effect.
Postconditions
Post conditions are guarantees a method makes about what must be true at the conclusion of a method. It is the method’s responsibility to live up to those promises. These are declared at the beginning of a method, just like preconditions. The tools take care of checking them at the right times. There are two types of post conditions you can write with the CodeContract class.
- CodeContract.Ensures
  This method states conditions that must be true upon successful method exit. For example, this condition says that when the method exits the value will not be null.
  
  CodeContract.Ensures(SomeSharedState != null);

CodeContract.EnsuresOnThrow<TException>
This method makes guarantees about exceptional termination from a method. In general, such guarantees can only be made for very specific exceptions, so letting TException be Exception is not a good idea. For example, the following condition says that if an IOException escapes this method, the given variable is not null.

CodeContract.EnsuresOnThrow<IOException>(SomeSharedState != null);

Special values for postconditions
As you can imagine, it is often necessary to refer to certain values in postconditions, such as the result of the method, or the value of a variable at method entry. There are methods in the CodeContract class that allow this; they are valid only in a postcondition.
- CodeContract.Result<T>()
  This method represents the value returned from a method. The T parameter indicates the return type. For example, the following condition says that the result of this method is always non-negative.
  
  CodeContract.Ensures(CodeContract.Result<Int32>() >= 0);
- CodeContract.OldValue<T>(T value)
  This method represents the value as it was at the start of the method or property. It captures the pre-call value in a shallow copy. For example, this condition says that the method increased the value of SomeSharedInt.
  
  CodeContract.Ensures(SomeSharedInt > CodeContract.OldValue(SomeSharedInt));
- CodeContract.ValueAtReturn<T>(out T value)
  This method is just to let you refer to the final value of an out parameter in a postcondition. Since you write postconditions at the beginning of a method, the compilers would otherwise complain as the parameter hasn’t been assigned yet. Other by-reference parameters don’t need this method. Normal parameters don’t need this method because the compiler doesn’t care if it sees references to their values before it sees an assignment to them.
Object Invariants
The last major kind of contract that can be expressed with the CodeContract class is the object invariant. Object invariants are statements about what must be true at all public method exits for an object. It is up to the programmer to ensure these invariants are maintained. Object invariants are contained in a separate method that is marked with the ContractInvariantMethodAttribute. The name of the method does not matter, but it must be parameter-less and return void. That method contains some number of calls to the CodeContract.Invariant method. Here’s an example of an invariant stating that the value in Data is always non-negative.

        [ContractInvariantMethod]
      void ObjectInvariant() {
          CodeContract.Invariant(Data >= 0);
      }

CodeContract.Assert and CodeContract.Assume
I’ll briefly touch on the other two types of contracts included in the class. Assert and Assume, unlike the others, aren’t part of a method’s signature, and unlike the others they don’t come at the beginning of a method. They’re simply statements about what must be true at a particular point in the code. There exists many other ways to do this verification, but the main value with these methods is that they can be recognized by the tools. The difference between Assert and Assume is that an Assume is your way to tell the static checker that instead of trying to prove the expression to be true, it should add the expression to its body of facts for your code.

Please note that this is only what is in the CTP. This is a feature still under development, and things could change before we ship. In fact, we’re considering a couple of changes. One is changing the class name from CodeContract to Contract to make contracts more readable. Another is adding more sophisticated runtime failure behavior that is customizable by applications and hosts.

History and Design Decisions

The idea for this feature originated with the Spec# project in Microsoft Research. Spec# has its own programming language, an extension of C#, which includes contracts in the syntax of the language. We’re working closely with a team from Microsoft Research to bring contracts to the BCL. One of the main questions we’ve had about the way we are including contracts in the .NET Framework is why we chose to implement them as library calls instead of making them part of “the language.”

There are quite a few reasons for this, but one of them is that the CLR is the Common Language Runtime, and we wanted a feature that all of the languages using the .NET Framework could use, not just one of them. The best way to do this was with library calls. As far as compilers need to be concerned, these are just static method calls. Spec# was great because it had contracts integrated into the language, but that only solved the problem for one language.

People have brought up attributes as another way we could have accomplished this. One argument for that approach is that contracts are declarative, and this would make them look more like part of the method signature instead of the method body. The problem with this approach is that attributes are very limited in what they can express. It would be difficult or impossible to use attributes for all of the contracts you can write in code.

If you’re concerned about it because of the aesthetics, you might want to check out Scott Guthrie’s demo of VS2010, which starts about 88 minutes into this PDC2008 Keynote.

More Information

There are a few places you can go if you’re looking for more information.

Microsoft Research’s code contracts website.
For discussion of contracts, we’re currently sharing the Pex forum until we get our own. Please use the Code Contracts Community thread.
To provide feedback, you can make comments to this post, send mail via this blog’s contact form, or e-mail codconfb_at_microsoft_dot_com.

What's New in the BCL in .NET 4.0 CTP [Justin Van Patten]

BCLTeam — Wed, 05 Nov 2008 03:25:00 GMT

The Visual Studio 2010 and .NET Framework 4.0 CTP is available for download as of last week. The CTP contains new functionality throughout the .NET Framework, including several new BCL features and improvements:

Code Contracts
System.Diagnostics.Contracts provides a language-agnostic way to express coding assumptions in .NET programs. The contracts take the form of pre-conditions, post-conditions, and object invariants. Contracts act as checked documentation of your external and internal APIs. The contracts are used to improve testing via runtime checking, enable static contract verification, and documentation generation. We partnered with Microsoft Research to deliver this feature in .NET 4.0. More information and tools are available on Microsoft Research’s code contracts website. There’s also a highly-rated PDC session available online.
Parallel Extensions
We worked with the Parallel Extensions team to add the Task Parallel Library (TPL), Coordination Data Structures, and Parallel LINQ (PLINQ) to the BCL in .NET 4.0. This includes an improved ThreadPool scheduling algorithm for tasks, the static Parallel class, concurrent collections in System.Collections.Concurrent, and other coordination data structures such as LazyInit<T>, CountdownEvent, Barrier, SemaphoreSlim, SpinLock, SpinWait, and ManualResetEventSlim. More information is available over at the PFX team’s blog. Also, check out Daniel Moth’s PDC session on Parallel Programming for Managed Developers with the Next Version of Microsoft Visual Studio.
BigInteger
System.Numerics.BigInteger is an arbitrary-precision integer data type. We worked with the Microsoft Solver Foundation team to deliver a highly performant big integer implementation. BigInteger supports all the standard integer operations, including bit manipulation. It can be used from any .NET language, and some of the new .NET languages—such as F# and IronPython—have support built-in to the language.
Memory Mapped Files
System.IO.MemoryMappedFiles exposes the memory mapping functionality provided by Windows as first-class managed APIs. Memory mapped files can be used to efficiently edit very large files and can also be used to create shared memory for inter-process communication. Along with this feature, we’re also introducing System.IO.UnmanagedMemoryAccessor, a new class that enables random access to unmanaged memory similar to how UnmanagedMemoryStream enables sequential access to such memory.
ResourceManager Improvements
The ResourceManager in System.Resources has been improved to respect the user’s preferred UI languages when looking for localized resources, instead of only using the CurrentUICulture’s parent chain. This means if the user has specified that she prefers French and Spanish, the ResourceManager will look for French and Spanish resources before falling back to the neutral resources. This change is present in Silverlight 2 as well as .NET 4.0.
Compression Improvements
The compression algorithms in System.IO.Compression have been improved in .NET 4.0. DeflateStream and GZipStream no longer inflate already compressed data. This means that in most cases you’ll see much better compression ratios when using these streams on .NET 4.0. We’ve also removed the 4 GB size limit, so you can now compress streams over 4 GB in size.
String Security Changes
The default partial matching overloads on System.String (StartsWith, EndsWith, IndexOf, and LastIndexOf) have been changed to be culture-agnostic (ordinal) by default. In addition, ToUpper and ToLower on System.String and System.Char have been changed to use the invariant culture instead of the current culture. Although we have ~~guidance~~ ~~and~~ ~~FxCop rules~~ that recommend always using overloads that take a StringComparison parameter, unaware developers often just use the default overloads. In previous versions of .NET, these default overloads do a culture-sensitive comparison using the current culture. This can often lead to subtle bugs, most notably security vulnerabilities, when unaware developers use the default overloads to do security-sensitive string comparisons. This change helps mitigate these vulnerabilities. The change is present in both Silverlight 2 and .NET 4.0. Even with these changes, our guidance still stands: whenever an overload exists that takes a StringComparison parameter, use it instead of an overload that does not take this parameter. It makes your code clearer and easier to maintain. This is especially important because the default overloads for String.Compare and String.CompareTo will remain culture-sensitive because these methods are most often used when sorting strings to be shown to the user. We plan to add a compat switch in the beta that will allow an app to specify whether it wants the old behavior.

UPDATE for .NET 4 Beta 1 In order to maintain high compatibility between .NET 4 and previous releases, we have decided to revert this change. The behavior of String's default partial matching overloads and String and Char's ToUpper and ToLower methods now behave the same as they did in .NET 2.0/3.0/3.5. The change back to the original behavior is present in .NET 4 Beta 1. We apologize for any interim confusion this may cause. We continue to recommend being explicit about the string comparison behavior you want, by always specifying a StringComparison value for the methods on String that accept it.

We’re also evaluating a number of potential new features and improvements for .NET 4.0 beta:

Variance annotations
The next versions of C# and VB support safe co- and contra-variance for generic interface and delegate types. Co-variance means that a generic of a type, e.g. an IEnumerable<String>, can be treated as a generic of any supertype, e.g. an IEnumerable<Object>. Contra-variance means that a generic of a type, e.g. an Action<Object>, can be treated as a generic of a subtype, e.g. an Action<String>. In C#, co-variance is annotated with the “out” keyword and contra-variance is annotated with the “in” keyword. We are annotating several interfaces and delegates in the BCL for variance. You can learn more about co- and contra-variance in Anders Hejlsberg’s PDC session on The Future of C#.
Tuples
We are providing common tuple types in the BCL to facilitate language interoperability and to reduce duplication in the framework. A tuple is a simple generic data structure that holds an ordered set of items of heterogeneous types. Tuples are supported natively in languages such as F# and IronPython, but are also easy to use from any .NET language such as C# and VB.
SortedSet<T>
We plan to add a SortedSet<T> collection along with an ISet<T> interface. SortedSet<T> uses a self-balancing tree which maintains data in sorted order for performance guarantees with insertion, deletion, and searches. Both the new SortedSet<T> and the existing HashSet<T> implement ISet<T>.
File System Enumeration Improvements
We plan to add new file system enumeration APIs to System.IO.Directory and System.IO.DirectoryInfo that return IEnumerable<T>’s instead of arrays. These new APIs are more efficient than the array-based APIs because they do not need to allocate a (potentially large) array and you can access the first results immediately instead of waiting for the entire enumeration to take place. We’re also planning to add new convenience APIs for efficiently reading, writing, and appending lines from/to a text file using IEnumerable<String>. These new APIs are useful in LINQ scenarios where you may want to quickly and efficiently query the contents of a text file and write out the results to a log file without allocating any arrays.

There are also a bunch of improvements to the CLR in .NET 4.0. Here’s a high-level summary:

In Process Side-by-Side: support for multiple CLR versions running in the same process.
Interop improvements: no PIAs (check out Under the Hood: Advances in the .NET Type System and Managed and Native Code Interoperability: Best Practices)
Simplified security model
GC improvements (notifications and low-latency background collections)
Profiling improvements (ability to attach/detach performance and memory profilers on the server)
Debugging enhancements (support for Windows Error Reporting mini-dumps)

You can learn more about the next version of the CLR in Joshua Goodman’s PDC session on Microsoft .NET Framework: CLR Futures.

Over the next couple of weeks we’ll be posting more about the new functionality that’s available in the CTP. Do note that we’re working on many other improvements for 4.0 that we’re not quite ready to announce just yet.

As always, we’d love to hear what you think of the CTP and announcements so far.

BCL Team at PDC 2008 [Justin Van Patten]

BCLTeam — Sat, 25 Oct 2008 06:51:00 GMT

Melitta, Kim, and I will be at PDC next week, along with several other members of the CLR team. We’ll mostly be hanging out in the lounges chatting with folks and answering questions. We’ll also be at the Ask-the-Experts night on Wednesday, so if you’ll be at PDC next week, stop by and say hi! We love chatting with customers and listening to your feedback and suggestions.

This year’s PDC has tons of different sessions on a vast array of .NET technologies including ASP.NET, WCF, WF, ADO.NET, Entity Framework, Data Services, XAML, WPF, Silverlight, and many more. Here are some that you may find of interest:

The first five sessions listed above are the ones most closely related to the BCL and CLR; the first two will specifically touch on some of the features that our team (the BCL team) is delivering in .NET 4.0. Of course there are plenty of other topics to keep an eye on, such as: parallelism, “Oslo”, Visual Studio, cloud computing, Windows 7, Internet Explorer, SQL Server, etc.

Stay tuned to this blog next week for more details on what we’ll be unveiling next week at PDC 2008.

The Compare Contract [Kim Hamilton]

BCLTeam — Mon, 06 Oct 2008 18:29:00 GMT

A breaking change?

We recently heard from a customer who observed different sorting behavior in .NET FX 3.5 SP1 compared to 3.5 RTM.

The different behavior was demonstrated with the following code. The class StringWrapper provided a custom sort in which nulls (null StringWrapper references) were moved to the end of the array. To achieve this, StringWrapper implemented IComparable<StringWrapper> and in its CompareTo method, nulls were always greater than non-nulls.

public class MyClass {
    public static void Main() {
        StringWrapper a = new StringWrapper();
        a.Value = "a";
        StringWrapper b = new StringWrapper();
        b.Value = "b";
        StringWrapper c = new StringWrapper();
        c.Value = "c";
        StringWrapper d = null;

        Console.WriteLine("Sort 1:");
        StringWrapper[] src1 = new StringWrapper[] { a, c, d, b };
        Array.Sort(src1);
        PrintStringWrappers(src1); // print elements, method included at end
    }
}

public class StringWrapper : IComparable<StringWrapper> {
    private string _value;
    public string Value {
        get { return _value; }
        set { _value = value; }
    }

    // Recall that CompareTo returns:
    // <0 if this object is less than other
    //  0 if this object is equal to other
    // >0 if this object is greater than other
    public int CompareTo(StringWrapper other) {
        if (other == null) return -1; // nulls are greater than any non-null
        return Value.CompareTo(other.Value);
    }

    public override string ToString() {
        return Value;
    }
}

This custom comparison apparently worked in .NET FX 3.5 RTM, but not .NET FX 3.5 SP1.

3.5 RTM output:

a
b
c
null

3.5 SP1 output:

a
b
null
c

Did the custom comparer really work?

The custom comparer worked in that example, in which only one of the array elements was null. Let’s throw in another null and see what happens:

Console.WriteLine("Sort 2:");
StringWrapper[] src2 = new StringWrapper[] { a, d, d, c, b };
Array.Sort(src2);
PrintStringWrappers(src2);

3.5 RTM output:

a
b
null
null
c

The problem is in the CompareTo method, but it isn’t obvious. The first line in CompareTo actually violates the IComparable<T>.CompareTo contract that any object compares greater than a null reference.

public int CompareTo(StringWrapper other) {
    if (other == null) return -1; // violates CompareTo contract!
    return Value.CompareTo(other.Value);
}

The Compare Contract

IComparable<T>.CompareTo has the following requirements (from the MSDN docs)

For objects A, B, and C, the following must be true:

A.CompareTo(A) is required to return zero.
If A.CompareTo(B) returns zero, then B.CompareTo(A) is required to return zero.
If A.CompareTo(B) returns zero and B.CompareTo(C) returns zero, then A.CompareTo(C) is required to return zero.
If A.CompareTo(B) returns a value other than zero, then B.CompareTo(A) is required to return a value of the opposite sign.
If A.CompareTo(B) returns a value x that is not equal to zero, and B.CompareTo(C) returns a value y of the same sign as x, then A.CompareTo(C) is required to return a value of the same sign as x and y.

By definition, any object compares greater than a null reference (Nothing in Visual Basic), and two null references compare equal to each other.

The requirement that any object compares greater than null is a bit of a footnote at the end, so it makes sense that this may not be well known (we should highlight this more).

Still, why did the behavior change? (Gory details and a 3.5 RTM performance bug)

The high-level problem is that .NET’s sorting makes assumptions based on the Compare contract, so in some cases sorting will special case null, because it “knows” your comparer will adhere to the contract and return values consistent with that assumption. If you don’t adhere to the contract, you’ll get bitten at some point.

The details are messier. Over the years, a variety of changes were made to improve the performance of Sort. There was a brief window (released in 3.5 RTM) where, during QuickSort, the swap and pivot steps were broken, and in intermediate steps, it would actually unsort certain already sorted arrays (the array had to contain null). The end result would be correct, but because the elements were incorrectly swapped, sorting took much longer than it should.

We fixed this bug in SP1, and this fix caused the different behavior between RTM and SP1.

Comply with Compare contract

The only way to ensure stability across versions is to comply with the Compare contract. Otherwise you fall prey to implementation quirks (or even bugs) in the runtime sort implementation. We’d like to be free to change it between releases, because we’d like to keep improving the performance!

Enough preaching, what are my options?

The interesting thing is that the StringWrapper implementation gets you mostly there. The apparent goal of the wrapper is to implement a special sort of strings that pushes nulls to the end. You can do what you want if you create a StringWrapper with null values, as follows:

public int CompareTo(StringWrapper other) {
    if (other == null) return 1;
    if (Value == null && other.Value == null) return 0;
    if (Value == null) return 1;
    if (other.Value == null) return -1;
    return Value.CompareTo(other.Value);
}

This actually obeys the compare contract and won’t be brittle to changes in our sort implementation.

Full Source

using System;
using System.Collections.Generic;

public class MyClass {
    public static void Main() {

        StringWrapper a = new StringWrapper();
        a.Value = "a";
        StringWrapper b = new StringWrapper();
        b.Value = "b";
        StringWrapper c = new StringWrapper();
        c.Value = "c";
        StringWrapper d = null;
        StringWrapper e = new StringWrapper();
        e.Value = "e";
        StringWrapper f = new StringWrapper();
        f.Value = "f";


        Console.WriteLine("Sort 1:");
        StringWrapper[] src1 = new StringWrapper[] { a, c, d, b };
        Array.Sort(src1);
        PrintStringWrappers(src1);
        // .NET FX 3.5 RTM output: a b c null
        // .NET FX 3.5 SP1 output: a b null c

        Console.WriteLine("-----");
        Console.WriteLine("Sort 2:");
        StringWrapper[] src2 = new StringWrapper[] { a, d, d, c, b };
        Array.Sort(src2);
        PrintStringWrappers(src2);
        // .NET FX 3.5 RTM output: a b null null c
    }

    private static void PrintStringWrappers(StringWrapper[] swArray) {
        foreach (StringWrapper sw in swArray) {
            if (sw == null)
                Console.WriteLine("null");
            else
                Console.WriteLine(sw);
        }

    }
}

public class StringWrapper : IComparable<StringWrapper> {
    private string _value;
    public string Value {
        get { return _value; }
        set { _value = value; }
    }

    // original CompareTo -- violates Compare contract!
    public int CompareTo(StringWrapper other) {
        if (other == null) return -1;
        return Value.CompareTo(other.Value);
    }

    // alternate CompareTo, which obeys Compare contract and moves null Values to the end
    /*
    public int CompareTo(StringWrapper other)
    {
        if (other == null) return 1;
        if (Value == null && other.Value == null) return 0;
        if (Value == null) return 1;
        if (other.Value == null) return -1;
        return Value.CompareTo(other.Value);
    }
    */

    public override string ToString() {
        return Value;
    }
}