Charlie Calvert's Community Blog : LinqFarm

HDI Video: Generate from Usage in Visual Studio 2010 with Karen Liu

Charlie Calvert — Sat, 07 Nov 2009 23:45:00 GMT

Karen Liu, the Lead Program Manager for the Visual C# and Visual Basic IDEs, has created a new video on Generate from Usage (GFU), a feature found in Visual Studio 2010, Beta 2. This post recaps what is included in the video, including the sections that show how GFU can be used to enhance test first development. The video is shot in both VB and C#. Since this is a C# blog, I’ll show only C# code, and will translate the VB code that Karen shows into C#.

Download the source code from the LINQ Farm

Generate from Usage (GFU) is a new feature in 2010 that allows you to write code that consumes a library or API before that API even exists. Using Visual Studio menus or shortcuts, you can automatically generate classes, constructors, methods, fields and properties from the code you typed in the editor.

As the video begins, Karen first types in the code to initialize a class called Automobile that does not yet exist:

Code Snippet
 var myCar = new Automobile(Make: "Honda", Model: "Accord");
 

Notice that the class name Automobile is not colored in teal, which is Visual Studio’s way of telling you that it does not yet it exist. When seen inside Visual Studio, the word Automobile will also have red squiggly (or wavy) underline, and a small blue smart tag under the letter A, as shown in Figure 1.

Figure 1: The red squiggle is Visual Studio’s way of telling you that the type Automobile will not compile. The blue smart tag under the letter A let’s the user know that an expansion tip is available by simultaneously pressing the Control key and a period (Ctrl + .).

The smart tag under the letter A is Visual Studio’s way of telling you to press the control and period keys to bring up a special expansion tip, as shown in Figure 2. Here we see two options, one for directly creating a class called Automobile, and the other for bringing up a dialog which allows us to define or tweak the details of the type of we create.

Figure 2: Viewing an expansion tip in Visual Studio.

You can also bring up the options to create a new class or type by right clicking on the word Automobile and selecting Generate from the popup menu, as shown in Figure 3.

Figure 3: A second way to generate a class is to right click the word Automobile and choose Generate from the popup menu.

If you select the option to create a new class, then Visual Studio will automatically generate a new file called Automobile.cs and place inside it a new class called Automobile. The entire generated file is shown in Listing 1.

Listing 1: The code generate when you choose to create a new class based on an undefined identifier in Visual Studio 2010.

Code Snippet
 using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
 
namespace DeleteMe
{
    class Automobile
    {
    }
}
 

You can now go back to your original source file where you will see that there is still a red squiggly and blue tool tip, as shown in Figure 4. These hints are shown because we have not yet created a constructor for the Automobile class. If we again press control plus period, then the option to generate these code elements is made available to us, as shown in Figure 4.

Figure 4: Selecting the smart tag a second time brings up an option to automatically generate the constructor and associated fields for your class.

The code that is generated by selecting the expansion tip is shown in Listing 2. Notice that on lines 10 and 11 fields were created for your class and on lines 16 and 17 code was generated inside the constructor for initializing them. A comment in the form of a TODO item is also added to your class. These TODO items are visible in the Visual Studio Task List. You can access the Task List by choosing View | Task List from the menu, or by pressing Ctrl+W, T. Be sure to choose the Comments option from the drop down at the top of the Task List.

Listing 2: The code for the Automobile’s constructor is automatically generated by the IDE.

Code Snippet
 using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
 
namespace GenerateFromUsageSample
{
    class Automobile
    {
        private string Make;
        private string Model;
 
        public Automobile(string Make, string Model)
        {
            // TODO: Complete member initialization
            this.Make = Make;
            this.Model = Model;
        }
    }
}
 

Karen next types in code for a TurnLeft method, and for a property called IsFacingNorth. As she types each item, a smart tag appears, and again she has the option to press Ctrl+. to automatically generate code for the Automobile class. The code typed in the program’s entry point is shown in Listing 3, and the code generated by the IDE is shown mostly in Listing 4. Notice, however, that code for a private object called distance was automatically generated in the Main method. The IDE actually gives you the option to create either a private field, as shown here, or a public property.

Listing 3: The complete code for Karen’s first sample includes a constructor, a method called TurnLeft, and a property called IsFacingNorth.

Code Snippet
  
namespace GenerateFromUsage
{
    class Program
    {
        private static object distance;
        static void Main(string[] args)
        {
            var myCar = new Automobile(Make: "Honda", Model: "Accord");
            myCar.TurnLeft(distance);
            myCar.IsFacingNorth = true
        }
    }
}
 

Listing 4: The code generated by the IDE for the TurnLeft method and the IsFacingNorth property.

Code Snippet
 using System;
 
namespace GenerateFromUsage
{
    class Automobile
    {
        private string Make;
        private string Model;
 
        public Automobile(string Make, string Model)
        {
            // TODO: Complete member initialization
            this.Make = Make;
            this.Model = Model;
        }
 
        internal void TurnLeft(object distance)
        {
            throw new NotImplementedException();
        }
 
        public bool IsFacingNorth { get; set; }
    }
}
 

When looking at Listing 4, notice that code for generating a NotImplementedException is automatically generated in the TurnLeft method. The IDE was also smart enough to discern that the IsFacingNorth property returns a bool. All the lines of code shown in Listing 4 were generated by typing just three lines of code in the main method found in Listing 3.

The C# Side

In the second part of the video Karen shows how you can use generate from usage (GFU) to enhance your experience when creating unit tests. Many theorists advocate using a test first methodology in which you first create a test, and then write the code that you want to test. You will see that GFU can be used to make this style of development quite natural and simple to use inside of Visual Studio.

NOTE: This section of the post requires high end versions of Visual Studio that contain the test wizards. Even if you don’t have those tools, you can still follow along to see how Generate from Usage can be used in all versions of Visual Studio to make it easier to create unit tests.

Begin by starting a new console application called Customer. Choose Tests | New Test from the Visual Studio menu. Select Basic Unit Test from the dialog, and name it Customers, as shown in Figure 5. When you press OK in the Add New Test dialog you will be prompted for the name of your unit test project. Type in CustomerTests. When you are done, the Solution Explorer for your project should look as it does in Figure 6.

Figure 5: Create a new unit test project that will house a file called Customers.

Right click on the Customer node in the solution explorer and choose Add | New Folder to create a new directory called Models. When you are done you should see a new node in the solution explorer, as shown in Figure 6.

Figure 6: This solution contains two projects, one a simple console application called Customer that contains a folder called Models. The other project is called CustomerTests and it is designed to hold unit tests.

I want to check if my Customer list is create correctly. In the Customers file from your test project, create a method called IsCustomerListValid. Inside the method create code to initialize a CustomerList object, as shown in Listing X.

Listing 5: A simple unit test with the code to initialize a class called CustomerList. Note that the CustomerList is not colored in Teal, since its declaration has not been created yet.

Code Snippet
 using System;
using System.Text;
using System.Collections.Generic;
using System.Linq;
using Microsoft.VisualStudio.TestTools.UnitTesting;
 
namespace CustomerTests
{
    [TestClass]
    public class Customers
    {
        [TestMethod]
        public void IsCustomerListValid()
        {
            CustomerList cust = new CustomerList();
        }
    }
}
 

At this stage, there is no CustomerList, so in Visual Studio you will again see the red squiggles and the blue smart tag, just as I showed in Figure 2. Without needing to take your hands off the keyboard, you can press control plus dot to bring up the expansion tips as in Figure 2. This time select New Type. A dialog comes up like the one shown in Figure 7. In the dialog we can change the accessibility, the kind of code to generate and the project file and directory where we want to place it. Set the Access to public and the Kind to class. Use the Project drop down to select the Customer project and the the Create new file drop down to select the Model directory. Type in CustomerList.cs as the name of the file to create.

Figure 7. Selecting the kind of type that you want to create, as well as the project, folder and file in which you want to place it.

When you press OK in the Generate New Type dialog, the new type will be created, and the IDE will add the using directive and the correct references. You can now fill out your test and your generated class as shown in Listing 6 and 7. Note that we have used generate from usage to enhance the CustomerList class with a method called Add. I then manually wrote code to create a list into which the text we pass can be inserted.

Listing 6: A simple unit test.

Code Snippet
 using Customer.Models;
using Microsoft.VisualStudio.TestTools.UnitTesting;
 
namespace CustomerTests
{
    [TestClass]
    public class Customers
    {
        [TestMethod]
        public void IsCustomerListValid()
        {
            CustomerList cust = new CustomerList();
            cust.Add("Karen");
            Assert.IsNotNull(cust);
        }
    }
}
 

Listing 7: The using directive, namespace, CustomerList class and header for the Add method were auto-generated. I manually created the list and inserted the call to Add an item into it.

Code Snippet
 using System.Collections.Generic;
 
namespace Customer.Models
{
    public class CustomerList
    {
        List<string> list = new List<string>();
 
        public void Add(string p)
        {
            list.Add(p);
        }
    }
}
 

You can now use the Test | View menu to bring up the Test List Editor and Test View to select and run your test.

Summary

In this post you got a second look at the code used in Karen Liu’s generate from usage video. You saw that GFU can be used to create classes, constructors, parameters, properties and fields. You also explored the powerful Generate New Type dialog which gives you the flexibility to choose the kind of type you want to create, as well as the project, directory and file in which you want to insert it. Finally, you saw that generate from usage can be powerful aid when you are engaged in test first development.

Resources

Silverlight Revisited

Charlie Calvert — Fri, 03 Apr 2009 04:56:05 GMT

I’ve written about Silverlight in previous posts published on this blog. Recently, however, I’ve found my thoughts coming back to this subject. This is hardly unusual: many people are interested in Silverlight.

The reason for the attraction to this technology is not hard to discover. Silverlight allows us to build visually appealing web applications with a sophisticated, well designed programming language such as C#. Though there are now many sophisticated tools for creating web applications, but most of them still depend on fragmented models that force us to integrate multiple, hard to use technologies into a patchwork design that is poorly integrated. Applications built on these models can be hard to debug, and hard to maintain. They also lack the ability to easily support rich interfaces that would be taken for granted in a desktop application.

I’ve spent most of my programming career working in either Delphi or C#, and I have a huge respect for the elegance of these languages, and in the power of object-oriented design to help us break complex problems into manageable, easy to maintain solutions. It is, of course, easy to create poorly designed C# programs. Nevertheless, I believe that C# in particular and Silverlight in general has the capacity, the potential, to support elegant solutions in a way that many web technologies do not.

In short, I’m interested in Silverlight for two reasons:

It allows me to create a rich interface with sophisticated visual techniques including animation.
It provides a thoroughly modern programming model, and a set of rich API’s for performing common programming tasks.

So there, whether one agrees with me or not, is my motivation to revisit Silverlight. Once I had found my inspiration, my next step was to visit the Silverlight site, and download the tools to integrate Silverlight into Visual Studio 2008. The installation worked smoothly on a copy of XP that I had running in a virtual machine. Though I haven’t done it yet, I also intend to download Expression Blend.

Once I had a set of Silverlight tools integrated into Visual Studio, I set about experimenting with them. After a bit, I came up with the application shown in Figure 1. Not bound by such quotidian obligations as having a practical purpose, or a pleasing interface, it didn’t take me long to put this program together. It’s only redeeming feature was that it gave me a chance to experiment with several Silverlight technologies including:

XAML and WPF
A tool called a StackPanel, which helped me organize the interface, which consists primarily of simple controls such as buttons and text areas.
Two different types of Canvases on which I could I draw.
Several different graphical shapes and paths, as well as the Ink technology handed down from Tablet PC.
A simple LINQ query.

Figure 1: A simple Silverlight program, running in the Internet Explorer, which demonstrates how to use several key technologies.

In my next post, I’ll talk some about the code for this application, and describe how its various parts fit together. Overall, I found putting this code together was blessedly simple, and overall quite a bit of fun. At least from what I’ve seen so far, one of Silverlight’s great strengths is its ease of use.

Download the SilverlightDrawingTest from the LINQ Farm.

Query Data with Parallel LINQ

Charlie Calvert — Tue, 16 Dec 2008 09:00:26 GMT

This post shows a simple way to write code that takes advantage of multiple processors. You will see that LINQ queries can allow you to side step the difficult tasks normally involved in writing multi-threaded code. To get started, all you need is a little basic knowledge of how to write simple LINQ queries.

The code shown in this post uses a pre-release version of PLINQ called the Microsoft Parallel Extensions to .NET Framework 3.5. When PLINQ finally ships, it will run only on .NET 4.0 or later. The version I'm using that runs on top of 3.5 is for evaluation purposes only. There will never be a shipping version that runs on .NET 3.5.

This LINQ provider is being created at Microsoft by the Parallel Computing team; it is not the work of the C# team that created LINQ to Objects and LINQ to SQL. Here is the website for the Parallel Computing team:

http://msdn.microsoft.com/en-us/concurrency/

At the time of this writing, these extensions were available only in pre-release form. You could download them either as Visual Studio 2008 compatible extensions to .NET 3.5, or as part of the pre-release version of Visual Studio 2010. Since the download sites might change over the coming months, I suggest that you find these resources by going to the Parallel Computing site, or to the Visual Studio site:

http://msdn.microsoft.com/en-us/vs2008

Parallel LINQ, or PLINQ, is only a small part of the Parallel Extensions to the .NET Framework. It is, however, an important part. Since it is a simple and natural extension of the LINQ syntax, I think developers familiar with that technology will find it easy to use.

Consider this code:

var list = Enumerable.Range(1, 10000);

var q = from x in list.AsParallel()
        where x < 3300
        select x;

foreach (var x in q)
{
    Console.WriteLine(x);
}

These lines look nearly identical to the code you have seen in many simple LINQ samples. The only significant difference is the call to AsParallel at the end of the first line. Though we have often used type inference to hide the return type of a LINQ query, I'm going to pause and take a second look at this instance. Rather than returning IEnumerable<T>, this version of PLINQ returns IParallelEnumerable<int>:

IParallelEnumerable<int> q = from x in list.AsParallel() etc….

In the near future, PLINQ queries of this type will probably return ParallelQuery<int>. Because this product is still evolving, it might be simplest to use var, at least during the pre-release phase, and let the compiler choose the type. That way you can save typing, avoid problems with anonymous types, and you need not concern yourself about changes in the API as the product develops. It is almost always appropriate to use var to designate the return type of a LINQ query, and there are only special circumstances when you would do otherwise.

Here are the results from this first PLINQ query:

The numbers shown here are in a relatively random order because they are being returned from different threads. It is important to remember that the sequence of values returned by LINQ is not always guaranteed to be presented in a particular order. If Order is important in your code, you can add a call to AsOrdered to the query after the call to AsParallel. Alternatively, you could insert a GroupBy clause to establish the desired ordering. Otherwise developers should assume that the ordering from a PLINQ query will be entirely random

Now that you understand the basics of Parallel LINQ, let’s move on to look at a more interesting example. Improved performance is the main reason to write code that can run in parallel. The program shown in this post uses a timer to demonstrate how PLINQ can improve performance in a program.

Performance improvements become more evident when our code has access to more processors. The code I show here runs faster on a two processor machine, but it really starts to come into its own on a four processor machine. Moving up to even more processors yields more powerful results. Here, for instance, are the results showing an improvement of 1.33 times when using two processors, and almost two times when using 4 processors:

2 Processors = 1.44 x improvement:
Linear: 00:00:13.15
Parallels: 00:00:09.10

4 Processors = 1.96 x improvement:
Linear: 00:00:15.00
Parallel: 00:00:07.68

These tests are being running against pre-release software, so these numbers are almost certain to change before release, and of course different machines will yield different results. Furthermore, the degree of improvement that you see is likely to change depending on the type of algorithm you run, the number of cores on your machine, the architecture of the machine, how many caches there are and how they’re laid out, etc. Though it is rare, some queries show superlinear performance enhancements. In other words, there is a greater than 4x speedup on a 4-core box. An improvement of 2 times, such as the one shown, or even a 3 time improvement, is common.

This sample program is called FakeWeatherData, and it is available for download from the LINQ Farm on Code Gallery. It features a simple LINQ to XML query run against a file with 10,000 records in it. The data I'm querying is not real, but consists of random dates and temperatures generated by a simple algorithm included in the FakeWeatherData program.

The XML file is structured like this:

<?xml version="1.0" encoding="utf-8" ?>
<Samples>
  <Sample>
    <Year>1973</Year>
    <Month>May</Month>
    <Day>15</Day>
    <Temperature>10</Temperature>
  </Sample>
  <Sample>
    <Year>1970</Year>
    <Month>Feb</Month>
    <Day>10</Day>
    <Temperature>14</Temperature>
  </Sample>
  <Sample>
    <Year>1970</Year>
    <Month>Jan</Month>
    <Day>15</Day>
    <Temperature>11</Temperature>
  </Sample>

  ... Many lines of code omitted here

</Samples>

There is also a simple C# class used by the program to encapsulate the data from the XML file:

class WeatherData
{
    public string Year { get; set; }
    public string Month { get; set; }
    public string Day { get; set; }
    public string Temperature { get; set; }
}

The parallel version of the query in the program looks like this:

for (int i = 0; i < NUM_REPS; i++)
{
    var list = (from x in doc.Root.Elements("Sample").AsParallel()
                where x.Element("Year").Value == "1973" &&
                   x.Element("Month").Value == "Apr" &&
                   x.Element("Day").Value == "15"
                select new WeatherData
                {
                    Day = x.Element("Day").Value,
                    Month = x.Element("Month").Value,
                    Temperature = x.Element("Temperature").Value,
                    Year = x.Element("Year").Value
                }).ToList();

}

Accompanying this code is a similar LINQ query that does not use PLINQ

for (int i = 0; i < NUM_REPS; i++)
{
    var list = (from x in doc.Root.Elements("Sample")
                where x.Element("Year").Value == "1973" && 
                   x.Element("Month").Value == "Apr" && 
                   x.Element("Day").Value == "15"
                select new WeatherData
                {
                    Day = x.Element("Day").Value,
                    Month = x.Element("Month").Value,
                    Temperature = x.Element("Temperature").Value,
                    Year = x.Element("Year").Value
                }).ToList();

}

The program queries the data in the XML file first using the Parallel code, then using standard LINQ. By comparing the time it takes each block of code to execute you can get a sense of the relative improvement available through PLINQ. I'll show you how to make such comparisons in just a moment. I will also discuss some tools that will become available to help profile code of this type.

You can see that the PLINQ query contains a call to AsParallel, while the other query does not. Other than that the two queries are identical. The fact that the two queries look so much alike points to a primary strength of PLINQ: very little specialized knowledge is necessary in order to begin using it. This does not mean that the subject is trivial, but only that the barrier to entry is low. This is not the case with most concurrent programming models.

LINQ queries are designed to be read-only, working with immutable data. This is a good model for parallelism, because it makes it unlikely that data will mutate, thereby setting up the potential for a race condition. You should note, however, that PLINQ does nothing to prevent this from happening, it is simply that LINQ is designed to make it unlikely.

Note also that the declarative LINQ programming style ensures that developers specify what they want done, rather than how it should be done. This leaves PLINQ free to ensure that concurrent LINQ queries run in the safest manner possible. If LINQ had been defined more strictly, such that it had to process each element in a certain order, then the PLINQ team would have had a much more difficult task.

The code in both these queries pulls out only the records from the XML file that have their date set to April 15, 1973. Because of deferred execution, the query would not do anything if I did not call ToList(). As a result, I added that call and converted the result into a List<WeatherData>. Though hardly earthshaking in import, these calls ensure that the code actually does something, and thus gives PLINQ scope to take advantage of the multiple processers on your system.

Simple timers are created to measure the difference between the standard LINQ query and the PLINQ query. I've also used a method used in many of Parallel LINQ team's samples for displaying the time elapsed during a test run:

private static void RunTest()
{
    XDocument doc = XDocument.Load("XMLFile1.xml");

    Stopwatch sw = new Stopwatch();

    sw.Start();
    LinqOrdinarie(doc);
    sw.Stop();
    ShowElapsedTime("Ordinaire", sw.Elapsed);

    sw.Reset();

    sw.Start();
    ParallelLinq(doc);
    sw.Stop();
    ShowElapsedTime("Parallels", sw.Elapsed);
}

private static TimeSpan ShowElapsedTime(string caption, TimeSpan ts)
{
    string elapsedTime = String.Format("{0}: {1:00}:{2:00}:{3:00}.{4:00}",
        caption, ts.Hours, ts.Minutes, ts.Seconds,
        ts.Milliseconds / 10);
    Console.WriteLine(elapsedTime, "RunTime");
    return ts;
}

At least with the pre-release version of PLINQ that I've played with, I've found it very useful to set up timers to confirm that PLINQ is actually able to speed up an operation. My record at guessing which code will benefit from running in parallel is not good, and so I find that confirming the effectiveness of the code by explicitly measuring it is worthwhile. You can either use the simple StopWatch class from the System.Diagnostics namespace, as shown here, or else you can use a profiler. Note that a thread aware profiler might ship with some versions of Visual Studio 2010.

I've found that the advantages of concurrent LINQ become more obvious the longer the operation I'm timing lasts. As a result, I've placed the query inside a loop, and added a variable to the program called NUM_REPS. By setting NUM_REPS to a large number, say 500, you can clearly see the benefits that can be accrued when you run LINQ queries in parallel on multiple processors. Note that the first time PLINQ is used, its assembly will need to be loaded, the relevant types will need to be JIT compiled, and new threads will need to be spun up, etc. As a result, many developers see improved performance after they get past the initial warm-up time.

Though it is very easy to get started with PLINQ, there are still complexities inherent in the subject that you need to consider. For instance, PLINQ will sometimes develop a different partitioning scheme for your data depending on whether you are working with an Enumerable or an Array. To learn more about this subject, see the following post from the Parallel Programming team:

http://blogs.msdn.com/pfxteam/archive/2007/12/02/6558579.aspx

The simple PLINQ examples shown in this post should help you get started with this powerful and interesting technology. Parallel LINQ is still in its infancy, but already it provides means of greatly simplifying tasks that are not normally easy to perform.

LINQ Farm: Covariance and Contravariance in C# 4.0

Charlie Calvert — Tue, 28 Oct 2008 21:04:00 GMT

This post covers the upcoming C# 4.0 support for covariance and contravariance when working with delegates and interfaces. Eric Lippert’s series of posts on this subject are definitely the definitive reference at this time. I’m writing this overview of the subject simply as an appendix to his explanation, and as quick reference for folks who want to get up to speed on this technology. Please remember that this post covers pre-beta technology as defined in the October 2008 CTP of Visual Studio 2010.

The support for covariance and contravariance in the next version of the C# language will ensure that delegates and interfaces will behave as expected when you are working with generic types. In Visual Studio 2008, there are rare occasions when developers might expect a delegate or interface to behave one way, only to find that it does not conform to their expectations. In Visual Studio 2010 delegates will behave as expected. Other C# types support have always automatically supported covariance and contravariance. C# 4.0 will simply ensure that generic delegates and interfaces follow suit.

Consider this simple class hierarchy:

class Animal{ }
class Cat: Animal{ }

Here we have a class called Animal, and a simple descendant of it called Cat. Suppose you declare a delegate declaration that defines a method that returns an arbitrary type:

delegate T Func1<T>();

An instance of this delegate could be defined to return a Cat:

Func1<Cat> cat = () => new Cat();

The delegate declaration shown here defines a delegate that returns a type T. The second line of code initializes T to be of type Cat and assigns this delegate to a simple lambda that creates an instance of a cat and returns it. For those of you not yet comfortable with lambda syntax, I’ll add that the lambda shown here is simply a shorthand way of writing a method that looks like this:

public Cat MyFunc()
{
    return new Cat();
}

So one could also have written the cat delegate like this:

Func1<Cat> cat = MyFunc;

Given either of these declarations, our intuition tells us that we could assign a cat to a delegate that returns an Animal:

Func1<Animal> animal = cat;

This code looks like it should succeed because a Cat is type of Animal, and one should always be able to assign a smaller type of object, such as a Cat, to a larger type of object, such as a Animal. We think of the type Animal as being large, since a creature such as a Cat, Dog, Whale or Bird would be a type of Animal, and hence compatible with it. In other words, a big type like an Animal is assignment compatible with lots of smaller types, such as a Cat, Dog or Whale. We think of the type Cat as being smaller than an Animal, since only other cats can be assigned to it. It is not assignment compatible with as wide a range of types as an Animal, and hence we think of it as being smaller.

In fact, the assignment shown above does not work in Visual Studio 2008. The new feature being added in Visual Studio 2010 ensures that this assignment will work if you make one minor change to the declaration for your delegate. In particular, you need to use the keyword out in your type parameter:

delegate T Func1<out T>();

Now the assignment will succeed, and the code will run as expected, as shown in Listing 1. This type of assignment is called covariance.

Listing 1: Examples of covariance and contravariance.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace SimpleVariance
{
    class Animal { }
    class Cat: Animal { }

    class Program
    {
        // To understand what the new CoVariance and ContraVariance code does for you
        // Try deleting or adding the words out and in from the following 2 lines of code:
        delegate T Func1<out T>();
        delegate void Action1<in T>(T a);

        static void Main(string[] args)
        {
           // Covariance
            Func1<Cat> cat = () => new Cat();
            Func1<Animal> animal = cat;

            // Contravariance
            Action1<Animal> act1 = (ani) => { Console.WriteLine(ani); };
            Action1<Cat> cat1 = act1;
        }        
    }
}

Looking at Listing 1, you will notices that there is a second example included in it. This second example illustrates contravariance. Most developers, including myself, find contravariance more difficult to understand than covariance. It is, however, a similar concept.

The second example starts out by including a delegate declaration. This time the declaration defines a method that takes a parameter, rather than returning a value:

delegate void Action1<in T>(T a);

Notice also that we use the keyword in, rather than the keyword out. That is because we are passing a parameter in, rather than returning a result “back out” of a function.

Next we define a delegate that takes an animal as a parameter:

Action1<Animal> act1 = (ani) => { Console.WriteLine(ani); };

In the contravariant example shown in Listing 1, the assignment of act1 to the delegate cat1 would fail if we did not include the keyword in when declaring the Action1 delegate type. Here is the assignment:

Action1<Cat> cat1 = act1;

This would fail in C# 3.0 no matter what you did. It will work in C# 4.0 so long as you use the keyword in when declaring the delegate type Action1. If you omitted the keyword, then it would fail. For instance, the following declaration of Action1 would cause the assigned to fail because the keyword in is omitted:

delegate void Action1<T>(T a);

Contrast this declaration with the one in Listing 1.

In the contravariant example we use a lambda to define a method that is equivalent to a standard method that looks like this:

public static void Contra(Animal ani)
{
Console.WriteLine(ani);
}

I’m using lambdas in Listing 1 simply to keep the code short, not because lambdas are connected to covariance and contravariance. This code would still illustrate covariance and contravariance whether or not I used lambdas. What is important here is not the presence of lambdas, but the presence of a generic delegate.

Finally, lets take one moment to make sure you understand why the assignment of act1 to cat1 works. The peculiar thing about contravariance is that it appears that we are assigning a larger type to a smaller type. In other words, it appears that we are trying to fit a big thing like an Animal, into a small type like a Cat. This is not what we are doing. The point here is that act1 will work with any animal, and a Cat is animal, therefore you can safely make the assignment. In other words, cat1 can only be passed Cats, and all cats are animals, therefore anything you can assign to cat1 could also be safely passed to act1. This is why the assignment should work, and does in fact work if you use the keyword in.

As I say, contravariance is a bit confusing. If you find the subject a bit much, then you are joining a happy throng of other developers who struggle with the subject. My suggestion is just to relax, think about the example I’ve shown here for a bit, and you will probably have an “ah-ha” moment either in the next few moments, or sometime in the hopefully very lengthy remainder of your life.

Almost everyone who has written or talked about this subject, from Eric Lippert to Anders himself, points out that this technology is not so much an innovation as a means of bringing C# in line with developers expectations. Adding support for covariance and contravariance to generic interfaces and delegates simply ensures that the language behaves as many would intuitively expect it to behave. All you have to do is remember to add the the keywords in or out when you are making an assignment that involves delegates or interfaces, and the language isn’t behaving as you would expect it to behave.

I apologize for focusing only on delegates in this post. Hopefully I’ll find the time to come back and illustrate the same subject using interfaces. I should add that all the code shown here is written against a very shaky pre-beta version of the C# 4.0, and it is always possible, though that not necessarily likely, that this technology will change before it ships.

LINQ Farm: Preserving Formatting with LINQ to XML

Charlie Calvert — Tue, 30 Sep 2008 09:02:10 GMT

In a previous post, you saw how to work with line numbers when using LINQ to XML to read a file. This post continues in the same vein, but this time the focus is on how to:

Read in an XML file with an arbitrary format, and then write it back out to disk in exactly the same format.
Read in an XML file with an arbitrary format, and write it back out with standard formatting

Some scenarios where you might need this functionality include reading a document from one location and writing it to another location, or reading in a document, editing it, and writing it back out with new data, but the same format.

You can download the source for the code shown in this post.

How to Preserve Formatting

Consider the following block of XML:

<?xml version="1.0" encoding="utf-8" ?>
<alpha><beta> 
 sam 
</beta></alpha>

This document has an extremely idiosyncratic layout. Tags are nested on the same line and a text field stands on its own surrounded by linefeeds. If you wish to read in and write this document back out with the formatting preserved, here is how to proceed:

XDocument x = XDocument.Load(readFileName, LoadOptions.PreserveWhitespace);
x.Save(writeFileName, SaveOptions.DisableFormatting);

When calling XDocument.Load, pass in the LoadOptions.PreserveWhitespace flag from the following enumeration:

public enum LoadOptions
{
  None = 0,
  PreserveWhitespace = 1,
  SetBaseUri = 2,
  SetLineInfo = 4
}

When writing the document back out to disk, use SaveOptions.DisableFormatting, from the following enumeration:

public enum SaveOptions
{
  None = 0,
  DisableFormatting = 1
}

If you take these two steps, then the XML you write to disk will have the same formatting as the document you read in.

Modifying Oddly Formatted XML

If you read in XML that has an odd format, it is likely that linefeeds play a role in that formatting. Suppose we want to modify the XML document shown previously so that it looks like this:

<?xml version="1.0" encoding="utf-8" ?>
<alpha><beta>
  Sue  
</beta></alpha>

The value sam from the original document has been replaced with the word Sue. To make this substitution properly, you need to take care to preserve the linefeeds in the original document. In this example I show a relatively mindless way to preserve linefeeds. For good measure, I also add in code for working with line numbers.

XDocument x2 = XDocument.Load(readFileName,
    LoadOptions.PreserveWhitespace | LoadOptions.SetLineInfo);

XText value = (from c in x2.Elements().DescendantNodes().OfType<XText>()
               where c.Value.Trim().Length > 0
               select c).Single();

IXmlLineInfo x = (IXmlLineInfo)value;
Console.WriteLine("Line Number: {0}", x.LineNumber);
value.Value = Environment.NewLine + "  Sue  " + Environment.NewLine;

x2.Save(createFileName, SaveOptions.DisableFormatting);

This code reads in the original XML file with the PreserverWhitespace flag. I've also OR'd in the LoadOptions.SetLineInfo flag. I do this not because I have any real need to do so, but simply so you can see how to pass in two flags to the Load method.

The next block of code queries the document for all the text nodes that consist of something more than pure white space. I then output the line number of the node that was found. Finally, I modify the node, replacing the original text with the word Sue:

value.Value = Environment.NewLine + " Sue " + Environment.NewLine;

To modify the node, I simply replace the Value property with new text. Note that the code explicitly adds in a pair of linefeeds to preserves the original, idiosyncratic formatting. If you are confused about what is happening here, look up again at the original document. It begins with two tags:

<alpha><beta>

Then there is a linefeed, the word sam, and another linefeed and finally the closing tags:

</beta></alpha>

The code I've written simply preserves those linefeeds. I'm doing this not because I think it is a good idea to include white space like this in a document, but simply to show you that it is possible to preserve it if you have a need or desire to do so.

After modifying the document, the XML is written back to disk using the same technique explained in the previous section.

Cleaning Up XML

If you discover an XML file that has idiosyncratic formatting that you want to remove, LINQ to XML makes it fairly easy to clean up such a document. By default, LINQ to XML will write out a document with proper formatting. All you have to do is use the Load and Save methods without any of the LoadOptions or SaveOption flags shown in the previous two sections:

XDocument x2 = XDocument.Load(readFileName);
x2.Save(createFileName);

This code will not, however, remove arbitrary white space from a text node. For instance, the text node described above that begins and ends with a linefeed will still have linefeeds when you write the document back out. This may be exactly what you want. On the other hand, you may feel it leaves your document in a pretty ugly state. Here is how to clean up the problem

XDocument x2 = XDocument.Load(readFileName);

var query = from c in x2.Elements().DescendantNodes().OfType<XText>()
            select c;

foreach (var item in query)
{
    item.Value = item.Value.Trim();
}

x2.Save(createFileName);

This code finds all the text nodes in the document and strips away their white space with standard string Trim method. When you are done, you can write the document back out to disk. It will now have the clean formatting you expect:

<?xml version="1.0" encoding="utf-8"?>
<alpha>
  <beta>sam</beta>
</alpha>

Finally, if you want to strip away all white space, and end up with a document that sits on a single line, you can write code that looks just like the listing shown earlier in this section, except that the the call to Save the document should look like this:

x2.Save(createFileName, SaveOptions.DisableFormatting);

The output you create will appear all on one line, like this:

<?xml version="1.0" encoding="utf-8"?><alpha><beta>sam</beta></alpha>

If you did not clean up the text nodes as shown above, then your code would end up with linefeeds in it from the text nodes. The result, in our case, would look like this:

<?xml version="1.0" encoding="utf-8"?><alpha><beta>
  sam  
</beta></alpha>

The point here is that DisableFormatting turns off LINQ to XML’s attempts to format your code properly. If you read in strangely formatted code, then using this flag will preserve that idiosyncratic formatting. If you read in code with no attempt to preserve the original formatting, as we do in this section, then writing to disk with this option will end up stripping away all formatting, and leaving you with code all on one line except for white space in a text node.

Working with Text Files

In this post I've focused on reading in and writing XML files using the LINQ to XML API. It's perhaps useful to recall that you can also use standard .NET IO in order to perform similar tasks. This can be particularly helpful when you are testing your code, and want to be sure that your LINQ to XML routines are behaving as expected.

Here is a simple routine for reading in a XML file, or any other text file:

private static void ReadAsText(string fileName)
{
    string s = File.ReadAllText(fileName);
    Console.WriteLine(s);
}

And here is the C# call for writing a file to disk:

File.WriteAllText

If you download the sample code associated with this post, you will see that I use these routines to confirm that my code is working correctly.

Summary

In this post you've seen how to preserve white space in an XML document, and how to strip it out and return to standard formatting. You also had a chance to learn how to edit an idiosyncratically structured XML document without changing its format.

Download the source.

LINQ Farm: LINQ to XML and Line Numbers

Charlie Calvert — Fri, 26 Sep 2008 23:13:01 GMT

There are times when it is useful to know the line number of a node in an XML file. This information can be a helpful to users, particularly if you want to report an error. It can also be convenient to search for a node by line number, but that can, of course, be a very risky endeavor, as documents can be modified accidentally, and their line numbers changed without notice.

This post shows a few fundamentals about working with line numbers in a LINQ to XML program. The code shown in this post is taken from a project called XmlLineNumber. You can download this program from the LINQ Farm on Code Gallery.

Reporting a Line Number

Let’s begin our exploration by detailing a technique for reporting the number of a node that you have found in an XML file. To get started we need to use code from a class called XObject. As shown in Figure 1, XObject sits at the top of the LINQ to XML class hierarchy.

Figure 1: The core objects in the LINQ to XML class hierarchy

XObject implements an interface called IXmlLineInfo:

public interfaceIXmlLineInfo
{
    int LineNumber { get; }
    int LinePosition { get; }
    bool HasLineInfo();
}

The eponymous LineNumber property of this interface is able to store the information we want. To enlist it in our service we need only call XDocument.Load with LoadOptions.SetLineInfo:

XDocument xml = XDocument.Load(fileName, LoadOptions.SetLineInfo);

If you load this XML file into memory using SetLineInfo from the LoadOptions enumeration, then line numbers will be associated with the nodes in your document. The file we are loading is called FirstFourPlanets.xml. It’s a sweet little file that looks like this:

<?xmlversion="1.0" encoding="utf-8"?>
<Planets>
<Planet>
    <Name>Mercury</Name>
    <Moons/>
</Planet>
<Planet>
    <Name>Venus</Name>
    <Moons/>
</Planet>
<Planet>
    <Name>Earth</Name>
    <Moons> <Moon>
        <Name>Moon</Name>
        <OrbitalPeriod UnitsOfMeasure="days">27.321582</OrbitalPeriod>
      </Moon>
    </Moons>
</Planet>
<Planet>
    <Name>Mars</Name>
    <Moons>
      <Moon>
        <Name>Phobos</Name>
        <OrbitalPeriod UnitsOfMeasure="days">0.318</OrbitalPeriod>
      </Moon>
      <Moon>
        <Name>Deimos</Name>
        <OrbitalPeriod UnitsOfMeasure="days">1.26244</OrbitalPeriod>
      </Moon>
    </Moons>
</Planet>
</Planets>

Here is code that uses the IXmlLineInfo interface to report the line number of a node discovered through a standard LINQ to XML search:

XText phobos = (from x in xml.DescendantNodes().OfType<XText>()
                where x.Value == "Phobos"
                select x).Single();

var lineInfo = (IXmlLineInfo)phobos;
Console.WriteLine("{0} appears on line {1}", phobos, lineInfo.LineNumber);

This code looks through all the descendants of the root node for nodes of type XText which are equal to the word Phobos. It uses the LINQ query operator Single to ensure that the query returns only a single node. If the query returned more than one result, the call to Single would raise an exception, which in this case is the behavior we want. The program then casts the result as an instance of IXmlLineInfo, and reports the line number to the user:

Phobos appears on line 24

Searching by Line Number

Let's now turn things around and show how to search through an XML file and look for a node by line number. If you glance at the FirstFourPlanets.xml file, you will see that line 21 looks like this:

<Name>Mars</Name>

Here is code from the XmlLineNumbers sample showing how to search for that node by line number:

XDocument xml = XDocument.Load(fileName, LoadOptions.SetLineInfo);

var line = from x in xml.Descendants()
           let lineInfo = (IXmlLineInfo)x
           where lineInfo.LineNumber == 21
           select x;

foreach (var item in line)
{
    Console.WriteLine(item);
}

Note that the first line uses LoadOptions.SetLineInfo to ensures that line information is recorded when the document is loaded into memory.

The LINQ query shown here uses Descendants to iterate over the elements in the FirstFourPlanets.xml file. The where filter in the query checks to see if any of those elements has its line number set to 21. It happens that the 15^th element returned by the call to Descendants fits that search criteria, and so that node, and that node alone, is found when we foreach over the results.

Notice the cast to convert the XElement nodes returned by the call to Descendants:

let lineInfo = (IXmlLineInfo)x

This cast is necessary, since the actual fields of the IXmlLineInfo interface are not exposed by XElement.

Once again, I want to stress that reporting the line number of a node seems like a reasonable thing to do, but searching for an element by line number is usually not a good idea in production code. For unexplained reasons, code that was on line 532 has a way of migrating to line 533 when you least expect it. In any case, you now know enough to begin working with line numbers in a LINQ to XML program.

Download the source.

LINQ Farm: More on the LINQ Aggregate Operators

Charlie Calvert — Fri, 25 Jul 2008 04:00:17 GMT

The LINQ aggregate operators allow you to perform simple math operations over the elements in a sequence. This post is designed to walk you through those operators, and give you an overview of how to use them. Table 1 shows a list of the 7 aggregate operators.

Note: All the samples shown in this post are found in the AggregateOperators program found on Code Gallery LINQ Farm.

Table 1: LINQ includes 7 aggregate operators designed to help you perform simple math operations. The definitions shown in this table are over-simplifications that give you a general sense of what you can do with a particular operator.

Count	Count the elements in a sequence.
LongCount	Count the elements in a very, very long sequence.
Sum	Add the elements in a sequence
Min	Find the smallest element in a sequence
Max	Find the largest element in a sequence
Average	Find the average value in a sequence
Aggregate	Perform various binary operations on the elements in a sequence.

Except for the Aggregate operator itself, all of these operators have a simple, obvious default use. Several of these operators, do, however, have overloads that need a few sentences of explanation. I will show you one simple example of using the default behavior for the operators, and then dive a bit deeper with a second example that shows how to use at least one of the overloads.

The Count and LongCount Operators

The Count and LongCount operators return the number of elements in a sequence. The Count operator can find this number quickly by simply asking objects such as List<T> that support the ICollection<T> interface for the count. If that service is not available, then LINQ iterates over the items in a list to get the count. The LongCount operator provides the same basic functionality, but allows you to work with an Int64. A simple example of using the Count operator is shown in Listing 1.

Listing 1: A simple example of using the Count and LongCount operators.

public void ShowCount()
{
    var list = Enumerable.Range(5, 12);
    Console.WriteLine(list.Count());
}

The overloads for Count and LongCount allow you to pass in a lambda expression that performs custom calculations from which LINQ can derive the count for a sequence. For instance, you can write code that returns the number of even numbers in a collection:

var list = Enumerable.Range(1, 25);

Console.WriteLine("Total Count: {0}, Count the even numbers: {1}",
    list.Count(), list.Count(n => n % 2 == 0));

Our list consists of the numbers between 1 and 25. We call count once with the first version of the Count operator and get back the number 25.

The second overload of the Count operator takes a simple predicate. The declaration looks like this:

public static int Count<TSource>(this IEnumerable<TSource> source,
   Func<TSource, bool> predicate);

The predicate takes an integer and returns a bool specifying whether or not a particular value from the list passes a test. In our case, we simple ask whether or not the number is even. This computation will return the values 2, 4, 6 and so on up to 24, for a total of 12 elements.

The Min and Max Operators

The Min and Max operators are equally simple. Listings 2 and 3 show how it works. The first shows the behavior of the first overload of Min and Max, the second shows how to one of the other overloads to pose slightly more complex questions.

Listing 2: A simple example of using the Min and Max operators to determine and highest and lowest values in a sequence.

public void ShowMinMax()
{
    var list = Enumerable.Range(6, 10);
      Console.WriteLine("Min: {0}, Max: {1}", list.Min(), list.Max());
}

Our list consists of the number 6 through 15, so the code writes out the values 6 and 15 to the console.The C# source that implements Min and Max use the IComparable<T> or IComparable interfaces to perform the calculations. If you pass in a null argument you will get an ArgumentNullException.

For the more complex examples, I'm going to need a few rows of simple data, which I provide in Listing 3.

Listing 3: The following Item class and the GetItems method are used by most of the examples in this section of the text.

class Item
{
    public int Width { get; set; }
    public int Length { get; set; }

    public override string ToString()
    {
        return string.Format("Width: {0}, Length: {1}", Width, Length);
    }
}

private List<Item> GetItems()
{
    return new List<Item> 
    { 
       new Item { Length = 0, Width = 5 },
       new Item { Length = 1, Width = 6 },
       new Item { Length = 2, Width = 7 },
       new Item { Length = 3, Width = 8 },
       new Item { Length = 4, Width = 9 }
    };
}

There is no simple way to know maximum or minimum values from a list of Items. To find the largest Item do you choose the element with the greatest Length, the greatest Width, or some other value? To solve this problem the C# teams provided us with an overload of the Min and Max operators that take a delegate that we can use to select the proper value for the comparison:

public static int Max<TSource>(this IEnumerable<TSource> source, Func<TSource, int> selector);

Like nearly all the LINQ to Objects operators, Max is implemented as an extension method for the class IEnumerable<T>. It takes an extremely simple lambda that is passed an element from the enumeration and returns an integer. To see how this works, take a look at Listing 4.

Listing 4: A somewhat more complex use of Min and Max, demonstrating how to get minimum and maximum values for complex types with multiple fields.

List<Item> items = GetItems();
ShowList(items);
Console.WriteLine("MinLength: {0}, MaxLength: {1}", 
   items.Min(l => l.Length ), items.Max(l => l.Length));

As you can see, Min and Max both take a very simple delegate, which is implemented here as a lambda.

The lambda that is passed to Min looks like this: l => l.Length. This is lambda is so simple that it can be a bit confusing to people who are new to LINQ. Let's take one moment to be sure we understand what is happening.

We know that this LINQ operator must iterate over the sequence passed in to it, and we can assume that it passes each item it finds to the selector delegate. It then tests the result returned from selector, to see if it is the largest value returned. Without peeking at the real source code, it seems that Max might do something like the code in listing 5.

Listing 5: This method, which I created, mimics what occurs in the real Max method that ships with the C# 3.0 release.

public static class MyExtensions
{
    public static int Max<TSource>(this IEnumerable<TSource> source
      Func<TSource, int> selector)
    {
        int largest = int.MinValue;
        foreach (var item in source)
        {
            int nextItem = selector(item);
            if (nextItem > largest)
            {
                largest = nextItem;
            }
        }
        return largest;
    }

Assuming that we are working with a collection of Items, then selector, were it implemented as a standard method, would have to look something like this:

public int selector(Item item)
{
    return item.Length;
}

This method is semantically identical to the delegate we used in listing x: l => l.Length. It is very simple code that tells us which part of the Item class we are going to use to determine our max value.

It's all so simple that one feels a little like a character in Edgar Allan Poe's "The Purloined Letter:" the answer was hidden in plain sight. Once again we see that the biggest impediment to learning LINQ is the fear that it might be complicated. In practice, it is almost startlingly simple.

The Average Operator

Once one understands the pattern shown in our examination of the Min and Max operators, we find that it can be easily applied to most of the other Aggregate operators. Let’s look at the Average operator, which returns the average value from an enumeration.

For instance, one can find the average for a range of numbers like this:

var list = Enumerable.Range(0, 5);
  Console.WriteLine("Average: {0}", list.Average());

When run, this code tells us that the average of the numbers 0, 1, 2, 3, 4 is the value 2.

When working with a collection of Items, we face the same problem we had with Min and Max: How does one discover the average value for list of Items that define two properties called Length and Width? The answer, of course, is that proceed just as we did with Min and Max operators:

List<Item> items = GetItems();

double averageLength = items.Average(l => l.Length);
double averageWidth = items.Average(w => w.Width);
double averageValue = items.Average(v => v.Length + v.Width);
Console.WriteLine("AverageLength: {0}, AverageWidth: {1} AverageValue: {2}", averageLength, averageWidth, averageValue);

Again, we pass in very simple lambdas such as l => l.Length + l.Width or w => w.Width. Somewhere in the background code similar to what you see in the custom implementation for the Max operator found in listing 5.X. The code must iterate over the list, passing in each item to our lambda, which defines the value we want the Average operator to use in its calculations:

AverageLength: 2, AverageWidth: 7 AverageValue: 9

The Sum Operator

The Sum operator tallies the values in an enumeration. Consider the following simple example:

var list = Enumerable.Range(5, 3);
Console.WriteLine("List sum = {0}", list.Sum());

Our list consists of the numbers 5, 6 and 7. The Sum operator adds them together, producing the value 18.

working with a list of Items, the Sum operator faces the same problem we saw with the Min, Max and Average operators. It should come as no surprise that the solution is nearly identical:

var items = GetItems(); 
Console.WriteLine("Sum the lengths of the items: {0}", items.Sum(l => l.Length));

Here is the same pattern you saw with the Average, Min and Max operators: we pass in a simple lambda to help the Sum method know which part of an Item it should use as the operand when performing its simple addition. The result printed to the console is the value 10. If only the rest of our lives were quite this simple!

The Aggregate Operator

The Aggregate operator follows in the footsteps of the Sum operator, but it provides us with a few more options. Rather than taking a simple delegate like the other operators in this series, it asks for one similar to the lambda we worked with in a previous post:

public static T Aggregate<T>(this IEnumerable<T> source, Func<T, T, T> func);

We know what do to with delegates that looks like this. We could, for instance, create one that adds up a range of numbers:

var list = Enumerable.Range(5, 3);
Console.WriteLine("Aggregation: {0}", list.Aggregate((a, b) => (a + b)));

The aggregate operator gets passed the numbers 5, 6 and 7. The first time the lambda is called it gets passed 5 and 6, and adds them together to produce 11. The next time it is called it is passed the accumulated result of the previous calculation plus the next number in the series: (11 + 7) which yields 18. This is the same result we saw for the Sum operator in the previous section. This overload of the Aggregate operator is indeed very similar to the Sum operator, though it is more flexible, in that you can easily perform multiplication, division, subtraction and other operations instead of simple addition. For instance, this code performs multiplication, yielding the value 210:

list.Aggregate((a, b) => (a * b))

Before pushing on, I should backtrack a little and discuss two simple points that are often brought up when people talk about this first version of the Aggregate operator. If it is passed a list with one item, it returns that item. If it is passed a list with 0 items, it throws an InvalidOperationException.

A second overload of the Aggregate operator allows you to seed the process with an accumulator:

public static TAccumulate Aggregate<TSource, TAccumulate>(
    this IEnumerable<TSource> source, TAccumulate seed, 
    Func<TAccumulate, TSource, TAccumulate> func);

This is essentially the same operator as shown in the previous example, but now you can decide the starting point for the value that will be accumulated:

Console.WriteLine("Aggregation: {0}", list.Aggregate(0, (a, b) => (a + b)));

If we pass in a list with one item in it, say the number five, then the first time the lambda is called it would be passed the seed plus the sole item in the list:

(0 + 5)

The result, of course, is the number 5.

Suppose we pass in an accumulator of 0 plus the numbers 5, 6, 7.

var list = Enumerable.Range(5, 3);
Console.WriteLine("Aggregation: {0}", list.Aggregate(0, (a, b) => (a + b)));

In this case we would step through the following sequence:

0 + 5 = 5
5 + 6 = 11
11 + 7 = 18.

Again, we are doing essentially what we did with the Sum operator.

If you pass in a different seed, then you get a different result:

Console.WriteLine("Aggregation: {0}", list.Aggregate(3, (a, b) => (a + b)));

With a seed of 3, we get:

3 + 5 = 8
8 + 6 = 14
14 + 7 = 21

As mentioned earlier, the Aggregate operator allows us to perform not just addition, but multiplication, division or various other binary mathematical operations:

Console.WriteLine("Aggregation: {0}", list.Aggregate(1, (a, b) => (a * b)));

In this case the series looks like this:

1 * 5 = 5
5 * 6 = 30
30 * 7 = 210

Note that I passed in an accumulator equal to 1, so that we did not end up with the following series of operations:

0 * 5 = 0
0 * 6 = 0
0 * 7 = 0

In what I sometimes suspect might have been an excess of good spirits, the team added one final overload to the Aggregate operator:

public static TResult Aggregate<TSource, TAccumulate, TResult>(
                this IEnumerable<TSource> source, TAccumulate seed,
                Func<TAccumulate, TSource, TAccumulate> func,
                Func<TAccumulate, TResult> resultSelector);

This overload is nearly identical to the previous overload, but you are given one more, very simple, delegate that you can use to transform the result of your aggregation. For instance, consider this use of the Aggregate operator:

Console.WriteLine("Aggregation: {0}", list.Aggregate(0, (a, b) => (a + b),
    (a) => (string.Format("{0:C}", a))));

Please notice that the first two-thirds of this call mirror what we did earlier, and only the third parameter is new.

Suppose we pass in a sequence with the values 5, 6 and 7. As we've already seen, the process will begin by performing the following series of operations:

0 + 5 = 5
5 + 6 = 11
11 + 7 = 18

Once we have our result of 18, this number is passed to the last lambda in our call. It uses the string's Format method to transform it into a string in currency format:

$18.00

Like nearly everything in LINQ, this seems terribly complicated at first only to end up being reasonably simple. It is these kinds of simple operations, however, which provide us with the building blocks out of which we can safely create complex programs. This is what we mean when we apply the word elegant to a technology.

Note: All the samples shown in this post are found in the AggregateOperators program found on Code Gallery LINQ Farm.

LINQ Farm: More on Set Operators

Charlie Calvert — Sat, 12 Jul 2008 10:47:45 GMT

This is a second post on the LINQ Set operators, the first being published while LINQ was still in beta. As mentioned in the previous post, there are four LINQ set operators: Union, Intersect, Distinct and Except. Like the other 49 LINQ operators, these methods are designed to allow you to query data which supports the IEnumerable<T> interface. Since all LINQ query expressions, and most LINQ queries, return IEnumerable<T>, these operators are designed to allow you to perform set operations on the results of a LINQ query.

In this post I give four highly simplified examples of how to use each of the operators, and then end with a more complex example that shows how the operators might be used in a real world setting.

Download the source code.

Union

The Union operator shows the unique items from two lists, as shown in listing 1.

Listing 1: The Show Union method displays the number 1, 2, 3, 4, 5 and 6.

public void ShowUnion()
{
    var listA = Enumerable.Range(1, 3);
    var listB = new List<int> { 3, 4, 5, 6 };

    var listC = listA.Union(listB);

    foreach (var item in listC)
    {
        Console.WriteLine(item);
    }
}

Here two collections are joined together, but only the unique members of each list are retained.

Intersect

The Intersect operator shows the items that two lists have in common.

Listing 2: The ShowIntersect method displays the numbers 3 and 4

public void ShowIntersect()
{
    var listA = Enumerable.Range(1, 4);
    var listB = new List<int> { 3, 4, 5, 6 };

    var listC = listA.Intersect(listB);

    foreach (var item in listC)
    {
        Console.WriteLine(item);
    }
}

Here to collections are joined together, and only the unique, shared members of each list are retained.

Distinct

The Distinct operator finds all the unique items in a list.

Listing 3: The ShowDistinct method displays the number 1, 2 and 3.

public void ShowDistinct() 
{
    var listA = new List<int> { 1, 2, 3, 3, 2, 1 };
    var listB = listA.Distinct();

    foreach (var item in listB)
    {
        Console.WriteLine(item);
    }
}

Except

The Except operator shows all the items in one list minus the items in a second list.

Listing 4: The ShowExcept method prints out the numbers 1, 2, 5, and 6

public void ShowExcept() 
{
    var listA = Enumerable.Range(1, 6);
    var listB = new List<int> { 3, 4 };

    var listC = listA.Except(listB);

    foreach (var item in listC)
    {
        Console.WriteLine(item);
    }
}

In the Context of LINQ

The type of code listed above is useful, but it might be helpful to see these same operators used in the context of a LINQ query expression. You can then see how they can be used to analyze the results of queries to better understand the data that is returned.

You probably know that there are two similar collections used to create lists. One is the generic List<T> collection and the other is the old-style collection called ArrayList. We can use set operators to help us better understand the difference between these two classes.

Here are two queries retrieving the methods from the List<int> class and the ArrayList class:

var queryList = from m in typeof(List<int>).GetMethods()
                where m.DeclaringType == typeof(List<int>)
                group m by m.Name into g
                select g.Key;

var queryArray = from m in typeof(ArrayList).GetMethods()
                 where m.DeclaringType == typeof(ArrayList)
                 group m by m.Name into g
                 select g.Key;

Here is code to retrieve the interesection of these two lists:

var listIntersect = queryList.Intersect(queryArray);

And here is code that displays the resulting sequence:

Console.WriteLine("Count: {0}", listIntersect.Count());

foreach (var item in listIntersect)
{
    Console.WriteLine(item);
}

Alternatively, you could write the query like this:

var queryList = (from m in typeof(List<int>).GetMethods()
                 where m.DeclaringType == typeof(List<int>)
                 group m by m.Name into g
                 select g.Key).Intersect(from m in typeof(ArrayList).GetMethods()
                                         where m.DeclaringType == typeof(ArrayList)
                                         group m by m.Name into g
                                         select g.Key);

In either case, the following list would be displayed:

get_Capacity
set_Capacity
get_Count
get_Item
set_Item
Add
AddRange
BinarySearch
Clear
Contains
CopyTo
GetEnumerator
GetRange
IndexOf
Insert
InsertRange
LastIndexOf
Remove
RemoveAt
RemoveRange
Reverse
Sort
ToArray

And here is how to see the items that the generic lists supports that are not part of the old style collection:

var listDifference = queryList.Except(listIntersect);

And here is the result of this query:

ConvertAll
AsReadOnly
Exists
Find
FindAll
FindIndex
FindLast
FindLastIndex
ForEach
RemoveAll
TrimExcess
TrueForAll

Now you have a list of the methods the two classes share in common, and a list showing what the new generic class has that is not part of the older collection. The LINQ set operators made it easy for you to discover this information.

Download the source code.

LINQ Farm: Lambdas

Charlie Calvert — Sat, 28 Jun 2008 11:35:55 GMT

Lambdas are a simple technology with an intimidating name. They sound like they are going to be difficult to understand, but in practice prove to be relatively trivial.

LINQ has an almost inordinate need for its users to declare a large number of small, simple delegates. The architects of C# decided that forcing the users of LINQ to declare delegates using standard C# 2.0 delegate syntax was an overly verbose option. They wanted to find a shorter, more concise way to accomplish the same task.

The syntax they settled on looks like this:

Func<int, int, int> myLambda = (a, b) => (a + b);

This is a shorthand way of writing code which is semantically equivalent to the following:

public static int Add(int a, int b)
{
    return a + b;
}

Func<int, int, int> myDelegate = Add;

In the next few paragraphs I will compare these two ways of creating a delegate instance, and explain how they map back and forth.

It is obvious that the left hand side of the following two code fragments have the same meaning:

Func<int, int, int> myLambda = (a, b) => (a + b);
Func<int, int, int> myDelegate = Add;

But how can the right hand side be the same?

It turns out that that the expression on the right hand side of the first statement is a shorthand way of writing a method semantically equivalent to the Add method. Just to be clear, a lambda is not a reference to the Add method, it is second method that does the same thing as the Add method.

Here is the lambda:

(a, b) => (a + b);

And here is the Add method:

public static int Add(int a, int b)
{
    return a + b;
}

Here is the place in the Add method where we define the parameters it will take:

(int a, int b)

Here is the place in the lambda where we define the parameters that it will take:

(a, b)

Here is the place in the Add method where we define what it will return:

return a + b;

Here is the place in the lambda where we define what it will return:

(a + b)

As you can see, a lambda and a method do the same thing: they define a set of parameters and an executable body of code.

The type declarations for a lambda are resolved using a technique very similar to the one we employ for generic methods and generic delegates. To see how this works, look again at the full declaration for the lambda:

Func<int, int, int> myLambda = (a, b) => (a + b);

The generic delegate Func says that the method being implemented takes two integers as parameters, and returns an integer. The compiler takes this information and applies it to the lambda. Behind the scenes it resolves (a, b) to (int a, int b) and defines the function such that the body of the lambda (a + b) returns an integer. Thus we give the compiler enough information to convert our lambda into a method that performs the same action as the Add method.

The => symbol is called a lambda operator and is usually pronounced “goes to.” Thus the lambda above can be read as “a comma b goes to a plus b,” or “a and b goes to a plus b.”

Though you will rarely need to do so, you can explicitly declare the type of parameters to a lambda expression:

Func<int, int, int> f = (int a, int b) => (a + b);

For void functions that do not return a value, just use an empty set of parenthesis:

() => Console.WriteLine();

Lambdas have access to local variables:

public static void UseLocal()
{
    int n;
    Func<int> func = () => { n = 6; return n; };
    n = func();
    Console.WriteLine(n); // Outputs the number 6
}

You might be familiar with anonymous methods from C# 2.0. Semantically, anonymous methods and lambdas are identical, but the lambda syntax is easier to use. As a result, there is probably no reason for you to use anonymous methods in the future. Below is a lambda and anonymous method side by side:

Func<int, int, int> myLambda = (a, b) => (a + b);
Func<int, int, int> myAnonMethod = delegate(int a, int b)
{
    return a + b;
};

Both methods take two integers, add them together, and return the result. Commenting further on anonymous methods at this point would serve no purpose, since lambdas create the same result with less work.

In this post you have had a chance to look at lambdas. Their name is intimidating, and their syntax can be a bit confusing at first. Once you see beneath the facade however, this technology turns out to be relatively easy to master.

LINQ Farm: Extension Methods and Scoping

Charlie Calvert — Sat, 28 Jun 2008 11:17:15 GMT

There are a few scoping rules that you must keep in mind when using extensions methods. Problems with scoping and extensions methods are rare, but when you encounter them they are quite vexing.

An instance method will always be called before an extension method. The runtime looks first for an instance method, if it finds an instance method with the right name and signature, it executes it and never looks for your extension method. The following code illustrates this problem:

using System;

namespace ConsoleApplication1
{
    public class MyClass
    {
        public void DoThis()
        {
            Console.WriteLine("MyClass.DoThis");
        }
    }


    public static class MyExtensions01
    {
        // Can never be called as if it were an instance method of MyClass.
        public static void DoThis(this MyClass myClass)
        {
            Console.WriteLine("MyExtensions01.DoThis");
        }
    }


    class Program
    {
        static void Main(string[] args)
        {
            MyClass myClass = new MyClass();
            myClass.DoThis();                // Calls MyClass.DoThis
            MyExtensions01.DoThis(myClass);  // Calls MyExtensions01.DoThis
        }
    }
}

MyExtensions01.DoThis is a valid extension method for MyClass. However, it will never be called because MyClass.DoThis always takes precedence over it unless you explicitely call it as a static method of MyExtensions01.

In cases where you have two extension methods with the same name, an extension method in the current namespace will win out over one in another namespace. Ambiguity will become an issue, however, when you try to call two extension methods with the same name and signature in the same namespace, or in two different namespaces both used by the current namespace. See Listing 5 for an example of this problem.

The following code will not compile because the compiler finds the call to DoThat ambiguous:

using System;
using System.Collections.Generic;
using System.Linq;

namespace ExtensionScope
{
    public class MyClass
    {
        public void DoThis()
        {
            Console.WriteLine("Do this");
        }
    }
}

namespace Extensions01
{
    using ExtensionScope;

    public static class MyExtensions01
    {
        // Can never be called
        public static void DoThis(this MyClass myClass)
        {
            Console.WriteLine("Do this");
        }

        public static void DoThat(this MyClass myClass)
        {
            Console.WriteLine("Do bop");
        }
    }
}

namespace Extensions02
{
    using ExtensionScope;

    public static class MyExtensions02
    {
        // Can never be called
        public static void DoThis(this MyClass myClass)
        {
            Console.WriteLine("Do this");
        }

        public static void DoThat(this MyClass myClass)
        {
            Console.WriteLine("Do bang");
        }
    }
}

namespace ExtensionScope
{
    using Extensions01;
    using Extensions02;

    class Program
    {
        static void Main(string[] args)
        {
            MyClass m = new MyClass();
            m.DoThat();
        }
    }
}

This program throws a compile time error because the compiler does not know if you want to MyExtionsions01.DoThat() or MyExtension02.DoThat(). There are two ways to resolve this error:

You could remove the using directive for either Extensions01 or Extensions02. In this case, that would be a fine resolution, but if there were other methods or classes in both Extensions01 and Extensions02 that you wanted to use, then this could become a painful, or even unacceptable, choice.
You could explicitly state which method you want to call using standard static syntax: MyExtensions01.DoThat(m).
You could move either MyExtensions02 or MyExtensions01 into the ExtensionScope namespace:

namespace ExtensionScope
{
    public static class MyExtensions02
    {
        public static void DoThat(this MyClass myClass)
        {
            Console.WriteLine("MyExtensions02.DoThat");
        }
    }
}

This latter solution works so long as you have access to the source.

It should be clear that some of the issues discussed here can lead to trouble if you are not careful. In particular, you don't want to end up in a situation where forcing someone to remove a namespace results in them losing access to important functionality, nor do you want to force them to choose between functionality they desire and using your extensions.

It can also be a serious nuisance if you muddy a namespace with what many developers might consider superfluous methods. If you added 50 extension methods to the C# string class, then developers who just want to access the base functionality of that object would always have to contend with your methods, particularly when using IntelliSense.

To avoid or at least mitigate the seriousness of these problems, you should always place your extension methods in unique namespace separated from the rest of your code so that you can easily include or exclude the extension methods from a program. Listings 10 and 11 illustrate this technique.

Place your extensions in a separate file, and give them a unique namespace:

namespace MyCode
{
    public class MyCode
    {
        // Code omitted here
    }
   
}

namespace MyCode.Extensions
{
    public static class SpecialString
    {
        private static string[] stateCodes = 
                {"AL","AK","AZ","AR","CA","CO","CT","DE","FL",
                 "GA","HI","ID","IL","IN","IA","KS","KY","LA",
                 "ME","MD","MA","MI","MN","MS","MO","MT","NE",
                 "NV","NH","NJ","NM","NY","NC","ND","OH","OK",
                 "OR","PA","RI","SC","SD","TN","TX","UT","VT",
                 "VA","WA","WV","WI","WY"};

        public static bool IsState01(this string source)
        {
            if (source == null) return false;
            source = source.ToUpper(); 
            foreach (var item in stateCodes)
            {
                if (source == item)
                {
                    return true;
                }
            }
            return false;
        }

        public static bool IsState02(this string source)
        {
            return (source == null) ? false : stateCodes.Contains(source.ToUpper());
        }
    }
}

In this code I show you two alternative ways to implement the IsState extension method. The second, which uses LINQ, is probably easier to maintain. You can access the extension methods in a namespace MyCode.Extensions like this:

using System;
using MyCode;
using MyCode.Extensions;

namespace ConsoleApplication1
{
    class Program
    {            
        static void Main(string[] args)
        {
            MyCode myCode = new MyCode();
            // Use My Code here.
            string test = "WA";
            if (test.IsState02())
            {
                Console.WriteLine("{0} is a state", test);
            }
        }
    }
}

In this code your extension method is available and the code compiles. Comment out the third using statement and your extension method would not be available and the code would not compile. The developer would, however, still have access to the functionality found in the MyCode namespace. You could perhaps improve this technology by putting your extensions in their own assembly with its own namespace. You could then be sure that developers could choose to include or exclude the extra weight of your extension methods when they ship their code.

Though extension methods are particularly useful in LINQ, they are now a part of the language, and if used with caution, they can be useful. Placing them in their own namespace is a best practice that should help you get the most from this feature.

Charlie Speaking in Southern California

Charlie Calvert — Wed, 30 Apr 2008 09:36:07 GMT

I will be presenting on C# and LINQ three times in Southern California next week at a series of user group meetings. The events will be held in the San Diego and the Los Angeles area. Lisa Feigenbaum will be flying down from Redmond with me, and she will do VB presentations at the same set of meetings.

The schedule is as follows:

May 6, 6:00 PM: San Diego, http://sddotnetdg.org/
May 7, 6:30 PM: Irvine, http://www.ocdotnet.org/
May 8: 6:30 PM: Torrence, http://www.southbaynet.org/southbaynet.org/default.html

I will be talking on C# 3.0 Best Practices and LINQ:

C# 3.0 introduces a number of new language features such as query expressions, lambda expressions, extension methods, automatically implemented properties, local type inference and more. These are all features that can improve the quality of your code. They also provide new opportunities for making mistakes This talk focuses on both the good and the bad: how to use and how not to use the new features of C#. Each feature will be introduced with a small example, and you should be able to follow the talk even if you are not already familiar with the new language constructs. The talk will also explore the theoretical underpinnings of LINQ.

Lisa will be speaking on Visual Basic 2008 IDE Tips and Tricks:

In this talk, we’ll show how to turn yourself into a Visual Studio 2008 guru with the new language and IDE features. Tips and tricks will include how to maximize your VB IntelliSense experience, leverage Refactoring features, and improve the performance of your query and XML code. We’ll explore integrated XML, and show how to navigate XML gotchas and express what you wish in fewer lines of code. With respect to LINQ we’ll go deep into best practices, pitfalls to avoid, and answers to most frequently asked questions

Tuesday Night:

AMN Health Care
12400 High Bluff Drive
San Diego, CA 92130

Wednesday Night

Microsoft
Three Park Plaza, Suite 1800
Irvine, CA 92614

Thursday Night

American Honda Motors
Bldg 100, # 100-1E-13
1919 Torrance Blvd
Torrance, CA 90501

Source and deck from the talk.

LINQFarm: Understanding IEnumerable, Part I

Charlie Calvert — Mon, 28 Apr 2008 22:15:01 GMT

The IEnumerable<T> interface is a key part of LINQ to Objects and binds many of its different features together into a whole. This series of posts explains IEnumerable<T> and the role it plays in LINQ to Objects. If you hear people talking about IEnumerable<T>, and sometimes wished you better understood its significance, then you should find this text helpful.

Collections and IEnumerable<T>

Though LINQ to Objects can be used to query several C# types, it cannot be used against all your in-process data sources. Those that can be queried all support the IEnumerable<T> interface. These include the generic collections found in the System.Collections.Generic namespace. The commonly used types found in this namespace include List<T>, Stack<T>, LinkedList<T>, Queue<T>, Dictionary<TKey, Value> and Hashset<T>.

All of the collections in the System.Collections.Generic namespace support the IEnumerable<T> interface. Here, for instance, is the declaration for List<T>:

public class List<T> : IList<T>, ICollection<T>, IEnumerable<T>, IList, ICollection, IEnumerable

You will find IEnumerable<T> listed for all the other generic collections. It is no coincidence that these collections support IEnumerable<T>. Their implementation of this interface makes it possible to query them using LINQ to Objects.

LINQ to Objects and IEnumerable<T>

Consider the following simple LINQ query:

List<int> list = new List<int> { 1, 3, 2 };

// The LINQ Query expression
var query = from num in list
            where num < 3
            select num;

foreach (var item in query)
{
    Console.WriteLine(item);
}

The type IEnumerable<T> plays two key roles in this code.

The query expression has a data source called list which implements IEnumerable<T>.
The query expression returns an instance of IEnumberable<T>.

Every LINQ to Objects query expression, including the one shown above, will begin with a line of this type:

from x in y

In each case, the data source represented by the variable y must support the IEnumerable<T> interface. As you have already seen, the list of integers shown in this example supports that interface.

The same query shown here could also be written as follows:

IEnumerable<int> query = from num in list
                         where num < 3
                         select num;

This code makes explicit the type of the variable returned by this query. As you can see, it is of type IEnumerable<int>. In practice, you will find that most LINQ to Objects queries return IEnumerable<T>, for some type T. The only exceptions are those that call a LINQ query operator that return a simple type, such as Count():

int number = (from num in list
              where num < 3
              select num).Count();

In this case the query returns an integer specifying the number of items in the list created by this query. LINQ queries that return a simple type like this are an exception to the rule that LINQ to Objects queries operate on class that implement IEnumerable<T> and return an instance that supports IEnumerable<T>.

Composable

The fact that LINQ to Objects queries both take and return IEnumerable<T> enables a key feature of LINQ called composability. Because LINQ queries are composable you can usually pass the result of one LINQ query to another LINQ query. This allows you to compose a series of queries that work together to achieve a single end:

List<int> list = new List<int> { 1, 3, 2 };

var query1 = from num in list
             where num < 3
             select num;

var query2 = from num in query1
             where num > 1
             select num;

var query3 = from num1 in query1
             from num2 in query2
             select num1 + num2;

Here the results of the first query are used as the data source for the second query, and the results of the first two queries are both used as data sources for the third query. If you print out the results of query3 with a foreach loop you get the numbers 3 and 4. Though it is not important to the current subject matter, you might have fun playing with the code to understand why these values are returned.

Summary

By now it should be clear to you that IEnumerable<T> plays a central role in LINQ to Objects. A typical LINQ to Objects query expression not only takes a class that implements IEnumerable<T> as its data source, but it also returns an instance of this same type. The fact that it takes and returns the same type enables a feature called composability.

The next logical question would be to ask why this type plays such a key role in LINQ to Objects. One simple answer would be that the creators of LINQ decided that it should be so, and hence it is so. But one can still ask why they picked this particular type. What is it about IEnumerable<T> that makes it a useful data source and return type for LINQ to Objects queries? The answer to that question will be found in the second part of this series of articles.

Charlie's Deck and Demos from his Essence of LINQ Talk

Charlie Calvert — Tue, 08 Apr 2008 02:49:53 GMT

The sample programs and the slides from my Essence of LINQ talk are available on the Code Gallery LINQ Farm resource page.

LINQ Farm Seed: Using the Expression Tree Visualizer

Charlie Calvert — Wed, 13 Feb 2008 21:54:11 GMT

The Visual Studio 2008 CSharp samples include several valuable tools that LINQ developers can use to help expedite the development process. One of the is the Expression Tree Visualizer. This tool works in both Visual Studio Express and the other versions of Visual Studio that support C# development.

Note: If you are unfamiliar with expression trees, then you might want to view this post.

You can access the Expression Tree Visualizer by choosing Help | Samples from the Visual Studio 2008 menu system. From there you will be able to access links to the updated online version of the samples.

After you unzip the samples you should navigate to the LinqSamples directory, open the ExpressionTreeVisualizer project, and build it. By default, F6 is the build key in Visual Studio 2008.

After the build process is complete a library called ExpressionTreeVisualizer.dll will be created in the following directory:

...\LinqSamples\ExpressionTreeVisualizer\ExpressionTreeVisualizer\bin\Debug

Copy this DLL to your Visualizers directory. If the directory does not exist, you can create it inside the default user directory for Visual Studio. That directory would typically be located here:

...\Documents\Visual Studio 2008\Visualizers

One Windows XP system, the path might look like this:

...\My Documents\Visual Studio 2008\Visualizers

The Code Snippets, Projects and Templates directories also appear in this same location. You can also find the location of this directory by selecting Tools | Options | Projects and Solutions | General from the Visual Studio 2008 menu. You may need to restart Visual Studio after copying the ExpressionTreeVisualizer library to the Visualizers directory.

You can now test the visualizer by opening a project that creates an expression tree such as the Expression Tree Basics sample from the Code Gallery. Alternatively, you can create a default console application, add System.Linq.Expressions to the using statements, and add two lines of code to the program body:

using System;
using System.Linq.Expressions;

namespace ConsoleApplication107
{
    class Program
    {
        static void Main(string[] args)
        {
            Expression<Func<int, int, int>> expression = (a, b) => a + b;

            Console.WriteLine(expression);
        }
    }
}

If you are using the code shown above, set a breakpoint on the WriteLine statement and run the program. Hover your mouse under the variable expression, as shown in Figure 1. A small Quick Info box with a magnifying glass will appear.

Figure 1: Examining the variable expression at run time in Visual Studio with Quick Info.

If you click on the magnifying class, an option to view the Expression Tree Visualizer will appear, as shown in Figure 2.

Figure 2: By clicking on the magnifying class you can get access to the Expression Tree Visualizer.

If you select the visualizer option then it will appear and you can view the nodes of your expression tree, as shown in Figure 3.

Figure 3: A visualization of the expression tree for a simple lambda expression.

If you are working with a LINQ to SQL query, then can use the same technique to view an expression tree. You will, however, need to take one extra step because the expression tree is a member of the IQueryable variable returned from a typical LINQ to SQL query expression. Consider this simple LINQ to SQL query:

var query = from c in db.Customers
            where c.City == "London"
            select c;

At runtime, you can get Quick Info on the variable query, much as we did on the variable expression earlier in this post. Figure 4 shows how it looks:

Figure 4: Hover the mouse over the variable query, then drill down to view the expression tree which is found in the queryExpression field. (Double click on this image to see a full-sized version.)

When you select the magnifying class, you will be presented with an option to pop up the Expression Tree Visualizer, which will look like the image shown in Figure 3, but with different data.

Summary

This post describes how to use the Expression Tree Visualizer that ships with the Visual Studio 2008 samples.

LINQ Farm Seed 02: Aggregate Operator Part II

Charlie Calvert — Thu, 07 Feb 2008 00:08:00 GMT

LINQ Farm Seeds are short posts designed to be read in a few minutes. In the previous seed we used the first overload of the C# 3.0 LINQ Aggregate operator to sum a list of numbers. In this post we'll work with a list of words.

The code we will look at reverses the following string:

The end is the beginning, the beginning the end

If we pass this string in to our method, it will yield this result:

end the beginning the beginning, the is end The

Here is the method that we will use:

private string ReverseSentence(string sentence)
{
  string[] words = sentence.Split(' ');           

  return words.Aggregate((accumulator, b) => b + " " + accumulator);
}

The method first breaks the sentence down into a list of words:

string[] words = sentence.Split(' ');

It passes this list to the LINQ to Objects Aggregate operator. This overload of that operator takes a single lambda expression as a parameter. The lambda expression takes two parameters (accumulator, b) and returns them in reverse order (b accumulator). If it were a standard method rather than a lambda expression, it would look like this:

private static string Reverse(string accumulator, string b)
{
   return b + " " + accumulator;
}

The method would be called like this:

words.Aggregate(Reverse);

In our code, however, we use lambda expressions rather than standard methods.

As you recall from the previous post, the Aggregate operators passes in items two at a time from a list. The first parameter becomes the accumulator for the aggregation. This is why I chose to give the first parameter in the lambda expression the name accumulator:

return words.Aggregate((accumulator, b) => b + " " + accumulator);

You can watch the aggregation process take place in the debugger. Set a breakpoint on the call to Aggregate, and add the variables accumulator and b to your watch list. Step through the method with the F11 key. You can watch the result "accumulate" in the accumulator variable.

The process begins by passing the first two words into the lambda expression, where the first one is the accumulator:

(The) (end)

The method reverses their order and places a space between them:

end The

The next iteration sends in the result of the previous expression, plus the next word from the list:

(end the) (is)

The lambda expression reverses them:

is end The

The Aggregate operator now processes the result of the previous iteration in the accumulator plus the next word:

(is end The) (the)

Again our lambda expression swaps the parameters and places a space between them:

the is end The

Next time we pass in the following:

(the is end The) (beginning,)

This yields:

beginning, the is end The

The rest of the cycle looks like this:

(beginning, the is end The), (the) >===> the beginning, the is end The
(the beginning, the is end The) (beginning) >===> beginning the beginning, the is end The
(beginning the beginning, the is end The) (the) >===> the beginning the beginning, the is end The
(the beginning the beginning, the is end The) (end) >===> end the beginning the beginning, the is end The

I find this an immensely satisfying process to contemplate simply because it so very logically runs counter to my intuitions. I keep expecting it to do something different, but when I think it through I see exactly why it does what it does. How many real world uses for this I will discover I don't know, but I will treasure each one as I encounter it.

Listing 1 shows the complete code for this simple program. In this version of the ReverseSentence method, you see how to convert the sentence int List<string> rather than an array of string. I use the ToList() LINQ operator to perform this conversion, so you are seeing a sneak peak of an operator I haven't yet covered in this series.

Listing 01: This code shows how to use the Aggregate() and ToList() operators that are declared in the Enumerator class and which are part of LINQ to Objects.

using System;
using System.Collections.Generic;
using System.Linq;

namespace LinqFarmSeed02
{
    class Program
    {
        private static string ReverseString(string sentence)
        {
            List<string> words = new List<string>(sentence.Split(' ').ToList());

            return words.Aggregate((a, b) => b + " " + a);            
        }

        static void Main(string[] args)
        {
            string sentence = "The end is the beginning, the beginning the end";
            Console.WriteLine(sentence);

            string result = ReverseString(sentence);
            Console.WriteLine(result);
        }        
    }
}

In this post you have seen how to use the Aggregate operator to reverse the order of a list of words. Thinking through the way this operator works on a list of strings can help you understand exactly how it is implemented and exactly what it does.

-----------------

Download the source code from the LINQ Farm.

I want to get my hands dirty; show me more LINQ Farm posts.