Stuart Leeks

Options for creating a test project

stuartle — Thu, 20 Aug 2009 20:10:46 GMT

I was rooting through the Tools\Options dialog in Visual Studio recently and noticed the following options for Test projects:

Whilst these are probably reasonable defaults in terms of discoverability of features, I’ve always found that the first thing I did when creating a test project was to delete the introduction file and manual test (and sometimes the unit test, depending on what I’m doing). I’ve now unticked all of the options, so I get an empty project and just add what I need.

It’s a small thing, but I like it!

Setting the window title for a command prompt

stuartle — Mon, 17 Aug 2009 23:26:02 GMT

Very quick tip in what seems to have become a mini-series (Part 1, Part 2). I often find myself with a number of command windows open and switching between them can be problematic. To address this I use the “title” command to set the window title, and have found that setting the title to match the current directory works well. To achieve this quickly you can use the following command

@title %CD%

And, of course, you can wrap this in a batch file and put it in your path to make it easier to remember and use.

Finding files in your path environment

stuartle — Mon, 17 Aug 2009 23:24:16 GMT

Occasionally I’m working at the command prompt and have a utility that is in the path environment, but need to find where the file actually is. So, I wrote a little batch file that does exactly that

@echo off
echo Searching path for %1
if "%~$PATH:1"=="" goto :notfound
echo Found: %~$PATH:1
goto :end

:notfound
echo Unable to find %1
goto :end

:end

The batch file takes a single argument which is the filename to find. The magic happens in the "%~$PATH:1" part – this basically searches the directories in the PATH environment variable for first argument passed to the batch file. All of the rest of the batch file is geared towards producing nice output!

NOTE – use this at your own risk. If it deletes all of your files or replaces the text of your important documents with Lorem Ipsum then you didn’t get it from me!

NOTE 2 – if you want to find out about the other expansion options for environment variables then type “HELP FOR” at the command prompt.

** UPDATE 2009/08/20 **:

It turns out that there’s a better way to do this: the WHERE command – thanks Rupert!

The WHERE command will list all locations that it found a match in the path, also allows wildcards and is generally more flexible. Time to delete that batch file...

I’m not sure when this command appeared. It is listed in the command-line reference for Windows Vista, but not for Windows XP.

PowerShell script to clean and zip a directory

stuartle — Fri, 14 Aug 2009 00:14:37 GMT

As part of my role I’m often sending sample code to customers. Sometimes this is a small snippet inline in an email, but often it will be a zipped up Visual Studio solution. Simply zipping up the folder as-is ends up including bin and obj directories which bloat the zip. Performing a clean in Visual Studio helps, but I ended up writing a PowerShell script to perform the directory clean. It turned out to be reasonably simply to put together:

Get-ChildItem . -recurse -force -include bin,obj,*.tmp,*.user,*.sqlsuo,*.vssscc,*.vspscc,_UpgradeReport_Files | foreach -process {Remove-Item $_.FullName -Force -Recurse}

The command performs a recursive clean starting from the current directory and removes files and directories that I’m generally not interested in including in the zip file. So far this has made a massive difference to the size of zip files that I’ve sent. Although this has proved to be a useful script, I recently spent some time updating it to perform the directory zipping. This turned out to be a little trickier and I ended up pulling snippets together from a number of sources on the internet.

$Directory = Get-Item .
$ParentDirectory = Get-Item ..
$ZipFileName = $ParentDirectory.FullName + "\" + $Directory.Name + ".zip"

if (test-path $ZipFileName) {
echo "Zip file already exists at $ZipFileName"
return
}

set-content $ZipFileName ("PK" + [char]5 + [char]6 + ("$([char]0)" * 18))
(dir $ZipFileName).IsReadOnly = $false

$ZipFile = (new-object -com shell.application).NameSpace($ZipFileName)

$ZipFile.CopyHere($Directory.FullName)

The first few lines of this snippet set up a couple of variables to make life easier later. Then it tests if the zip already exists and bails out if so. Then we get on to the interesting task of creating the zip file using a combination of directly to the file and calling the Shell.Application object.

You can either save these as separate scripts, or combine them into a single script. Note that I’m still trying to convert to PowerShell after far too long writing batch files so I make no claims that this is the best way to write a PowerShell script. In fact I make no claims about this, so use with caution in case it deletes the document you spent all night finishing or wipes your hard drive.

Quickly creating an Outlook rule

stuartle — Thu, 13 Aug 2009 23:35:05 GMT

I’ve had a couple of conversations with people recently where they’ve said that they keep meaning to set up an Outlook rule to move specific messages to a separate folder. On each occasion the need to navigate the Tools\Rules And Alerts window seems to have been enough to put the task off. Unfortunately, they weren’t aware that you can right click on a message and choose Create Rule. This brings up a simpler dialog that contains pre-completed options based on the message you had selected (e.g. messages from this sender, with this subject, …) which is my preferred way of creating rules! Official documentation: http://office.microsoft.com/en-us/outlook/HA100968031033.aspx#4

Adding links containing spaces to an email message

stuartle — Fri, 07 Aug 2009 10:22:00 GMT

Possibly slightly off-topic for this blog, but so far I’m doing quite a good job of not settling down on a single topic ;-)

Outlook has the nice feature that if you type something that looks like a link into an email, it will automatically create the link for you. Now, suppose you have a share on a server: “\\server\Some Share” (without the quotes). If you simply type this in, Outlook is confused by the space and you end up with

Fortunately there is a simply workaround. Unfortunately, very few people seem to be aware of it! All you need to do is enclose the link text in angle brackets. So you’d actually type “<\\server\Some Share>” (without the quotes). Now when Outlook creates the link, it uses the full text between the brackets:

And that’s it! Now start using it, and tell everyone else about it. You can always point them to the official write-up if you’d rather.

ADPlus, Windows 7 and ASP.Net

stuartle — Wed, 22 Jul 2009 18:11:15 GMT

I’ve been working through some crash dumps with ADPlus and WinDBG recently and hit a couple of snags that seem to be related to running ADPlus on Windows 7. John Robbins gives details of the first issue I hit and points out that the output from tlist.exe (which ADPlus uses) had changed. The post also notes that the issue is fixed in later releases of the Debugging Tools (version 6.11.1.402 and later), so I upgraded. With 6.11.1.404 installed, I started to get a different error:

Microsoft (R) Windows Script Host Version 5.8
Copyright (C) Microsoft Corporation. All rights reserved.

Attaching the debugger to: W3WP.EXE_-MyAppPool-_-v_-v2.0-_-l_-webengine4.dll-_-a_-.-pipe-iisipm1417f766-136f-4838-8332-861176da3119_-h_-C-inetpub-temp-apppools-MyAppPool.config-_-w_--_-m_0_-t_20
(Process ID: 3412)
Error creating file: C:\Program Files\Debugging Tools for Windows (x64)\Hang_Mode__Date_07-22-2009__Time_15-32-0505\CDBScripts\PID-3412__W3WP.EXE_-MyAppPool-_-v_-v2.0-_-l_-webengine4.dll-_-a_-.-pipe-iisipm1417f766-136f-4838-8332-861176da3119_-h_-C-inetpub-temp-apppools-MyAppPool.config-_-w_--_-m_0_-t_20.cfg Error: 76 -
Path not found

On a successful run, I’d expect to see

Microsoft (R) Windows Script Host Version 5.8
Copyright (C) Microsoft Corporation. All rights reserved.

Attaching the debugger to: W3WP.EXE_MyAppPool
(Process ID: 3412)

The key difference is the text after “Attaching the debugger to:”. Looking through the adplus script, the name used there is built up from the process name and the application pool name. In order to get the name of the application pool, adplus uses the command line from the tlist output.

On my machine (and also on a colleague’s Windows 7 machine) the command line for w3wp.exe is along the lines of

c:\windows\system32\inetsrv\w3wp.exe -ap "MyAppPool" -v "v2.0" -l "webengine4.dll" -a \\.\pipe\iisipm1417f766-136f-4838-8332-861176da3119 -h"C:\inetpub\temp\apppools\MyAppPool.config" -w "" -m 0 -t 20

On a Windows Server 2008 box that I had access to the command line is similar, but the big difference is that the –ap parameter (which specifies the application pool name) appears at the end of the command line. Looking back at the adplus script, it seems to assume that the –ap parameter will be at the end. I made a small change to my adplus script to only pull back the quoted value (with no regard for how to handle embedded quotes). The point to make the change is after the following code

if g_CurrentProcesses(PCount).Name = "W3WP.EXE" Then
    ' read the next line
    if not objTextFile.AtEndofStream Then
        strAux = objTextFile.ReadLine
        ' check the command line for the -ap parameter
        If InStr(strAux, "-ap") Then
            arrAux = split(strAux, "-ap", -1, 1)
            strPackageName = trim(arrAux(1))

All that is needed is to add the following code immediately after the last line above (NOTE: as always, use this entirely at your own risk!!!)

' modifications start (http://blogs.msdn.com/stuartleeks/archive/2009/07/22/adplus-windows-7-and-asp-net.aspx)
If Mid(strPackageName, 1, 1) = """" Then
    strPackagename = Mid(strPackageName, 2, InStr(2,strPackageName, """") - 2)
End If
' modifications end

With this change in place, I’m back in business with adplus! As mentioned, I got this behaviour on my Windows 7 machine using version 6.11.1.404 of the Debugging Tools for Windows. I make no claims that the change above will work for you, or even that it won’t delete all your files ;-). Hopefully this will be addressed in a future release of the Debugging Tools…

Improving ObjectQuery.Include – Updated

stuartle — Fri, 24 Apr 2009 17:25:00 GMT

Having spent some time using the sample from my previous post on ObjectQuery.Include, I’ve encountered a bug! It turns out that the code generates the wrong include string for

context.Customers.Include(c => c.Order.SubInclude(o=>o.OrderDetail))

The fix for this is a small change to the BuildString method to recurse up the MemberExpression if necessary. The updated code is below - usual disclaimers apply!

    public static class ObjectQueryExtensions
    {
        public static ObjectQuery<TSource> Include<TSource, TPropType>(this ObjectQuery<TSource> source, Expression<Func<TSource, TPropType>> propertySelector)
        {
            string includeString = BuildString(propertySelector);
            return source.Include(includeString);
        }
        private static string BuildString(Expression propertySelector)
        {
            switch (propertySelector.NodeType)
            {
                case ExpressionType.Lambda:
                    LambdaExpression lambdaExpression = (LambdaExpression)propertySelector;
                    return BuildString(lambdaExpression.Body);

                case ExpressionType.Quote:
                    UnaryExpression unaryExpression = (UnaryExpression)propertySelector;
                    return BuildString(unaryExpression.Operand);

                case ExpressionType.MemberAccess:

                    MemberExpression memberExpression = (MemberExpression)propertySelector;
                    MemberInfo propertyInfo = memberExpression.Member;

                    if (memberExpression.Expression is ParameterExpression)
                    {
                        return propertyInfo.Name;
                    }
                    else
                    {
                        // we've got a nested property (e.g. MyType.SomeProperty.SomeNestedProperty)
                        return BuildString(memberExpression.Expression) + "." + propertyInfo.Name;
                    }

                case ExpressionType.Call:
                    MethodCallExpression methodCallExpression = (MethodCallExpression)propertySelector;
                    if (IsSubInclude(methodCallExpression.Method)) // check that it's a SubInclude call
                    {
                        // argument 0 is the expression to which the SubInclude is applied (this could be member access or another SubInclude)
                        // argument 1 is the expression to apply to get the included property
                        // Pass both to BuildString to get the full expression
                        return BuildString(methodCallExpression.Arguments[0]) + "." +
                               BuildString(methodCallExpression.Arguments[1]);
                    }
                    // else drop out and throw
                    break;
            }
            throw new InvalidOperationException("Expression must be a member expression or an SubInclude call: " + propertySelector.ToString());

        }

        private static readonly MethodInfo[] SubIncludeMethods;
        static ObjectQueryExtensions()
        {
            Type type = typeof(ObjectQueryExtensions);
            SubIncludeMethods = type.GetMethods().Where(mi => mi.Name == "SubInclude").ToArray();
        }
        private static bool IsSubInclude(MethodInfo methodInfo)
        {
            if (methodInfo.IsGenericMethod)
            {
                if (!methodInfo.IsGenericMethodDefinition)
                {
                    methodInfo = methodInfo.GetGenericMethodDefinition();
                }
            }
            return SubIncludeMethods.Contains(methodInfo);
        }

        public static TPropType SubInclude<TSource, TPropType>(this EntityCollection<TSource> source, Expression<Func<TSource, TPropType>> propertySelector)
            where TSource : class, IEntityWithRelationships
            where TPropType : class
        {
            throw new InvalidOperationException("This method is only intended for use with ObjectQueryExtensions.Include to generate expressions trees"); // no actually using this - just want the expression!
        }
        public static TPropType SubInclude<TSource, TPropType>(this TSource source, Expression<Func<TSource, TPropType>> propertySelector)
            where TSource : class, IEntityWithRelationships
            where TPropType : class
        {
            throw new InvalidOperationException("This method is only intended for use with ObjectQueryExtensions.Include to generate expressions trees"); // no actually using this - just want the expression!
        }
    }

Cheating at Scrabble with LINQ to Objects

stuartle — Fri, 06 Mar 2009 00:33:04 GMT

I read an interesting post on Eric Lippert’s blog recently where he was using LINQ to Objects to find possible words from a set of letters in Scrabble (he wasn’t actually cheating – that just makes for a more interesting title!). Eric made a great follow-up post that highlights the need for both defining performance goals and for profiling when working to improve performance. I thoroughly recommend reading both Eric’s posts because they are great posts - plus they set the scene for this post ;-). In Eric’s posts, he noted that he’d met his performance goals for the code but having taken a copy of the code and started to play with it, I became the new user. I guess that I can be a little impatient at times and wanted it to feel as though the results were returned instantly! I stuck with Eric’s decision to not simply cache the dictionary in memory – after all, I may want to use a dictionary that won’t fit into memory at some point.

Running the profiler against the code on my machine shows that even before the modifications in Eric’s second post, reading the data from disk is the bottleneck on my machine. This highlights another issue with profiling your application: you need to perform the profiling on representative hardware. In my case I’m running the application on a laptop so it may be that my hard drive is slower. Having said that, the changes from Eric’s post still made a noticeable difference to performance:

Method	Search time(ms)
Original	2700
Updated	1658

Having determined that reading from disk is the culprit (at least on my machine!), I looked through the code again. In the original code, the SearchDictionary method is called once with the original rack and then 26 further times as a result of adding an extra letter to the rack (to allow matching to existing tiles on the board).

First off, I refactored the code slightly and introduced a SearchResult class

class SearchResult
{
    public string Rack { get; set; }
    public IEnumerable<string> Words { get; set; }
}

This allows my Search method to return an IEnumerable<SearchResult> that includes the original rack plus the virtual racks formed by using a letter from the board. My implementation of the SearchDictionary method now handles searching with the additional letters. With this change in place I can now update the search method so that it only hits the file once. Without boring you with the details too much, I could have just added the extra items to the HashSet that Eric added in his second post but I wanted to be able to correlate the matches with the original rack for outputting the results (essentially I constrained myself to producing the same output as the original method). I created a type to correlate the original and expanded/canonicalised rack info (called RackInfo) and set up a dictionary keyed on the rack value after expansion for placeholders and canonicalisation. This dictionary can then replace the HashSet.

The original LINQ to Objects query to perform the matching was

from line in FileLines(DictionaryFilename)
where line.Length == originalRack.Length
where racks.Contains(Canonicalize(line))
select line;

Since we’re working with the original rack and the rack combined with additional letters, the line length clause needs a minor tweak. The racks.Contains also needs tweaking to use ContainsKey. Other than that, the main part of the query remains unchanged!

from line in FileLines(DictionaryFilename)
where line.Length == originalRack.Length || line.Length == originalRack.Length+1
where groupedRackInfos.ContainsKey(Canonicalize(line))
select new { Word = line, OriginalRacks = groupedRackInfos[Canonicalize(line)] };

Notice that the results of this query is an IEnumerable that returns each matched word along with the racks that produced a match for that word. The results need to be re-shaped back into the matches for each rack, but even with this small amount of extra work (done with LINQ to Objects, naturally!) the timings speak for themselves:

Method	Search time(ms)
Original	2700
Updated	1658
Single file pass	108

Not a bad result. In fact, it feels near enough to instantaneous to me so I’m calling it a day on this one!

I should probably point out that this post isn’t a dig at the method that Eric used. He was the only consumer and decided (perfectly validly) that the performance was sufficient. On my machine I decided that it wasn’t (hmm, maybe I need to become more patient!). The key things about this for me are that you need to define your performance goals, measure the performance against those goals and then profile your app to determine where the performance bottlenecks are. Oh, and that LINQ to Objects is pretty powerful – I was able to make this change very easily and with minimal change to the original query.

Creating database connections with Unity – part 2

stuartle — Fri, 09 Jan 2009 16:21:55 GMT

Last time we looked at how to set up the configuration file so that Unity would wire up an object that took an IDbConnection parameter in its constructor. Whilst the solution works, it is easy for the various connection strings to become buried away in the Unity configuration. Also, the application configuration file already has somewhere to put connection details: the connectionStrings Element. This seemed like the ideal place to store my connection strings so I wanted a way to hook this up using Unity. After hunting around the documentation for a while, I noticed that as well as specifying parameters as dependencies (as in the previous post) you can also specify a value. This value has to be entered as a string in the configuration file but you can use the typeConverter attribute to specify the type converter that should be used to turn the string into the appropriate type. At this point the light bulb flicked on and DbConnectionNameTypeConverter entered stage left.

NOTE: the code below isn’t production ready (and as always, is subject to the standard disclaimer: “These postings are provided "AS IS" with no warranties, and confer no rights. Use of included script samples are subject to the terms specified at http://www.microsoft.com/info/cpyright.htm”).

public class DbConnectionNameTypeConverter : TypeConverter
{
    public override object ConvertFrom(ITypeDescriptorContext context, System.Globalization.CultureInfo culture, object value)
    {
        string connectionStringName = (string) value;
        ConnectionStringSettings connectionStringSettings = ConfigurationManager.ConnectionStrings[connectionStringName];

        DbProviderFactory providerFactory = DbProviderFactories.GetFactory(connectionStringSettings.ProviderName);
        IDbConnection connection = providerFactory.CreateConnection();
        connection.ConnectionString = connectionStringSettings.ConnectionString;
        return connection;
    }
}

This code simply takes the string passed in and looks up the connection details with that name. It uses the DbProviderFactory to create the connection, so it will handle SqlConnection or any other connection type that is registered. With this class in place, we can move the connection configuration to the connectionStrings section:

<connectionStrings>
    <add name="AdventureWorks"
         connectionString="Data Source=(local);Database=AdventureWorks;Integrated Security=SSPI;"
         providerName="System.Data.SqlClient" />
</connectionStrings>

And then configure Unity to use the type converter to get an IDbConnection instance from the connection name:

<types>
    <type type="MyProject.IRepository, MyProject, Version=1.0.0.0, Culture=neutral"
          mapTo="MyProject.DefaultRepository, MyProject, Version=1.0.0.0, Culture=neutral">
        <lifetime type="transient" />
        <typeConfig>
            <constructor>
                <param name="connection" parameterType="IDbConnection">
                    <value value="AdventureWorks" type="IDbConnection" typeConverter="DbConnectionNameTypeConverter" />
                </param>
            </constructor>
        </typeConfig>
    </type>
</types>

And bingo! We get all of the Unity goodness from the previous post but reuse the existing configuration section to centralise the connection settings!

Creating database connections with Unity

stuartle — Fri, 09 Jan 2009 01:09:33 GMT

I was adding dependency injection to an existing project and opted to use Unity configured via the application configuration file. As I was running through the configuration, I hit a type that required a database connection which it took in as an IDbConnection reference in the constructor so I added the following config under the container element:

<types>
    <type type="MyProject.IRepository, MyProject, Version=1.0.0.0, Culture=neutral"
          mapTo="MyProject.DefaultRepository, MyProject, Version=1.0.0.0, Culture=neutral">
        <lifetime type="transient" />
        <typeConfig>
            <constructor>
                <param name="connection" parameterType="IDbConnection">
                    <dependency />
                </param>
            </constructor>
        </typeConfig>
    </type>
</types>

(Before going any further, I should point out that I’ve added a type alias to the config for IDbConnection to save entering “System.Data.IDbConnection, System.Data, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089” each time.)

If you’re not familiar with the configuration file options for Unity, each type element tells Unity what to do when that type is requested. In the case of IRepository the mapTo attribute instructs Unity to create an instance of DefaultRepository. The typeConfig section is used to specify how you want Unity to create the type and in this case we are instructing it to use the constructor that takes a single parameter of type IDbConnection. The dependency sub-element tells Unity to use itself to resolve the value for the IDbConnection parameter, so we need to add that to the config:

<types>
    <type type="MyProject.IRepository, MyProject, Version=1.0.0.0, Culture=neutral"
          mapTo="MyProject.DefaultRepository, MyProject, Version=1.0.0.0, Culture=neutral">
        <lifetime type="transient" />
        <typeConfig>
            <constructor>
                <param name="connection" parameterType="IDbConnection">
                    <dependency />
                </param>
            </constructor>
        </typeConfig>
    </type>
    <type type="IDbConnection" 
          mapTo="SqlConnection" >
        <typeConfig>
            <constructor>
                <param name="connectionString" parameterType="System.String">
                    <value value="Data Source=(local);Database=AdventureWorks;Integrated Security=SSPI;" />
                </param>
            </constructor>
        </typeConfig>
    </type>
</types>

We’ve now given Unity enough information to resolve the IRepository type and wire up the IDbConnection parameter in the constructor. One problem with this config is that whenever we try to resolve an IDbConnection we’re going to get a connection to AdventureWorks and my application needs a couple of different database connections. Fortunately Unity provides for this situation and we can add a name to the type mapping. So the IDbConnection specification becomes:

<type type="IDbConnection" 
          mapTo="SqlConnection" 
          name="AdventureWorksConnection">

and we can now refer to this type by name in the parameter dependency:

<dependency name="AdventureWorksConnection" />

This allows us to define multiple connection entries that we simply refer to by name when we need to. Using this approach, the same connection details can be referenced in multiple places in the config so connecting to a different server or database is simply a case of changing the connection string in a single place. Also, assuming our repository class is able to deal with multiple database providers we can simply switch out the provider by changing the type Unity is creating for us (currently SqlConnection).

There’s lots more to Unity and for more information the Introduction to Unity page on MSDN is a good place to start. Next time we’ll take advantage of some of the flexibility of Unity to fix an something that I don’t really like about the above solution.

Microsoft AutoCollage 2008

stuartle — Mon, 15 Sep 2008 11:27:56 GMT

I just thought I’d mention that Microsoft AutoCollage 2008 has been released. It is based on research from Microsoft Research in Cambridge and allows you to automatically create collages of images (not that you’d have guessed it from the name ;-) ). You can find out more, see example images, and download a free 30-day trial version from http://research.microsoft.com/AutoCollage (you can follow the download links to purchase it at the Microsoft online store). There are lots of great examples on the AutoCollage page, but I thought I’d add a personal collage…

(click to enlarge)

In case you’re wondering, the collage is of James - the recent arrival to the Leeks clan :-D

DataServiceQuery.Expand

stuartle — Mon, 15 Sep 2008 11:25:00 GMT

ADO.Net Data Services allows you to expose your LINQ To Entities model (or LINQ To SQL model, or even your custom IQueryable model) via a RESTful API with minimal coding. For example, if you’re working with the Northwind database you can use the URL http://server/Service.svc/Customers(‘ALFKI’)/Orders to retrieve the orders for customer ALFKI. This simplicity makes it easy to retrieve data in a variety of scenarios – you simply need to be able to issue HTTP requests to access the data. To make it really easy to consume the data from client-side javascript, you can even specify that you want the data to be in JSON format. In rich client scenarios, you can either work with HTTP requests or if you like to make things really easy you can create a LINQ data model to access your ADO.Net Data Service. Simply point DataSvcUtil at your service and it will generate the LINQ classes you need. There’s a very good article by Shawn Wildermuth in the September 2008 issue of MSDN Magazine called ‘Creating Data-Centric Web Applications With Silverlight 2’ [1] that takes you through the process of creating a service and the LINQ classes to access the service from Silverlight. The article also discusses the mechanisms that ADO.Net Data Services uses to discover your entities if you want to expose your custom model (or if you’re just interested in how it works). I’ve also included a few extra links in the references at the end of the post if you want to find out more. Now, back to the point of the post...

The client-side code can use the DataServiceQuery<T>.Expand method to specify what properties should also be retrieved (much like the Include method in my last post Improving ObjectQuery<T>.Include). So you can write the following query to retrieve the customer ALFKI from the Northwind database and also pull back the orders:

var q = from c in context.Customers.Expand("Orders/Order_Details")
        where c.CustomerID == "ALFKI"
        select c;

Notice the call to the Expand Method. Much like the Include method for LINQ To Entities (see my last post), this lets yoe specify that you want certain related data to be pre-fetched. If you wanted to get the order details included as well then you could change the query to:

var q = from c in context.Customers.Expand("Orders/Order_Details")
        where c.CustomerID == "ALFKI"
        select c;

One point to notice is that it uses a forward slash (‘/’) rather than a decimal (‘.’) as the path separator, other than that it’s pretty similar to Include. Applying the same logic as in my last post, we can quickly create extension methods that allow us to write the above queries as

var q = from c in context.Customers.Expand(c => c.Orders)
        where c.CustomerID == "ALFKI"
        select c;

and

var q = from c in context.Customers.Expand(c => c.Orders.SubExpand(o=> o.Order_Details))
        where c.CustomerID == "ALFKI"
        select c;

Note that you can also expand multiple properties on the same entity. So if you were querying for orders, you can also bring back the customer and order details:

var q = from o in context.Orders.Expand(o=>o.Customer).Expand(o=>o.Order_Details)
        where o.Customer.CustomerID == "ALFKI"
        select o;

The code for the extension method is shown below – note that the code isn’t production ready (and as always, is subject to the standard disclaimer: “These postings are provided "AS IS" with no warranties, and confer no rights. Use of included script samples are subject to the terms specified at http://www.microsoft.com/info/cpyright.htm”). Since this code is very similar to my previous post, and gives you the same benefits, I’ll refer you to that post if you want to know how the code works and what the benefits are!

public static class DataServiceQueryExtensions
{
    public static DataServiceQuery<TSource> Expand<TSource, TPropType>(this DataServiceQuery<TSource> source, Expression<Func<TSource, TPropType>> propertySelector)
    {
        string expandString = BuildString(propertySelector);
        return source.Expand(expandString);
    }
    private static string BuildString(Expression propertySelector)
    {
        switch (propertySelector.NodeType)
        {
            case ExpressionType.Lambda:
                LambdaExpression lambdaExpression = (LambdaExpression)propertySelector;
                return BuildString(lambdaExpression.Body);

            case ExpressionType.Quote:
                UnaryExpression unaryExpression = (UnaryExpression)propertySelector;
                return BuildString(unaryExpression.Operand);

            case ExpressionType.MemberAccess:
                MemberInfo propertyInfo = ((MemberExpression)propertySelector).Member;
                return propertyInfo.Name;

            case ExpressionType.Call:
                MethodCallExpression methodCallExpression = (MethodCallExpression)propertySelector;
                if (IsSubExpand(methodCallExpression.Method)) // check that it's a SubExpand call
                {
                    // argument 0 is the expression to which the SubExpand is applied (this could be member access or another SubExpand)
                    // argument 1 is the expression to apply to get the expanded property
                    // Pass both to BuildString to get the full expression
                    return BuildString(methodCallExpression.Arguments[0]) + "/" +
                           BuildString(methodCallExpression.Arguments[1]);
                }
                // else drop out and throw
                break;
        }
        throw new InvalidOperationException("Expression must be a member expression or an SubExpand call: " + propertySelector.ToString());

    }

    private static readonly MethodInfo[] SubExpandMethods;
    static DataServiceQueryExtensions()
    {
        Type type = typeof(DataServiceQueryExtensions);
        SubExpandMethods = type.GetMethods().Where(mi => mi.Name == "SubExpand").ToArray();
    }
    private static bool IsSubExpand(MethodInfo methodInfo)
    {
        if (methodInfo.IsGenericMethod)
        {
            if (!methodInfo.IsGenericMethodDefinition)
            {
                methodInfo = methodInfo.GetGenericMethodDefinition();
            }
        }
        return SubExpandMethods.Contains(methodInfo);
    }

    public static TPropType SubExpand<TSource, TPropType>(this Collection<TSource> source, Expression<Func<TSource, TPropType>> propertySelector)
        where TSource : class
        where TPropType : class
    {
        throw new InvalidOperationException("This method is only intended for use with DataServiceQueryExtensions.Expand to generate expressions trees"); // no actually using this - just want the expression!
    }
    public static TPropType SubExpand<TSource, TPropType>(this TSource source, Expression<Func<TSource, TPropType>> propertySelector)
        where TSource : class
        where TPropType : class
    {
        throw new InvalidOperationException("This method is only intended for use with DataServiceQueryExtensions.Expand to generate expressions trees"); // no actually using this - just want the expression!
    }
}

References:

1: Creating Data-Centric Web Applications With Silverlight 2, Shawn Wildermuth. MSDN Magazine, September 2008. http://msdn.microsoft.com/en-us/magazine/cc794279.aspx

2 ADO.Net Data Services team blog. http://blogs.msdn.com/astoriateam/

3 ADO.Net Data Services homepage on MSDN. http://msdn.microsoft.com/en-us/data/bb931106.aspx

4 Mike Taulty has a number of blog posts on ADO.Net Data Services. http://mtaulty.com/CommunityServer/blogs/mike_taultys_blog/archive/category/1027.aspx

Improving ObjectQuery.Include

stuartle — Thu, 28 Aug 2008 01:35:00 GMT

** UPDATE: There’s a bug in the code below – see this post for the update!

One of the great features of LINQ To SQL and LINQ To Entities is that the queries you write are checked by the compiler, which eliminates typing errors in your query. Unfortunately, the ObjectQuery<T>.Include function (which is used to eager-load data that isn’t directly included in the query) takes a string parameter, opening up opportunities for typos to creep back in. In this post I’ll present some sample code that illustrates one way that you can work round this. To start with, let’s take a quick look at a query against an entity model on the Northwind database.

var query = from customer in context.Customers
            where customer.Orders.Count > 0
            select customer;

This query simply retrieves customers with an order. If the code that uses the query results then needs to make use of the order data, it won’t have been loaded. We can use the Include function to ensure that this data is loaded up front:

var query = from customer in context.Customers.Include("Orders")
            where customer.Orders.Count > 0
            select customer;

Notice the Include(“Orders”) call that we’ve inserted which instructs Entity Framework to retrieve the Orders for each Customer. It would be much nicer if we could use a lambda expression to specify what property to load:

var query = from customer in context.Customers.Include(c => c.Orders)
            where customer.Orders.Count > 0
            select customer;

It turns out that this is very easy to achieve by using an extension method:

public static class ObjectQueryExtensions
{
    public static ObjectQuery<TSource> Include<TSource, TPropType>(this ObjectQuery<TSource> source, Expression<Func<TSource, TPropType>> propertySelector)
    {
        MemberExpression memberExpression = propertySelector.Body as MemberExpression;
        if (memberExpression== null)
        {
            throw new InvalidOperationException("Expression must be a member expression" + propertySelector);
        }
        MemberInfo propertyInfo = memberExpression.Member;
        return source.Include(propertyInfo.Name);
    }
}

This Include extension method allows the query syntax above with the lambda expression. When the Include method is called, it inspects the Expression Tree. If the method is used as intended, the tree will describe a accessing a member of the TSource class. We can then us the name of the member to call the original Include function.

Whilst this solves the problem as I described it above, what if the code consuming the query also needed the order details? With the original Include function we can write

var query = from customer in context.Customers.Include("Orders.Order_Details") 
            where customer.Orders.Count > 0
            select customer;

Notice that we can pass a path to the properties to include. The extension method we wrote doesn’t give us a way to handle this case. I imagine using something like the syntax below to describe this situation

var query = from customer in context.Customers.Include(c => c.Orders.SubInclude(o => o.Order_Details))
            where customer.Orders.Count > 0
            select customer;

Here, we’ve specified that we want to include Orders, and then also that we want to include Order_Details. Adding this support is a bit more code, but not too bad:

public static class ObjectQueryExtensions
{
    public static ObjectQuery<TSource> Include<TSource, TPropType>(this ObjectQuery<TSource> source, Expression<Func<TSource, TPropType>> propertySelector)
    {
        string includeString = BuildString(propertySelector);
        return source.Include(includeString);
    }
    private static string BuildString(Expression propertySelector)
    {
        switch(propertySelector.NodeType)
        {
            case ExpressionType.Lambda:
                LambdaExpression lambdaExpression = (LambdaExpression)propertySelector;
                return BuildString(lambdaExpression.Body);

            case ExpressionType.Quote:
                UnaryExpression unaryExpression= (UnaryExpression)propertySelector;
                return BuildString(unaryExpression.Operand);

            case ExpressionType.MemberAccess:
                MemberInfo propertyInfo = ((MemberExpression) propertySelector).Member;
                return propertyInfo.Name;

            case ExpressionType.Call:
                MethodCallExpression methodCallExpression = (MethodCallExpression) propertySelector;
                if (IsSubInclude(methodCallExpression.Method)) // check that it's a SubInclude call
                {
                    // argument 0 is the expression to which the SubInclude is applied (this could be member access or another SubInclude)
                    // argument 1 is the expression to apply to get the included property
                    // Pass both to BuildString to get the full expression
                    return BuildString(methodCallExpression.Arguments[0]) + "." +
                           BuildString(methodCallExpression.Arguments[1]);
                }
                // else drop out and throw
                break;
        }
        throw new InvalidOperationException("Expression must be a member expression or an SubInclude call: " + propertySelector.ToString());

    }

    private static readonly MethodInfo[] SubIncludeMethods;
    static ObjectQueryExtensions()
    {
        Type type = typeof (ObjectQueryExtensions);
        SubIncludeMethods = type.GetMethods().Where(mi => mi.Name == "SubInclude").ToArray();
    }
    private static bool IsSubInclude(MethodInfo methodInfo)
    {
        if (methodInfo.IsGenericMethod)
        {
            if (!methodInfo.IsGenericMethodDefinition)
            {
                methodInfo = methodInfo.GetGenericMethodDefinition();
            }
        }
        return SubIncludeMethods.Contains(methodInfo);
    }

    public static TPropType SubInclude<TSource, TPropType>(this EntityCollection<TSource> source, Expression<Func<TSource, TPropType>> propertySelector)
        where TSource : class, IEntityWithRelationships
        where TPropType : class
    {
        throw new InvalidOperationException("This method is only intended for use with ObjectQueryExtensions.Include to generate expressions trees"); // no actually using this - just want the expression!
    }
    public static TPropType SubInclude<TSource, TPropType>(this TSource source, Expression<Func<TSource, TPropType>> propertySelector)
        where TSource : class, IEntityWithRelationships
        where TPropType : class
    {
        throw new InvalidOperationException("This method is only intended for use with ObjectQueryExtensions.Include to generate expressions trees"); // no actually using this - just want the expression!
    }
}

This code still has the Include method with the original signature, and adds a couple of SubInclude extension methods. You can see that the code to extract the property name has been pulled out into a separate method (BuildString). This now also handles some additional NodeTypes so that we can handle the SubInclude calls inside the Include call. There are some checks in to ensure that we are dealing with the SubInclude calls at this point (using the IsSubInclude method). With this code, we can write the previous query as well as:

var query = from customer in context.Customers.Include(c => c.Orders.SubInclude(o => o.Order_Details).SubInclude(od=>od.Products))
            where customer.Orders.Count > 0
            select customer;

This query includes the Orders, Order Details, and Products as if we’d called Include(“Orders.Order_Details.Product”) and in fact this is what the code will do! Additionally, it doesn’t matter whether you chain the SubInclude calls (as above) or nest them:

var query = from customer in context.Customers.Include(c => c.Orders.SubInclude(o => o.Order_Details.SubInclude(od => od.Products)))
            where customer.Orders.Count > 0
            select customer;

Both of these queries have the same effect, so it’s up to you which style you prefer.

The code isn’t production ready (and as always, is subject to the standard disclaimer: “These postings are provided "AS IS" with no warranties, and confer no rights. Use of included script samples are subject to the terms specified at http://www.microsoft.com/info/cpyright.htm”). However, there are a couple of other features that I think are worth briefly mentioning:

The IsSubInclude function works against a cached MethodInfos for the SubInclude methods. Because these methods are generic, we have to get the generic method definition to test for comparison
The SubInclude functions are not intended to be called at runtime - they are purely there to get the compiler to generate the necessary expression tree!
Generic type inference is at work when we write the queries. Notice that we could write Include(c => c.Orders) rather than Include<Customer>(c => c.Orders). Imagine how much less readable the code would become, especially when including multiple levels.

I found it quite interesting putting this code together as it pulls in Extension Methods, LINQ To Entities and Expression Trees. The inspiration for this came from the LinkExtensions.ActionLink methods in ASP.Net MVC framework which do the same sort of thing for ActionLinks. I'’m not sure if I’m entirely satisfied with the syntax for including multiple levels, but it is the best I’ve come up with so far. If you’ve any suggestions for how to improve it then let me know!

A closer look at yield – part 3

stuartle — Wed, 13 Aug 2008 15:37:21 GMT

This was only going to be two posts, but after my last post I’d been mulling over a post that looks at the compiler generated code in a more general way. Whilst catching up on blogs posts this morning I saw that Raymond Chen has written a blog post entitled ‘The implementation of iterators in C# and its consequences (part 1)’ that does a better job than I would have! Better still, it looks like there’s more to come…