jaredpar's WebLog : Orcas

Type Inference and IEnumerable

Jared Parsons — Tue, 27 Nov 2007 01:32:10 GMT

This is somewhat of a follow up on a previous post I did on the difference between IEnumerable(Of T) and the IEnumerable interfaces.

I've seen several people type in the following code and wonder if there was a fundamental bug in the type inference code.

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        For Each cur In Controls
            cur.Text = "A Value"
        Next
    End Sub

This code will produce an error stating that "Text" is not a member of object. Users expected type inference to type the variable "cur" as Control. Unfortunately this is "By Design".

Much of the original .Net Framework was written before the CLR implemented support for generics. As a result all of the collection classes were loosely typed to Object by implementing IEnumerable. So in this case type inference will correctly type this as Object.

There are 2 ways to fix this problem.

Explicitly type the For Each variable to be the actual type of objects in the collection
Use a Shim to change the type of the collection. (see http://blogs.msdn.com/jaredpar/archive/2007/10/04/ienumerable-and-ienumerable-of-t.aspx)

VS2008 Ships

Jared Parsons — Tue, 20 Nov 2007 06:41:49 GMT

Orcas has shipped :)

http://msdn2.microsoft.com/en-us/vstudio/default.aspx

Lambda Unexpected Behavior

Jared Parsons — Tue, 20 Nov 2007 06:38:52 GMT

One item you strive to avoid when you design and implement a feature is unexpected behavior. Unfortunately there is one case we couldn't avoid with Lambda's in VB9. I just ran into the this problem when coding up a handler. I wanted to disable a button when the text of particular TextBox was empty. I wrote the following code to handle the situation.

AddHandler c.TextChanged, Function() okButton.Enabled = (0 <> c.Text.Length)

This doesn't quite do what I intended. This instead will simply compare the two values.

In VB9 Lambda Expressions are always an expression. In version 9 of VB, there is no concept of an assignment as an expression. There is only a statement version. As a result this doesn't do anything useful.

This has tripped up a few people along the way. It's an unfortunate side effect of only supporting expression lambdas.

I was able to work around this by defining a function which did what I intended. I called this function in the lambda expression and the problem was solved.

Extension Methods without 3.5 Framework

Jared Parsons — Fri, 16 Nov 2007 23:37:00 GMT

For a time I've been avoiding extension methods. Not because I'm opposed to using them but because of the 3.5 Framework.

A lot of the tools I own are designed to be very light weight tools that only require the user to have 2.0 installed on their machine. I find that the easier that tools are to install, the more likely people are to use them.

Extension methods require the ExtensionAttribute be available. Since the attribute is declared in a 3.5 Framework assembly it's not possible to use extension methods without the 3.5 framework. At least, that's what I thought up until I read an recent MSDN article.

You can simply define the ExtensionAttribute in your assembly and extension methods will start working. No references to the 3.5 framework required. It's a lightweight solution that adds the full power of extension methods to your program.

Namespace System.Runtime.CompilerServices
    Class ExtensionAttribute
        Inherits Attribute
    End Class
End Namespace

Singleton Pattern

Jared Parsons — Tue, 04 Sep 2007 20:58:00 GMT

Quite awhile back I posted about how to create a re-usable singleton pattern in .Net. Link is here. A bit of time has passed and I've altered the pattern a bit. The reasons for the change are some new type inference patterns and FxCop cleanliness.

The first pattern I introduced had a couple of FxCop violations. Namely Microsoft.Design CA1000 - Do not declare static members on generic types. The logic here being that static methods don't have any type inference benefits as you must explicitly add the type into the name of the type you were using (in this case Singleton).

Secondly because T was at a class level rather than a method level I couldn't have granualar methods which had differing set of constraints. The result was a pattern that was not always easy to write out.

The new pattern takes care of both of these problems. It has two methods. One of which can be satisfied with a trivial lambda expression. The other can easily be used for classes that satisfy the generic constraint new() with no additional lambda.

    class c1
    {
        public static c1 Instance1 
        {
            get { return Singleton.GetInstance(() => new c1());}
        }

        public static c1 Instance2
        {
            get { return Singleton.GetInstance<c1>(); }
        }

        public c1()
        {

        }
        
    }

Below is the new singleton pattern.

    public delegate T Operation<T>();

    /// <summary>
    /// Used for classes that are single instances per appdomain
    /// </summary>
    public static class Singleton
    {
        private static class Storage<T>
        {
            internal static T s_instance;
        }

        [SuppressMessage("Microsoft.Reliability", "CA2002")]
        public static T GetInstance<T>(Operation<T> op)
        {
            if (Storage<T>.s_instance == null)
            {
                lock (typeof(Storage<T>))
                {
                    if (Storage<T>.s_instance == null)
                    {
                        T temp = op();
                        System.Threading.Thread.MemoryBarrier();
                        Storage<T>.s_instance = temp;
                    }
                }
            }
            return Storage<T>.s_instance;
        }

        public static T GetInstance<T>()
            where T : new()
        {
            return GetInstance(() => new T());
        }
    }

    #endregion
}

Edit: Originally forgot to add the signature for Operation<T>

C# Type inference change for Orcas

Jared Parsons — Tue, 07 Aug 2007 19:06:42 GMT

While playing around with a batch of Orcas code recently I found a welcome change to the C# type inference rules for Orcas. The return type of a generic delegate can now be inferred from the actual return values. Here is some sample code demonstrating the problem.

    class Program
    {
        public delegate T Operation<T>();

        public static void M<T>(Operation<T> del)
        {

        }

        static void Main(string[] args)
        {
            M(delegate { return 42; });
        }
    }

In Whidbey the above code will fail with a compiler error.

error CS0411: The type arguments for method 'ConsoleApplication34.Program.M<T>(ConsoleApplication34.Program.Operation<T>)' cannot be inferred from the usage. Try specifying the type arguments explicitly.

This is really frustrating in Whidbey because the you have to modify the call to include the actual type specifier.

            M<int>(delegate { return 42; });

However this code now works without modification in Orcas. This is a welcome and extremely useful change.

And yes, the same inference is possible with VB in Orcas. In Whidbey this was not an issue because VB did not support lambda expressions.

Module Module1

    Public Delegate Function Operation(Of T)() As T

    Public Sub M(Of T)(ByVal del As Operation(Of T))

    End Sub

    Sub Main()
        M(Function() 1)
    End Sub

End Module

Closures in VB Part 6: Limitations

Jared Parsons — Tue, 07 Aug 2007 01:36:40 GMT

For previous articles in this series please see ...

As powerful as closures are in the language they do have a few limitations. We worked hard in Orcas to put as few limitations in Orcas as possible. Below are the current limitations and some insight into why they exist this way.

1. Cannot use "ByRef" parameters in a closure

Example:

    Sub LiftAByRef(ByRef x As Integer)
        Dim f = Function() x
    End Sub

Message: error BC36639: 'ByRef' parameter 'x' cannot be used in a lambda expression.

The problem here is the expectation surrounding x. Any change in the value of "x" inside the method "LiftAByRef" should be reflected in the calling function. Normally for any lifted parameter we add a new field inside the closure and all read/writes are redirected into that value. For "ByRef" parameters we would additionally have to ensure that all writes are make to the parameter. Even in the presence of an exception. Not a trivial task.

2. Cannot use "Me" in a closure created inside a structure.

Example:

Structure S1
    Public F1 As Integer

    Public Sub M1()
        Dim f = Function() F1
    End Sub
End Structure

Message: error BC36638: Instance members and 'Me' cannot be used within a lambda expression in structures

Closures capture values by reference. It's not possible to capture the "Me" of a structure by reference in VB. The only other option is to capture them by value. If we did that then all changes to members of a structure inside a lambda would not affect the structure in which they were created; merely the value copy. This is very different behavior from every other place that closures are used. To avoid confusing behavior this is not a legal operation.

3. Cannot use a Restricted Type in a closure

Example:

    Sub LiftRestrictedType()
        Dim x As ArgIterator = Nothing
        Dim f = Function() x.GetNextArgType().GetModuleHandle()
    End Sub

Message: error BC36640: Instance of restricted type 'System.ArgIterator' cannot be used in a lambda expression.

This hopefully will not affect many users. There are several types in the CLR that are considered restricted because they have special semantics. Typically they are special cased by the CLR and as such we can't use them in a closure. Several of these cannot be used in VB at all. They are ...

System.TypedReference
System.ArgIterator
System.RuntimeArgumentHandle

4. Cannot Goto into scope that contains a closure

Example:

    Sub BadGoto()
        Dim x = 0

        GoTo Label1
        If x > 5 Then
Label1:
            Dim y = 5
            Dim f = Function() y
        End If
    End Sub

Message: error BC36597: 'GoTo Label1' is not valid because 'Label1' is inside a scope that defines a variable that is used in a lambda or query expression.

If you look back at Part 3 of this series you will see that a lot of work goes into initializing closures inside of a scope. Unfortunately allowing a user to jump into a block that contains a closure makes respecting these rules very difficult. In a even trivial example in makes the resulting code mostly unreadable. We decided to disable this in Orcas and reconsider it in a future release.

It is perfectly legal however to jump into any scope that is currently visible regardless of whether or not in contains a closure. Because jumping into a visible scope does not affect the creation of a variable lifetime (just the ending), it does not add any complications to the code.

5. Cannot mix "On Error Goto" and Closures

Message: error BC36595: Method cannot contain both an 'On Error GoTo' statement and a lambda or query expression.

Because of restriction #4 we must disable this scenario as well since it's very easy to hit this scenario with "On Error Goto".

Expression Evaluator Sample Released

Jared Parsons — Wed, 01 Aug 2007 19:37:00 GMT

The new version of the Visual Studio 2008 Beta2 SDK was just released. Among the many new samples is a basic expression evaluator sample. This sample shows you how to add basic debugger support to a managed language in Visual Studio.

This is the first version of the sample and mainly has support for locals and member expansion. I'm planning on adding more features in future releases. Since this is mainly to provide users with guidelines on how to build an EE, any requests we get will be given a lot of consideration.

http://www.microsoft.com/downloads/details.aspx?FamilyID=d9000e2c-bd3f-4717-a181-723960814e16&displaylang=en

Coding Quiz: Anonymous Type Types

Jared Parsons — Wed, 01 Aug 2007 18:34:37 GMT

Question: How can you create a variable in VB which is typed as an anonymous type but not actually create an instance of that type?

Answer in comments.

Note, doing this is not particularly useful it came about while I was playing around with a feature a few days ago.

Closures in VB Part 5: Looping

Jared Parsons — Thu, 26 Jul 2007 19:39:05 GMT

For previous articles in the series please see

Once again sorry for the long delay between posts.

Looping structures can cause unintended consequences when used with Lambda expressions. The problem occurs because lambda expressions do not execute when they are constructed but rather when they are invoked. For example take the following code.

    Sub LoopExample1()
        Dim list As New List(Of Func(Of Integer))
        For i = 0 To 3
            list.Add(Function() i)
        Next

        For Each cur In list
            Console.Write("{0} ", cur())
        Next
    End Sub

Many users are surprised to find out the above will print "4 4 4 4 ". The reason goes back to my previous 2 posts on variable lifetime and scope. All "For" and "For Each" blocks in Vb have 2 scopes.

Scope where iteration variables are defined
Body of the for loop

The first scope is entered only once no matter how many times the loop is executed. The second is entered once per iteration of the loop. Any iteration variables that are defined in a For/For Each loop are created in the first scope (in this case "i" and "cur"). Hence there is only one of those variables for every loop iteration and the lambda function lifts the single variable "i".

This has thrown off many users because the behavior works most of the time. For instance if I switched the code to run "Console.Write" inside the first loop, it would print out "0 1 2 3 " as expected.

To mitigate against this problem the above code will actually produce a warning in VB.

warning BC42324: Using the iteration variable in a lambda expression may have unexpected results. Instead, create a local variable within the loop and assign it the value of the iteration variable.

There are two ways to fix this problem depending on the behavior you want. If you see this warning and don't know if it affects you, the safest change is to do the following.

    Sub LoopExample2()
        Dim list As New List(Of Func(Of Integer))
        For iTemp = 0 To 3
            Dim i = iTemp
            list.Add(Function() i)
        Next

        For Each cur In list
            Console.Write("{0} ", cur())
        Next
    End Sub

This will cause "i" to be created in the second scope. Hence there will be a different value for every loop iteration and the code will print out "0 1 2 3" as expected.

If you do want the code to print out "4 4 4 4 " then add "Dim i = 0" before the start of the loop.

Closures in VB Part 4: Variable Lifetime

Jared Parsons — Fri, 15 Jun 2007 23:47:44 GMT

For previous articles in this series please see

Sorry for the long delay between posts here. We're getting Orcas out the door and getting this series completed takes a back door to shipping.

Originally I wanted to talk about looping structures next. However when I started writing that post I realized that I had to talk about lifetime before the looping structures would make sense.

Prior to Orcas the lifetime of a variable in VB was the entire function. This presented several problems from a closures perspective. Imagine you had a looping structure and the value was used in a lambda expression.

    Sub LifetimeExample()
        Dim list As New List(Of Func(Of Integer))
        For i = 0 To 5

            Dim x = i * 2
            If True Then
                list.Add(Function() x)
            End If
        Next

        For Each f In list
            Console.Write(f() & " ")
        Next
    End Sub

In this example if we left the lifetime rules unchanged, there would be a single variable "x" for the entire function. That means that we would end up printing out

10 10 10 10 10

This is somewhat unexpected and essentially means that VB could not support complex Lambda scenarios. To fix this we altered the lifetime of variables to be tied to the scope they were contained in. The end effect is that each iteration of the loop has a separate "x" since each iteration enters and leaves the scope of the "if" statement. As a result it will print out

0 2 4 6 8 10

We did make one backcompat adjustment for this change. The lifetime of variables in VB was visible if you tried to use an uninitialized variable in a loop/goto. For instance the following code will also print out 0 2 4 6 8 10 because it takes advantage of the fact that the variable "x" has a lifetime longer than the loop.

    Sub VisibleLifetime()
        For i = 0 To 5
            Dim x As Integer
            Console.WriteLine(x)
            x += 2
        Next
    End Sub

To make sure that we didn't break any existing code we had one little errata for the change. When a variable's scope is re-entered, and hence recreated, and it is not initialized to a value it will get the last value of the variable.

Closures in VB Part 3: Scope

Jared Parsons — Fri, 25 May 2007 23:59:45 GMT

For previous articles in this series please see

Thus far in the series we've only lifted variables that are declared in the same block/scope. What happens if we lift variables in different scope? The answer is that one closure class will be created for every unique scope where a lifted variable is declared and all of the variables in that scope that are lifted will be placed in that closure. Once again, examples speak best

    Sub Scope1()
        Dim x = 5
        Dim f1 = Function(ByVal z As Integer) x + z
        Console.WriteLine(f1(5))
        If x > 2 Then
            Dim y = 6
            Dim g = 7
            Dim f2 = Function(ByVal z As Integer) z + y + g
            Console.WriteLine(f2(4))
        End If
    End Sub

The code will end up looking like so ...

    Class Closure1
        Public x As Integer

        Function Lambda_f1(ByVal z As Integer)
            Return x + z
        End Function

    End Class

    Class Closure2
        Public y As Integer
        Public g As Integer

        Function Lambda_f2(ByVal z As Integer)
            Return y + z + g
        End Function
    End Class

    Sub Scope1()
        Dim c1 As New Closure1()
        c1.x = 5
        Console.WriteLine(c1.Lambda_f1(5))
        If c1.x > 2 Then
            Dim c2 As New Closure2()
            c2.y = 6
            c2.g = 7
            Console.WriteLine(c2.Lambda_f2(4))
        End If
    End Sub

There are a couple of items to take away from this example.

Only two closure classes were created even though three variables were lifted. The number of closures only depends on the number of scopes of all of the lifted declared variables.
The closures are created at the begining of the scope they are associated and not at the begining of the method. This will be more important in the next part of the series.
Each lambda instance is attached to the closure associated with the scope the lambda is declared in.

The next twist is what were to happen if the lambda "f2" were to also use the variable "x". As it's currently written there is no association between Closure1 and Closure2 therefore there is no way for it to access the lifted variable. The answer is two fold. Firstly to reduce clutter I pasted the closure classes as if they were separate entries. In fact Closure2 would appear as a nested class of Closure1 in the real generated code.

Secondly if x were used inside of "f2", the real use would be "c1.x". That's (almost) no different than "someOtherVar.x". Therefore the instance of c1 will be lifted into Closure2.

Dim f2 = Function(ByVal z As Integer) z + y + g + x

Woud result in the following definition of Closure2 ...

    Class Closure2
        Public y As Integer
        Public g As Integer
        Public c1 As Closure1

        Function Lambda_f2(ByVal z As Integer)
            Return y + z + g + c1.x
        End Function
    End Class

In deeply nested lambdas and scopes this type of lifting will continue recursively.

That's it for this entry, the next article will talk about looping structures and possibly variable lifetime.

Closures in VB Part 2: Method Calls

Jared Parsons — Fri, 04 May 2007 00:56:51 GMT

For previous articles in this series, please see

Part 1 - The basics

This part of the series will focus on how method calls are handled in closures. As stated in the previous article, the purpose of closures is to allow all operations inside a lambda or query expression that would normally be available inside the function or sub. To do this closures often need to capture (or lift) relevant variables from the function into the generated class.

There are 2 types of methods and method calls that closures have to handle.

Method calls to a shared method or methods on modules.
Method calls to instance members of a class

Scenario #1

Below is an example of a method call inside a lambda expression for scenario #1.

Module M1

    Function MyValue() As Integer
        Return 42
    End Function

    Sub Example1()
        Dim x = 5
        Dim f = Function() x + MyValue()
    End Sub

End Module

Here we are calling a module method inside a lambda. Module Methods or Shared methods can be called from anywhere because they require no specific variable for the call. This requires no special work from closures as the call can just be made naturally.

    Class Closure
        Private x As Integer

        Function Lambda_f() As Integer
            Return x + M1.MyValue
        End Function
    End Class

Scenario #2

Calling an instance method is more difficult than a shared method because it requires the referenc "Me". If you don't type this specifically in code the VB Compiler will add it for you under the hood. To make this work the closures code will also "lift" the variable "Me" in the same way that it lifts normal variables in a function.

Calling a instance method inside a lambda expression is little difference than calling a member method on a variable used in a lambda. The only difference is the variable is "Me". For example

Class C1
    Private m_myValue As Integer

    Function MyValue() As Integer
        Return m_myValue
    End Function

    Sub Example2()
        Dim x = 5
        Dim f = Function() x + MyValue()
    End Sub
End Class

In this case we need to access both "x" and "Me.MyValue()" from the closure. The generated code will create space for both of these variables and the transformed code in Example2 will store both of the values.

Class Closure
    Private x As Integer
    Private OriginalMe As C1

    Function Lambda_f()
        Return x + OriginalMe.MyValue()
    End Function
End Class

    Sub Example2()
        Dim c As New Closure
        c.x = 5
        c.OriginalMe = Me
        Dim f = New Func(Of Integer)(AddressOf c.Lambda_f)
    End Sub

As usual, the generated code is much uglier but this essentially what will be generated. That wraps it up for method calls. In the next part, I will discuss the variable liftetime and scoping issues that come into play with closures.

Closures in VB: Part 1

Jared Parsons — Sat, 28 Apr 2007 01:50:00 GMT

One of the features I implemented for VB 9.0 is lexical closure support. This a great addition to the VB language and I wanted to do a series of blog posts to describe this feature and how it will impact your code.

Lexical Closures (more often referred to as simply Closures) are the underpinnings for several new features in Visual Basic 9.0. The are part of the guts of Lambda and Query expressions. This will be a several part series on Closures in VB 9.0; how they work, their limitations, pitfalls surrounding their use.

To start off, let's get a basic summary of what a Closure is. Wikipedia defines it as "... a is a semantic concept referring to a function paired with an environment ...". I prefer to describe it as follows. A closure is a feature which allows users to seemlessly access an environment (locals, parameters and methods) from more than one function. Even better are samples :)

    Class C1
        Sub Test()
            Dim x = 5
            Dim f = Function(ByVal y As Integer) x + y
            Dim result = f(42)
        End Sub

    End Class

In this code we have a lambda expression which takes in a single parameter and adds it with a local variable. Lambda expressions are implemented as functions in VB (and C#). So now we have two functions, "Test" and "f", which are accessing a single local variable. This is where closures come into play. Closures are responsible for making the single variable "x" available to both functions in a process that is referred to as "lifting the variable".

To do this the compiler will take essentially 4 actions.

Create a class which will contain "x" in order to share it among both functions. Call it "Closure" for now
It will create a new function for the lambda expression in the class "Closure". Call it "f" for now
Create a new instance of the class "Closure" inside the sub "Test"
Rewrite all access of "x" into the member "x" of "Closure".

    Class Closure
        Public x As Integer

        Function f(ByVal y As Integer) As Integer
            Return x + y
        End Function
    End Class

    Class C1
        Sub Test()
            Dim c As New Closure()
            c.x = 5
            Dim f As Func(Of Integer, Integer) = AddressOf c.f
            Dim result = f(42)
        End Sub

    End Class

Now "x" is shared amongst both functions and the user didn't have to know anything about the code we generated. You can see from this simplified example just how much code Closures and all of the other new VB 9.0 features are saving you here (Type Inference, Lambda Expressions).

Note this is only a simulation of what is generated when you use a closure, the actual generated code is much uglier and involves lots of unbindable names "$Lambda_1", etc ...

In the next part of this article I'll dive into some more uses of closures (multiple variables, method access, terminology, etc...).