VB.Net and other things

Projecting in more details (single, multiple items, anonymous types)

VladimirSadov — Thu, 28 Jun 2007 05:55:00 GMT

One interesting aspect of VB queries is that you can select multiple values. When you do this, the values get combined into an anonymous type. “Anonymous types” is a new feature for VB9. Essentially an anonymous type is a class that contains one or more named values. Compiler simply generates an anonymous type with number, type and names of contained values matching the variables produced by the selector. Here is a simple example:

Module Module1

Sub Main()

Dim arr() As Integer = {1, 2, 3, 4}

Dim q = From n In arr Select num = n, square = n ^ 2

For Each record In q

Console.WriteLine(record)

End Sub

End Module

The output:

{ num = 1, square = 1 }

{ num = 2, square = 4 }

{ num = 3, square = 9 }

{ num = 4, square = 16 }

The values that are combined into an anonymous type do not need to be defined in the same operator. As long as you have more than one iteration variable at the end of the query, they will be parts of composite element of resulting sequence. Explicit Select-ing is not required. Example:

Module Module1

Sub Main()

Dim arr1() As Integer = {1, 2, 3, 4}

Dim arr2() As String = {"a", "b", "c"}

' this query defines control variable n

' then (after filtering) it adds variable k

' and does more filtering

' at the end we have both n and k variables visible

Dim q = From n In arr1 _

Where n > 1 _

From k In arr2 _

Where (k <> "b")

For Each record In q

Console.WriteLine(record)

End Sub

End Module

Output:

{ n = 2, k = a }

{ n = 2, k = c }

{ n = 3, k = a }

{ n = 3, k = c }

{ n = 4, k = a }

{ n = 4, k = c }

Flattening a nested list using queries.

VladimirSadov — Thu, 28 Jun 2007 03:12:00 GMT

This seems to be a common question - " I have a list of lists (or array or arrays), how can I flatten it into a single list/array? "

It is surprisingly trivial to do with queries. It takes one line of code -

Module Module1

Sub Main()

'create a list of lists

Dim list As New List(Of List(Of Integer))

For i = 0 To 5

Dim inner_list As New List(Of Integer)

For j = 0 To 3

inner_list.Add(j)

list.Add(inner_list)

'======

' get flattened IEnumerable

'=====

Dim q = From i In list, j In i Select j

'print out the sequence

For Each j In q

Console.Write(j)

Console.WriteLine()

'can make it into an array if needed

Dim arr() As Integer = q.ToArray

'print out the array

For Each j In arr

Console.Write(j)

Console.WriteLine()

End Sub

End Module

Adding more operators with Where as an example ( Query Design continued ):

VladimirSadov — Sat, 09 Jun 2007 05:42:00 GMT

This post is a part of a series of posts about query design. For the previous post see: http://blogs.msdn.com/vladimirsadov/archive/2007/06/07/query-pattern-composability-query-design-continued.aspx

As you might have seen queries have a lot more operators than just Select. Here we will take a look how we can add support for Where to our queryable type.

It is surprisingly easy to implement Where. Compiler will use any function that is named Where, can be applied to the source collection and takes a delegate function that takes an element of the collection and returns a Boolean result.

I will not do delayed execution tricks in this sample for simplicity (do not want to implement all that IEnumerable support again), however it is fairly trivial to generalize this sample so that both Select and Where are delay-executed.

Class MyGenericQueryable(Of T)

Public elements As IEnumerable(Of T)

Public Function [Select](Of S)(ByVal projection As Func(Of T, S)) As MyGenericQueryable(Of S)

Dim new_col As New MyGenericQueryable(Of S)

Dim new_elements As New List(Of S)

new_col.elements = new_elements

For Each element In elements

new_elements.Add(projection(element))

Return new_col

End Function

Public Function Where(ByVal predicate As Func(Of T, Boolean)) As MyGenericQueryable(Of T)

Dim new_col As New MyGenericQueryable(Of T)

Dim new_elements As New List(Of T)

new_col.elements = new_elements

For Each element In elements

If predicate(element) Then

new_elements.Add(element)

End If

Return new_col

End Function

End Class

Module Module1

Sub Main()

Dim arr As Integer() = New Integer() {1, 2, 3}

Dim col As New MyGenericQueryable(Of Integer)

col.elements = arr

Dim q = From i In col Where i > 1 Select i

For Each t In q.elements

Console.WriteLine(t)

End Sub

End Module

Delayed Execution ( Query Design continued ):

VladimirSadov — Sat, 09 Jun 2007 05:34:00 GMT

This post is a part of a series of posts about query design. For the previous post see: http://blogs.msdn.com/vladimirsadov/archive/2007/06/07/query-pattern-composability-query-design-continued.aspx

Here is another important example of a queryable type. There are two differences from the previous examples:

It uses System.Func delegate type that is defined in System.Core.dll . There is nothing special in using this type. VB compiler will accept any delegate type that matches the queryable pattern. However using predefined delegate type reduce the amount of code to type.

Another very important difference is that the Select does not create another collection with projected elements. Instead it creates an object that “knows” how to produce projected elements when asked for them. There are many advantages in doing that. First of all we may not need all the elements so it makes sense to delay computation until the elements are needed. Secondly for a big source collection, the resulting collection will take much less space. This is especially important if you compose a query with multiple projections applied in sequence.

Basically instead of creating another collection for every Select in a composed query we will build a pipeline of enumerable types that represent projection delegates. When another element is requested from the query, by delegating the request through the chain of these enumerable types, we will get the next element from the source collection and every projection in the pipeline will be applied to the element.

This concept is commonly known as a delayed execution. It does complicate the debugging somewhat as there seem to be too many levels of indirection. However the benefits far outweigh that.

Here is the sample
(it is not as big as it seems as most parts are just required to implement IEnumerable pattern):

Class EnumerableQueryable(Of T)

Implements IEnumerable(Of T)

Private elements As IEnumerable(Of T)

Sub New(ByVal elements As IEnumerable(Of T))

Me.elements = elements

End Sub

Public Function [Select](Of S)(ByVal selector As Func(Of T, S)) As EnumerableQueryable(Of S)

Dim new_enum = New Enumerable4Select(Of T, S)(Me, selector)

Dim new_col As New EnumerableQueryable(Of S)(new_enum)

Return new_col

End Function

Public Function GetEnumerator() As System.Collections.Generic.IEnumerator(Of T) Implements System.Collections.Generic.IEnumerable(Of T).GetEnumerator

Return elements.GetEnumerator

End Function

Public Function GetEnumerator1() As System.Collections.IEnumerator Implements System.Collections.IEnumerable.GetEnumerator

Return GetEnumerator()

End Function

End Class

Class Enumerable4Select(Of T, S)

Implements IEnumerable(Of S)

Private collection As IEnumerable(Of T)

Private selector As Func(Of T, S)

Public Sub New(ByVal coll As IEnumerable(Of T), ByVal func As Func(Of T, S))

collection = coll

selector = func

End Sub

Public Function GetEnumerator() As System.Collections.Generic.IEnumerator(Of S) Implements System.Collections.Generic.IEnumerable(Of S).GetEnumerator

Return New Enumerator4Select(collection.GetEnumerator(), selector)

End Function

Public Function GetEnumerator1() As System.Collections.IEnumerator Implements System.Collections.IEnumerable.GetEnumerator

Return GetEnumerator()

End Function

Class Enumerator4Select

Implements IEnumerator(Of S)

Private collection As IEnumerator(Of T)

Private selector As Func(Of T, S)

Public Sub New(ByVal coll As IEnumerator(Of T), ByVal func As Func(Of T, S))

collection = coll

selector = func

End Sub

Public ReadOnly Property Current() As S Implements System.Collections.Generic.IEnumerator(Of S).Current

Get

Return selector(collection.Current)

End Get

End Property

Public ReadOnly Property Current1() As Object Implements System.Collections.IEnumerator.Current

Get

Return Current

End Get

End Property

Public Function MoveNext() As Boolean Implements System.Collections.IEnumerator.MoveNext

Return collection.MoveNext()

End Function

Public Sub Reset() Implements System.Collections.IEnumerator.Reset

collection.Reset()

End Sub

Private disposedValue As Boolean = False ' To detect redundant calls

' IDisposable

Protected Overridable Sub Dispose(ByVal disposing As Boolean)

If Not Me.disposedValue Then

If disposing Then

' TODO: free managed resources when explicitly called

End If

' TODO: free shared unmanaged resources

End If

Me.disposedValue = True

End Sub

#Region " IDisposable Support "

' This code added by Visual Basic to correctly implement the disposable pattern.

Public Sub Dispose() Implements IDisposable.Dispose

' Do not change this code. Put cleanup code in Dispose(ByVal disposing As Boolean) above.

Dispose(True)

GC.SuppressFinalize(Me)

End Sub

#End Region

End Class

Here is some code to test the above sample:

Module Module1

Sub Main()

Dim col As New EnumerableQueryable(Of Integer)(New Integer() {1, 2, 3})

Dim q = From i In col Select i_sqr = i * i Select i_sqr + 1

For Each i In q

Console.WriteLine(i)

End Sub

End Module

Scalar queryable example ( Query Design continued ):

VladimirSadov — Sat, 09 Jun 2007 05:31:00 GMT

This post is a part of a series of posts about query design. For the previous post see: http://blogs.msdn.com/vladimirsadov/archive/2007/06/08/generic-queryable-example-query-design-continued.aspx

Here is one more example of a queryable type. While weird and fairly useless it illustrates that queryable type does not have to be a collection. This one represents a single scalar value.

Public Delegate Function SelectorDelegate(Of T, S)(ByVal arg As T) As S

Class ScalarQueryable(Of T)

Public element As T

Public Function [Select](Of S)(ByVal projection As SelectorDelegate(Of T, S)) As ScalarQueryable(Of S)

Dim new_col As New ScalarQueryable(Of S)

Dim new_element As S

new_col.element = new_element

new_col.element = projection(element)

Return new_col

End Function

End Class

Module Module1

Sub Main()

Dim col As New ScalarQueryable(Of Integer)

col.element = 1

'note that the the result of the query is

'another ScalarQueryable that contains a string

Dim q = From i In col Select i_sqr = i * i

Console.WriteLine(q.element)

End Sub

End Module

Generic queryable example ( Query Design continued ):

VladimirSadov — Sat, 09 Jun 2007 01:15:00 GMT

Here is another example of a queryable collection. In this case we have a collection that is generic so it can be used with elements of different types. What is more interesting is that MyGenericQueryable can be used with query projections that change type of the element.

Public Delegate Function SelectorDelegate(Of T, S)(ByVal arg As T) As S

Class MyGenericQueryable(Of T)

Public elements() As T

Public Function [Select](Of S)(ByVal projection As SelectorDelegate(Of T, S)) As MyGenericQueryable(Of S)

Dim new_col As New MyGenericQueryable(Of S)

Dim new_elements(elements.Length - 1) As S

new_col.elements = new_elements

For I As Integer = 0 To elements.Length - 1

new_col.elements(I) = projection(elements(I))

Return new_col

End Function

End Class

Module Module1

Sub Main()

Dim arr As Integer() = New Integer() {1, 2, 3}

Dim col As New MyGenericQueryable(Of Integer)

col.elements = arr

'note that the projection produces an element of a different type

'while col contains integer elements, q contains strings

Dim q = From i In col Select i_string = i.ToString

For Each t In q.elements

Console.WriteLine(t)

End Sub

End Module

Query pattern, Composability ( Query Design continued ):

VladimirSadov — Fri, 08 Jun 2007 04:40:00 GMT

This post is a part of a series of posts about query design. For the previous post see: http://blogs.msdn.com/vladimirsadov/archive/2007/02/19/polymorphic-selector-and-extension-methods-query-design-continued.aspx

Here I want to summarize what exactly the minimum requirements are for a type to be considered queryable with some explanations why we have such rules.

Quaryable type requirements:

1. There must be a method called Select that can be called with the collection instance. Note that this can be either an instance method or an extension method. As long as you can make a call like col_instance.Select(..) it is acceptable as a select operator.

Why: by having a proper Select method a type indicates that it can be used in query expressions. It is also the method that represents “Select” in the query syntax.

Dim result = From el in col_inst Select el.ToString
translates into:
Dim result = col_inst.Select(...)

2. Select must take exactly one argument. The argument must be of a function delegate type that takes one argument.

Why: The argument must be a delegate because compiler will generate a function that takes one argument and will pass this function to the Select method in form of a delegate.

3. While the selector delegate can be (and often is) a generic type, the delegate’s argument type (let’s call it T) must be completely inferable from the collection type.

Why:T must be known because it indicates the type of collection elements. Since the collection is completely in charge of iterating itself we need some way to know what type its elements have. In the above example T would be the type of el. Knowing the type of iteration variable compiler can verify that projection expression (el.ToString in the above example) makes sense. T is also used to provide intellisense when you type “el.” .

Example of very simple queryable type:

Public Delegate Function SelectorDelegate(ByVal arg As Integer) As Integer

Class MyQueryable

Public elements() As Integer

Public Function [Select](ByVal projection As SelectorDelegate) As MyQueryable

Dim new_col As New MyQueryable

new_col.elements = elements.Clone

For I As Integer = 0 To elements.Length - 1

new_col.elements(I) = projection(elements(I))

Return new_col

End Function

End Class

You can test this queryable type with code like this:

Module Module1

Sub Main()

Dim arr As Integer() = New Integer() {1, 2, 3}

Dim col As New MyQueryable()

col.elements = arr

Dim q = From I In col Select square = I * I

For Each t In q.elements

Console.WriteLine(t)

End Sub

End Module

Output:

1,4,9

Note that result of Select in our example is MyCollection that is a queryable type. This is not a requirement of queryable pattern. Select could return any type it wants. However returning a queryable type is very convenient as you can “compose” query by chaining Select operators:

This is a valid expression in VB. The second Select will be applied to the result of the first select

Dim q1 = From I In col Select square = I * I Select sq_2 = square * 2

If you replace the query in the previous example with the new query, you will get output:

2,8,18

Polymorphic selector and Extension methods ( Query Design continued ):

VladimirSadov — Tue, 20 Feb 2007 02:45:00 GMT

This post is a part of a series of posts about query design. For the previous post see: http://blogs.msdn.com/vladimirsadov/archive/2007/02/05/query-expression-query-design-continued.aspx

So far we have followed design decisions for the language integrated queries to the point where user specifies the query in the following form.

Dim num_squares = From cur_num In numbers Select result = cur_num * cur_num

Compiler transforms this into code that iterates over the source collection (numbers in this case), applies user specified expression to every element and produces the result.

If you think about universal applicability of this pattern you may notice that there are different kinds of collections. How can compiler know how to iterate over those? One possible solution is to handle only known kinds of collections (like those that implement IEnumerable(of T) ). We certainly know how to iterate such collections, but requiring that all queryable collections implement IEnumerable(of T) seems to be overly restrictive. Also it seems that producing the result should also be type-specific as we would expect the result of the query to be of the same kind as the source collection. For example if we are making a query on some kind of Set type or a Data Table we would expect to get back another Set or Data Table.

One of the object oriented design principles tells that these kinds of type specific operations should be handled polymorphic ally by the types themselves. In this case the queryable types could all derive from a common ancestor and override functions that we need to do a query. That would certainly solve the problem, but it would still require all queryable types to share some common ancestor which is too restrictive and does not work for most of the types that already exist. We really want to be able to do queries on unrelated types to make queries truly extensible.

A good example of solving similar problem is the “For Each” operator. In addition to IEnumerable types “For Each” can handle types that implement “collection pattern”. The pattern requires that type has “GetIterator” function with appropriate signature and compiler goes from there by using this function. It seems to be a good idea to have some kind of queryable pattern that compiler can recognize and use for query translation and that is exactly how VB queries are implemented:

When compiling a query VB compiler looks for a function called “Select” with a signature that can accept a representation of the user expression. It is the “Select” function that does all the work of iterating over the source collection, applying the expression and collecting results into the result collection.

Dim num_squares = From cur_num In numbers Select result = cur_num * cur_num

Becomes:

Dim num_squares = numbers.Select(AddressOf VB_ CompilerGeneratedFunction)

' =====================================

' synthetic function that performs actions specified in user expression

'======================================

Function CompilerGeneratedFunction(ByVal cur_num As Integer) As Integer

Dim result = cur_num * cur_num

Return result

End Function

Now that we have a well defined way to detect and use queryable collections in queries, we just need to figure how to add these missing “Select” functions to various types. And surely enough in Orcas there is a mechanism to do this – it is called “Extension methods”. Without going into details it makes possible to declare a shared method in such a way that it could be called as an instance function of its first argument. Example:

User defines:

Shared Function Select(target as MyCollection, selector as SomeDelegate) as MyCollection

The function can be used as an instance function of MyCollection:

Dim Col1 as New MyCollection

‘ this should work as long as selector_del is of type SomeDelegate

Col1.Select(selector_del)

To make this even better one of the part of the LINQ technology that will ship with VB9 are extensive libraries of extension methods that do all kind of query operations for common types such as IEnumerable(of T), arrays or Xml data. These implementations (also called query operators) are also very type agnostic through usage of generics so they should handle most if not all the scenarios that query user may ever require.

The result is that when we combine pattern matching for queryable types in the compiler and standard implementations of this pattern in LINQ libraries, it becomes a complete solution that allows users to apply language integrated queries to common collection types right away.

The pattern matching mechanism is also extremely flexible and extensible. If somebody wants to make some type queryable or to override an already provided implementation with a more specific one, it is very easy to do. One can simply add a Select function to the type. In fact through usage of Extension Methods the types do not really need to be modified to become queryable as you can provide Select that is an extension method.

Query expression ( Query Design continued ):

VladimirSadov — Mon, 05 Feb 2007 23:02:00 GMT

This post is a part of a series of posts about query design. For the previous post see: http://blogs.msdn.com/vladimirsadov/archive/2007/01/30/representing-the-user-code-by-delegates-query-design-continued.aspx

In the previous posts we have designed a very handy function that can iterate any IEnumerable(of T) and apply given expression to all elements, but the disadvantages are obvious – more code to type. We tried to reduce the amount of actual code that user writes, but now we require that:

1) User code is placed into a function.

2) Instance of a delegate representing user code needs to be created and passed into the function.

3) User needs to specify type arguments for the delegate

The good part is that these actions are extremely mechanical. They are so mechanical, that they can be done by the compiler itself. We just need a new language construct that will take only essential information from the user. The essential information that is needed here is the source collection and the expression that is applied to elements. In addition to this we need to specify the name for the current element so that we could refer to it inside the expression.

Dim result = <KEYWORD> current_number <SEPARATOR> numbers <SEPARATOR> expression

Interestingly enough this is very similar to SELECT statement in SQL. This is not a coincidence though as the SQL query basically performs the same action of projecting elements of one table into another using user specified expression.

SELECT expression FROM current_number IN numbers

So it should not be surprising that VB syntax for query expressions uses same keywords as in SQL. In fact the initial syntax for VB queries was almost identical to SQL. However it was discovered that there are many advantages such as better intellisense when iteration variable (current_number) is already declared by the time when user starts typing the expression. As a result the syntax was revised and From clause was brought to the beginning of the query expression.

Example:

' query that gets number names

Dim num_names = From cur_num In numbers Select result = cur_num.ToString

The code clearly describes what we want – we want to get all elements in “numbers” collection and generate a sequence of “result” values that are produced by applying “ToString()” function to each element. Based on the query, compiler can extract the expression that we specified in Select clause and generate a synthetic function that performs this action.

' translation of the query

Dim num_names = IterateAndApply(numbers, AddressOf CompilerGeneratedFunction)

' =====================================

' synthetic function that is automatically generated by the compiler

' it performs the actions specified in Select clause

'======================================

Function CompilerGeneratedFunction(ByVal cur_num As Integer) As String

Dim result = cur_num.ToString

Return result

End Function

This is much better. It takes only one line to produce a sequence of number names. The rest of the code is either compiler generated or in a library. In order to fully appreciate the convenience, let’s consider another example. What if instead of strings we need to produce a sequence of squares?

Code:

Dim num_squares = From cur_num In numbers Select result = cur_num * cur_num

This is it!! This is all the code that you need to write!!!

Compiler can take care of generating a function that will perform the Select expression (also called projection). The rest of what is needed (iterating the source collection, applying the projection and forming the result sequence) did not really change and the same library functions can be reused.

No more duplication of code. No more off-by-one bugs and out-of-range exceptions. All we need is to make sure that the projection does what we wanted. Reusability rules!!!

Representing the user code by delegates ( Query Design continued ):

VladimirSadov — Tue, 30 Jan 2007 23:55:00 GMT

This post is a part of a series of posts about query design. For the previous post see: http://blogs.msdn.com/vladimirsadov/archive/2007/01/22/generic-implementation-of-iterateandapply-query-design-continued.aspx

Lets’ now consider the implementation of Apply function. In theory we could somehow interpret the string inside the Apply function, but having code in a form of string still causes problems. Consider the following example:

Dim number_names As New List(Of Integer)

number_names = IterateAndApply(numbers, _

"squares.Add(current_number.ToString())")

Take a look at the expression. While the code seems fine, our Apply function expects that the resulting type of the expression matches the type of the collection elements. This particular expression takes Integer and returns String.

We can add another type argument to Apply function to represent the return type of the expression:

Function Apply(Of T, S)(ByVal item As T, ByVal code As String) As S

However this just replaces one problem with another. Someone somewhere must specify S. Generic type argument T can be inferred from the function arguments (in particular from “item”). This is not the case for S. Since “code” is just a string, there is no way of knowing the type of the results of this expression. In fact there is no way to know at compile time that the expressions returns anything or even runs.

Consider this example

Dim number_names As New List(Of Integer)

number_names = IterateAndApply(numbers, "BLAH BLAH BLAH")

This is not going to run, but compiler has no idea.

Clearly a string is a very bad representation of user code. Besides the problems with interpreting it provides no information on what this code returns or whether this code will work.

Fortunately there is a way to write a piece of code and describe its arguments and return type. For that we need to put the code in a separate function and pass a reference to that function into the IterateAndApply as a delegate:

Dim numbers As New List(Of Integer)

numbers.Add(42)

Dim number_names As New List(Of Integer)

Dim MyCodeRef As Func(Of Integer, String) = AddressOf MyCode

number_names = IterateAndApply(numbers, MyCodeRef)

Function MyCode(ByVal current_number As Integer) As String

Return current_number.ToString

End Function

' =====================================

' implementation of IterateAndApply

'======================================

'delegate type to represent user expressions

' T is the type of argument

' S is the type of the result

Delegate Function Func(Of T, S)(ByVal arg As T) As S

' source - the collection that is being iterated

' code - reference to user supplied code to apply to each element

' returns a collection of results when user's code is applied to

' all lements.

Function IterateAndApply(Of T, S)(ByVal source As IEnumerable(Of T), _

ByVal code As Func(Of T, S) _
) As IEnumerable(Of S)

Dim result As New List(Of S)

'iterate the source

For Each item In source

'apply the code that user has provided to each element

result.Add(Apply(item, code))

Return result

End Function

'function that applies given code to the given object and returns
'the result

Function Apply(Of T, S)(ByVal item As T, ByVal code As Func(Of T, S)) As S

Return code.Invoke(item)

End Function

The code above solves a lot of problems. First of all there is no need for “magic” to apply the user code. In fact Apply function becomes so trivial, there is no need to even have it. The inner loop of IterateAd Apply could simply look like this:

For Each item In source

'apply the code that user has provided to each element

result.Add(code.Invoke(item))

Secondly, because user code is contained in a regular function and expressed as regular code, existing checks in the compiler can verify the syntax, type safety and everything else that can be verified at compile time.

Generic implementation of IterateAndApply ( Query Design continued):

VladimirSadov — Mon, 22 Jan 2007 21:31:00 GMT

This post is a part of a series of posts about query design. For the previous post see: http://blogs.msdn.com/vladimirsadov/archive/2007/01/14/reusability-is-the-reason-to-have-queries.aspx

So far we have a pretty good implementation of IterateAndApply, but there are several issues with the code.

1) We are resorting to magic function ApplyCode.

2) IterateAndApply works with any list of integers. This is good, but what about lists of Doubles or a Queue or an Array?

Ok, #1 is a serious matter and we will just assume for now that magic method “just exists”. However supporting different types seems to be fairly easy – IterateAndApply can be generic function.

' source - the collection that is being iterated

' code - user supplied code to apply to each element

' returns a collection of results when user's code is applied to all

' elements.

Function IterateAndApply(Of T)(ByVal source As IEnumerable(Of T), _

ByVal code As String) As IEnumerable(Of T)

Dim result As New List(Of T)

'iterate the source

For Each item In source

'apply the code that user has provided to each element

result.Add(Apply(item, code))

Return result

End Function

'hypothetical function that applies given code to the given object and
'returns the result

Function Apply(Of T)(ByVal item As T, _

ByVal code As String) As T

'todo: code gets “magically” applied to the item and result is returned

End Function

The code above should handle all kinds of element types. In fact this function can iterate any collection as long as it implements IEnumerable(of T) . Upon execution of IterateAndApply and Apply the type of T will be inferred from the type of the source collection.

Reusability is the reason to have Queries.

VladimirSadov — Sun, 14 Jan 2007 21:36:00 GMT

Programmers do not like writing the same code over and over again. There are many reasons for this. First of all it is not productive. Secondly it is often hard to maintain as similar code pieces often require same changes. Copying code within a big project is plain evil. This is the primary way of creating those annoying bugs that just do not want to get away – you fix them here, then there, then you find them somewhere again…

There are many ways by which languages can help with code reuse already – procedural programming, Object Oriented Programming etc… However there are still too many cases where you have to write code like this:

Dim numbers As New List(Of Integer)

numbers.Add(42)

Dim squares As New List(Of Integer)

For Each current_number As String In numbers

squares.Add(current_number* current_number)

Next

As you can see this is a very simple pattern where you iterate over a collection and perform some action with each element. The only “smart” part of this code is the expression inside the loop; the rest is just the way to get elements one-by-one and to apply the expression. You may already intuitively feel that it would be great if you could write the iterating part once and then just supply the “smart” part of the loop while reusing “iterating” part

squares = IterateAndApply(numbers, _

"squares.Add(current_number * current_number)")

This code looks way simpler!!!
The hypothetical function IterateAndApply does the iterating, applies the expression to the elements, collects results and returns the resulting collection.

Obviously somebody has to implement IterateAndApply, but it can be implemented only once and called everywhere.

A possible implementation of IterateAndApply could look like this:

' source - the collection that is being iterated

' code - user supplied code to apply to each element

' returns a collection of results when user's code is applied

' to all elements.

Function IterateAndApply(ByVal source As List(Of Integer), _

ByVal code As String) As List(Of Integer)

Dim result As New List(Of Integer)

'iterate the source

For Each item In source

'apply the code that user has provided to each element

result.Add(Apply(item, code))

Return result

End Function

' hypothetical function that applies given code to the given object

' and returns the result

Function ApplyCode(ByVal item As Integer, _

ByVal code As String) As Integer

'todo: code gets “magically” applied to the item and result is returned

End Function

You may notice that the resulting code is somewhat bigger than For Each loop. However since this code is going to stay in one instance the overall number of lines in the project could be drastically reduced if you use this function a lot. And as we know writing less code is good for so many reasons.

How to do a Join in VB.

VladimirSadov — Mon, 02 Oct 2006 05:41:00 GMT

Well, if you are using May CTP, then there is only one kind of Join operations that you can do in VB - cross-join.

The syntax for performing cross-join of two collections follows:

Dim arr1 = New Integer() {1, 2, 3}
Dim arr2 = New String() {"Hi", "Bye"}

Dim q = From i In arr1, j In arr2 Select i, j

For Each element In q
Console.WriteLine(" i = " & element.i & " j = " & element.j)
Next

===== Expected output:

i = 1 j = Hi
i = 1 j = Bye
i = 2 j = Hi
i = 2 j = Bye
i = 3 j = Hi
i = 3 j = Bye

Can I put CDATA inside a CDATA node?

VladimirSadov — Fri, 26 May 2006 19:38:00 GMT

Well. The answer is no. CDATAs cannot be nested in general. The nested CDATA terminating sequence will terminate the outer CDATA. However there is a trick where you can put CDATA in two consequitive CDATA sections. It works by splitting the terminating sequence in two halves.

There is one caviat though - VB 9.0 does not allow a sequence of more than one CDATA as a standalone object (one is fine though). You will have to put this sequence into an element <a> ... </a> or a fragment <> ... </> .

Here is an example (element is used):

Module Module1

Sub Main()

Dim c1 = _

Dim c2 = _

Console.WriteLine(c1.Value)

Console.WriteLine(c2.Value)

End Sub

End Module

======== Output:

<![CDATA[some cdata here]]>
<![CDATA[some cdata here]]>

Factorial in VB 9.0 using closures.

VladimirSadov — Tue, 23 May 2006 05:14:00 GMT

You cannot declare a closure directly in VB right now, however in the last CTP verion of VB 9.0 you can create some closures indirectly using quries syntax.

The following is by no means is an effective implementation of factorial function :-) , but it should be an interesting use case of closures.

Module Module1

Sub Main()

'create a closure

Dim factorial_closure = fac(10)

'get result (most of the calculations will happen here)

Dim result = factorial_closure(0)

Console.WriteLine(result)

End Sub

Function fac(ByVal x As Integer) As IEnumerable(Of Integer)

' this function roughly means this:

'fun fac 0 = 1

' | fac n = n * fac(n-1);

If x = 0 Then Return {1}

Return From dummy In {Nothing} _

Select x * fac(x - 1)(0)

End Function

End Module

Another way to find duplicates in two strings.

VladimirSadov — Sat, 22 Apr 2006 06:47:00 GMT

I think the LINQ way is a way more expressive compared to the loops although conceptually they do the same thing.

Function LINQ(ByVal arr1 As String(), ByVal arr2 As String) As String()

Dim duplicates = From s0 In arr1, s1 In arr2 _

Where s0 = s1 _

Select s0

Return duplicates.ToArray()

End Function

Function Loops(ByVal arr1 As String(), ByVal arr2 As String()) As String()

Dim l As New List(Of String)

For i As Integer = 0 To UBound(arr1)

For j As Integer = 0 To UBound(arr2)

If arr1(i) = arr2(j) Then l.Add(arr1(i))

Return l.ToArray()

End Function

Have you ever said "not today"?

VladimirSadov — Wed, 19 Apr 2006 20:19:00 GMT

Well, that is how you may have no posts for long time.... :-)

Running V1.1 and V1.0 apps in 64bit mode.

VladimirSadov — Fri, 24 Jun 2005 07:18:00 GMT

As you may noticed from my previous post loader treats v1.0 and v1.1 managed executables as 32bit only regardless of the ILONLY flag. The main reason for this is that before v2.0 there was no way to express that an executeble has or does not have dependencies on a particular platform. However everything was supposed to run on 32bit. So, to be on the safe side loader on 64bit machines treats pre-v2.0 managed executables as 32bit only. Basically if you do not have dependencies on 32bit components such as COM32 libraries or native dlls, the code could potentially run in 64bit mode, but loader cannot know that.

There is one interesting observation - v2.0 runtime is backward-compatible with previous versions and you can load and use v1.0 and v1.1 assemblies from a v2.0 assembly. Also the decision on whether to run as 32bit or 64bit is done only once per process based on the format of the entry point executable. As a result you can create a simple wrapper/launcher in v2.0 which will run natively and in turn load and execute pre-2.0 executables. By the time they are loaded the process will be already running as 64bit.

NOTE. This is basically a way to get around loader safekeeping so should be used for demonstration only.
Also this will obviously work only as long as your pre V2.0 code is not using 32bit dependencies.

'compile this as an "anycpu" application, say Launcher.exe
'run this as " Launcher.exe my_V1_1_foo.exe "
Module Module1
   Sub Main()
        'get the name of the executable
         Dim exe_path As String = My.Application.CommandLineArgs(0)
         'find the entry point of the executeble
         Dim entry_point As Reflection.MethodInfo = Reflection.Assembly.LoadFile(exe_path).EntryPoint
         'run it!!
         entry_point.Invoke(Nothing, New Object() {})
   End Sub
End Module

Loading platform neutral (anycpu) executables on 64bit machine.

VladimirSadov — Wed, 22 Jun 2005 03:23:00 GMT

Just like other Windows executable files Managed code is stored in PE format. To be more precise it can use PE (32bit) or PE+ (64bit) formats.

In unmanaged world the format of the executable tells the loader whether to run the image as 64 bit or 32 bit process. However this is not always true for managed code. Managed code can be platform neutral and be able to run as either a 32bit process or a 64bit process. Obviously neutral code has to be stored in PE (32bit) format so that it could be loaded on 32bit OSes. So what happens when we run such executable on a 64bit machine?

When windows loader loads an executable image it checks for managed modules by inspecting COFF header of the executable. If managed code is detected, then IMAGE_COR20_HEADER is examined. The structure is stored at location indicated by IMAGE_DIRECTORY_ENTRY_COMHEADER and describes the managed executable. For the loader the most interesting part of the header structure is Flags field and in particular ILONLY flag. ILONLY indicates that the executable contains pure IL code and can run either as 32bit or as a 64bit process. On 64bit machines ILONLY flag causes the loader to load the image as 64bit and invoke 64bit CLR. As the last touch upon loading 64bit CLR transforms loaded PE header into PE+ format on the fly to keep everything right.

This is not a complete process though - In addition to Flags loader checks the version of the IMAGE_COR20_HEADER. Anything less than 2.5 indicates CLR1.0 or CLR1.1 executable. For compatibility reasons such executables will run in 32bit mode.

Implementing Using from simpler parts.

VladimirSadov — Fri, 10 Jun 2005 06:16:00 GMT

Using is a very convenient language element. When I have to write something that should be compilable by older version of the compiler, "Using" is one of the new features that I really miss. Here is an example how to achieve the same behavior without Using:

Sub foo1()
   Using f As New IO.StreamWriter("somefile.txt")
      f.WriteLine("hello")
   End Using
End Sub

Sub foo2()
   Dim f As New IO.StreamWriter("somefile.txt")
   Try
      f.WriteLine("hello")
   Finally
      'assume that the code inside Try may assign Nothing to f,
      'but in most cases check for Nothing is not necessary
      If f IsNot Nothing Then f.Dispose()
   End Try

End Sub

If you compile functions above and check IL code produced you will find that it is identical:

.method public static void foo1() cil managed   // foo2 is identical
{
// Code size       35 (0x23)
.maxstack 2
.locals init ([0] class [mscorlib]System.IO.StreamWriter f)
IL_0000: ldstr      "somefile.txt"
IL_0005: newobj     instance void [mscorlib]System.IO.StreamWriter::.ctor(string)
IL_000a: stloc.0
.try
{
    IL_000b: ldloc.0
    IL_000c: ldstr      "hello"
    IL_0011: callvirt   instance void [mscorlib]System.IO.TextWriter::WriteLine(string)
    IL_0016: leave.s    IL_0022
} // end .try
finally
{
    IL_0018: ldloc.0
    IL_0019: brfalse.s IL_0021
    IL_001b: ldloc.0
    IL_001c: callvirt   instance void [mscorlib]System.IDisposable::Dispose()
    IL_0021: endfinally
} // end handler
IL_0022: ret
} // end of method Module1::foo1

On this simple level it seems clear how Using works and the feature itself seems faily simple - just syntactical sugar for a Try/Finally block. However all the convenience of Using is appreciated when you have to nest Using or use several Using blocks in the same method. Suddenly Using is a clear winner over the Try/Finally implementation in terms of lines of code, better scoping and readability.

Just compare these two methods (they do the same thing):

Sub foo3()
   Using reader As New IO.StreamReader("f1.txt"), writer As New IO.StreamWriter("f2.txt")
      writer.WriteLine(reader.ReadLine)
   End Using
End Sub

Sub foo4()
   Dim reader As New IO.StreamReader("f1.txt")
   Try
      Dim writer As New IO.StreamWriter("f2.txt")
         Try
            writer.WriteLine(reader.ReadLine)
         Finally
            If writer IsNot Nothing Then writer.Dispose()
         End Try
   Finally
      If reader IsNot Nothing Then reader.Dispose()
   End Try
End Sub

Example for my previous post

VladimirSadov — Fri, 10 Jun 2005 05:01:00 GMT

So here is an example how you can create a Nullable(Of Nullable ...)) type via indirection if Nullable were allowed to be passed to an As Structure type argument:

'in ClassLibrary1.dll you have this:
Public Class cls1(Of T As Structure)
Function foo() As Nullable(Of T)
End Function
End Class

'in Program1.exe you reference ClassLibrary1.cls and do this:
Module Module1
   Sub Main()
      Dim var1 As New cls1(Of Nullable(Of Integer))   '<=== nothing seems to be wrong here
'....... untill you do this call:
      var1.foo()     '<==== this would return Nullable(of Nullable(Of Integer)) !!!!!!!!
   End Sub
End Module

Why Nullable cannot be passed to an “As Structure” type parameter.

VladimirSadov — Sat, 04 Jun 2005 06:10:00 GMT

Here is an interesting experiment:

Class cls1(Of T As Structure)

End Class

Module Module1

Sub Main()

Dim v As cls1(Of Nullable(Of Integer))

End Sub

End Module

We all know that Nullable(Of T) is a structure. However the code above gives a compile error:

'System.Nullable' does not satisfy the 'Structure' constraint for type parameter 'T'. Only non-nullable 'Structure' types are allowed.

It seems that Nullable is explicitely not allowed to be passed to a type parameter constrained to be a value type. Why? The reason is pretty simple. The purpose of Nullable is to add additional “No Value” state to value types. However it would be weird to allow to do this more than once.
Consider a value type Boolean – it can have 2 possible states:

True, False

Nullable(Of Boolean) will have 3 states:

True, False, “No Value”

How many states would Nullable(of Nullable(of Boolean)) have? Well, since Nullable adds another “No Value” state it would have 3 states:

True, False, “No Value”, “No Value”

Now this is very weird. Type with multiple “No Value” states seems just plain wrong. (Also consider cases with Nullable used more than once).

Hence it should be invalid to pass Nullable as a type argument to Nullable. Even more – to guarantee that we never get into Nullable(Of Nullable …) situation indirectly, compiler has to reject passing Nullable to any “As Structure” type parameter.

So, after all, can 32bit code call into 64bit code?

VladimirSadov — Sat, 22 Jan 2005 02:55:00 GMT

Q: A 32bit app, can run on a 64bit OS, so it must be able to access 64bit, right? So can a 32bit app access my 64bit library component/library/dll. Should I test this scenario?

Short answer is “most likely no”, the long answer is:

Communication between 32bit and 64bit code is a complicated matter. Consider a case where a function from 64bit foo.dll returns a 64bit pointer or a handle that is not even representable in a 32bit number. How can this be safely passed to/from 32bit code? To avoid these problems OS prevents 32bit and 64bit code from even being loaded in the same process. The only exception are the 64bit dlls that are required by WOW64 ( Wow64.dll, Wow64win.dll, 64bit version of Ntdll.dll and Wow64cpu.dll on Itanium).

COM and .Net add more layers on top of this, but the idea is the same – 32bit and 64bit user code cannot be mixed in the same app. Note that you still can communicate between 32bit and 64bit code in different processes (via various IPCs, DCOM, remoting, sockets etc).

Indeed When 32bit code makes a system call (CreateFile, VirtualAlloc etc), your call will eventually end up in 64bit code because the OS is 64bit, but before that it will be intercepted by WOW64 layer and translated into a corresponding 64bit system call. WOW64 will take care of pointer and handle translation. So if your code is a kernel driver, then yes, it could be accessed by a 32bit app indirectly (via WOW64).

When 32bit code tries to call into some arbitrary 64bit code (3^rd party 64bit dll, COM32 object etc) it will not be able to do this. WOW64 cannot help here since it does not know how to deal with 3^rd party components. The loader will not even load a 64bit dll in the address space of a 32bit process.

64bit Framework documentation on MSDN.

VladimirSadov — Mon, 10 Jan 2005 03:55:00 GMT

MSDN has started adding documentation about .Net on 64bit platforms. Here is the link: http://msdn.microsoft.com/netframework/programming/64bit/default.aspx

Do not say you have not seen it. :-)

Why CLR is two-in-one on 64bit platform

VladimirSadov — Fri, 01 Oct 2004 21:29:00 GMT

As I mentioned in my previous post when we install .Net framework on a 64bit machine we put both 32 bit and 64 bit runtimes on it. A reasonable question is “Why do we need 32 bit runtime on a 64 bit machine?”. The answer is “To enable your .Net app to run as a 32 bit process on a 64 bit platform”.

One of important features of 64bit platforms is ability to execute both native 64 bit code and 32 bit x86 code. The problem is that 32 bit and 64 bit native code cannot be mixed within the same process. There are many conceptual and technical problems that make this impossible or not worth doing. As a result if you have a component which is 32bit and has to be loaded into the client’s address space, the whole process must be running as 32bit process as well. This scenario is relatively common and includes 32bit dlls, COM32 inproc servers etc. Obviously unmanaged applications are never impacted by this feature as they are precompiled for a particular architecture so no client-server bitness mismatch can occur.

Now imagine the following scenario. There is some very cool .Net application that uses some COM32 component that does not have 64bit counterpart (note that most platform components have both 32bit and 64bit implementation on 64bit OS). It would be just wrong if your .Net app would be jit-compiled into 64bit code and loaded as a 64bit process as it would not be able to use the 32bit component that it needs. Ooops! We need x86 CLR here.

To resolve this situation both 32 bit and 64 bit .Net runtimes are installed at the same time and there is a way to tell the OS process loader whether a managed application requires loading in a particular way. If you use VB, then you can use /platform switch to tag your executables as requiring a particular platform.

Passed to compiler	Exe runs as:
/platform:x86	Always runs as a 32bit process. Even when run on a 64 bit OS. Can use 32 bit resources
/platform:x64	Can run on x64 architectures only. Runs as a 64 bit process.
/platform:Itanium	Can run on ia64 architectures only. Runs as a 64 bit process.
/platform:anycpu	Such code can run on either x86, x64 or ia64 architectures and therefore should not have architecture specific dependencies. It will run as a 32 bit process on x86 and as a 64 bit process on ia64 and x64.
No platform was specified	Same as /platform:anycpu