Welcome to MSDN Blogs Sign in | Join | Help

Visual Debugger for MSBuild Projects

Folks, Hi!

A recent feature poll on the MSBuild blog seems to indicate that a debugger for MSBuild projects is the #1 vNext request from you. Lot of other folks (such as here and here) have explicitly blogged about this requirement.

Over the last week I put together a Visual Debugger for MSBuild Projects. The v1 alpha feature list looks like:

v1 Feature Set:

  • Load project and arbitrary target files
  • Set/View build environment
  • Step-Into tasks and targets (F11), Run (F5), Break execution, Step into imported projects, step into CallTarget & MSBuild tasks, task & target batching
  • Set breakpoints on tasks and targets, delete/view breakpoints, locate breakpoints in code
  • View call stacks (esp. useful for stepping into CallTargets & MSBuild tasks), locate stack frame in code
  • View debugger messages, MSBuild traces in Output window
  • RO View of evaluated Items and Properties in break mode
  • Handles out-of-box VS2008 projects and complex Microsoft's Developer Divisions's MSBuild based builds
  • Requires only .NET Framework 3.5 RTM (no VS2008 required)

I have also uploaded the binaries and how-to-get-started instructions on http://www.codeplex.com/msbdbg.

Please head over there to give this a spin. I have also listed there the known issues, the v2 plans, and other details there.

I am looking forward to listening to you on the right, wrong, the good, bad, the ugly and anything else that you will care to tell me about.

I am hoping this will go some distance it helping everyone get away from printf debugging and get into full fledged debugging for MSBuild projects. Esp. considering MSBuild projects files are no longer used only for builds. (Why, my wife has even filed a patent for using MSBuild as a test harness)

Here are a few screen shots:

MSBuild Debugger Saying 'hello'...

opendbg  

Set Breakpoint and Select Target to Debug...
setbpandselecttargettodebug
Breakpoint Hit and Output Window...
bphitandoutputwindow
Break at Arbitrary Location, See Call Stack and Properties...
breakNcallstackNproperties
Step-In, See Items and Active Breakpoints...

stepinNitemsNbreakpoints

Posted by parthopdas | 10 Comments

Model-View-Controller Design Pattern...

This post is really to update my blog so that is doesn't get deleted due to massive levels of inactivity.

I am also experimenting with Windows Live Writer - it totally kicks ass!

Windows Liver Writer

Anyways getting back to the point...

This particular diagram conveys pretty much everything about M-V-C:

Model-View-Controller

 

Some additional links on M-V-C (Yes, yes... Google can give this information - but I am trying to save my blog really)

See you later...

Posted by parthopdas | 1 Comments

MEDC India 2006, Bangalore

Hi Folks...

The 1st ever Microsoft Mobile & Embedded Developer Conference in India – MEDC-India 2006 – was held at Bangalore’s Grand Ashok Hotel during the 4th-6th April 2006.

Aarthi and I delivered a session on Creating Custom Controls For Managed Code With Designtime Support In Visual Studio 2005, targeted at folks indulging in custom control development on .NET CF.

We enjoyed delivering the session and the response from you guys was awesome. It was really nice talking to you guys. Thank you for all the feedback. We are looking forward to meeting you again and please keep the feedback coming in. Please also feel free to contact me at parthad at microsoft dot com if you need any help from us.

In case you want the session material, it is attached (MOB301 - MEDC2006.zip). It contains the slides as well as the demos and walkthroughs.

On a lighter note, we did take a couple of photos. Here is one of them:

Guys from VSD team @ MEDC-India 2006 

The guy closest to the camera is Anand. The 13 year old kid is Aarthi. Partho is on her left. Sriram on her right. Varada on Sriram's right. On Partho's left is Amit and on his right is Gopinath.

Oh another thing... We met the great Doug Boling. The guys is simply awesome - delivering 5 back-2-back sessions in a single day - all hardcode WinCE kernel related! Manav and I had beer with him and talked about everything from why Hillary Clinton would/wouldn't become the President to what sux in Visual Studio 2005!

Thats all for now... See you in a while!


Posted by parthopdas | 7 Comments
Attachment(s): MOB301 - MEDC2006.zip

Destination SQL Server 2005 (Mobile Edition)

Hi Folks...

The title of this post may make it sound like a movie. But it ain't. This is a contemporary post noir set on the devices of a gritty, yet colorful SQL Server neighborhood... :D (That was gross, I concur! Hopefully the meat of the post will have some substance and better humor than its opening line)

The reason for this post was is the following question posted on one of our internal Data on Devices related aliases.

We wish to push SQL Server Mobile database file from a desktop application developed in VS2005 to the mobile device Here it is in a nutshell:

1) POPULATE the mobile database with data on the desktop
2) PUSH the mobile database to the mobile device

We have following Questions

1) Figuring out how to populate a SQL Server Mobile database file from a desktop application developed in VS2005 (and to do so directly, without RDA, Replication or XML).
2) Identify the best way to PUSH the above SQL Server Mobile file to the mobile device (i.e., do we still use RAPI's DesktopToDevice?)
3) Determining which .DLL's our application will need to install on the end-users Desktop PC to enable both the POPULATE and PUSH functionality as developed
4) Making sure we have Microsoft's permission to redistribute said .DLL's.

As you have noticed, we have a rich data-on-devices story with Visual Studio 2005. A lot of work went into this area since Visual Studio 2003 and data-on-devices story is now on par with the desktop data scenarios. Which means that the answer to the above question needs to be broadcasted to a much bigger audience...

Here it is, from Laxmi Narsimha Rao, my colleague from SQL Server 2005 Mobile Edition team:

I wanted to let you know the following things before jumping into answering your questions:

1) SQL Mobile is unrestricted only for devices and tablet-pc
2) SQL Mobile is unrestricted for desktops that have either VS 2005 or SQL 2005 installed

Given that information, coming to your operations:

1) POPULATE – This is done on the desktop. It means that you should either have VS 2005 or SQL 2005 in order for SQL Mobile to work. 
    a. Say the desktop has VS 2005, you can populate the database using our System.Data.SqlServerCe namespace 
        i. SqlCeEngine – Create Database 
        ii. SqlCeConnection and SqlCeCommand – Populate the database 
        (Please note that SQL Mobile supports parameterized queries. Let the customer know about this to get best out of SQL Mobile.)
2) PUSH – This is nothing but copying the database file created in the above step to the device. There are lot of ways you can do this: 
    a. VS 2005 Auto Deployment – Add SQL Mobile .SDF file to the device project and when you hit F5, it automatically deploys the .SDF file too 
    b. Use Active Sync File Explorer and copy manually 
    c. Use RAPI

And the answers for your questions:

1) RDA, Merge Replication – None of these is needed to populate the database when you have the data at your hand. But any of these is needed when you have the data in SQL Server.
2) As explained above
3) This depends on what your device app is going to do with database on device. If your app is using the DB just as a local store, then you just need first two cabs. If your device app is going to do some sync with SQL Server, then you need the third cab.  
4) Covered by earlier answers and information 

Let me know if you need some more info and/or other clarifications on the above topic. In fact any topic on Data-on-Devices.

As a side note, I own the Visual Studio for Devices, Data components and I do have plans for a few posts on Data-on-Device. But if there is something specific that you want me to make a post on, I'd be glad. Just let me know...

See you then, in my next post...

Understanding the TypeDescriptor: A Metadata Engine for Designtime Code

Hi folks - long time no see... E.L.D.S.? Yeah, sure! Never mind that, here is a dump of my understanding of the TypeDescriptor. Hope you find it useful!

What is Metadata in the context of CLR? Read all about it here. And what makes this concept really powerful is that any code can access the CLR metadata engine (using managed and unmanaged API) to inspect any given managed code down to each IL instruction. The CLR metadata engine also has API to let other code emit managed code. But once emitted and baked, the CLR metadata engine will only let you inspect the managed code – you will not be able to extend or modify any of it.

Now we all know that the all-powerful and mighty .NET Framework Designtime infrastructure lets every component have 2 views of itself – a designtime view and a runtime view. You might have also observed that the designtime view is not quiet the same as the runtime view. If you haven’t, consider the following:

  1. Every Form on the design surface of a Visual Studio 2005 DeviceApplication project has a property called FormFactor. Can you locate it in System.Windows.Forms.dll using Reflector or ILDAsm or the Reflection API?
  2. If the ErrorProvider component is present on the design surface, all other controls on the design surface have new property called Error on errorProvider1. Can you locate this property in System.Windows.Forms.dll?

This means that the designtime infrastructure is displaying a metadata different from what is returned by reflection API. Clearly someone is modifying the metadata before the designtime infrastructure displays it. Now who would that be?

The System.ComponentModel.TypeDescriptor of course!

TypeDescriptor is a metadata engine provided by the .NET FCL. The MSDN documentation says the following:

... In contrast, TypeDescriptor is an extensible inspection mechanism for components: those classes that implement the IComponent interface...

That is not quiet correct. TypeDescriptor is an extensible inspection mechanism not just for components but for all Types and for individual instances of any given Type. Although if the target is a component, it may have a Site and the Site may proffer services, e.g. the ITypeDescriptorFilterService (discussed below) or the IExtenderProvider, which can further enhance the extensibility of the TypeDescriptor.

To restate: Unlike the native CLR metadata engine, TypeDescriptor lets you inspect as well as modify the metadata (add, change and delete) of the target in any conceivable manner. The term target, for the rest of this text will refer to an element of the set of all .NET Framework Types (including Types imported from COM) and all instances of every .NET Framework Type.

Note that an update to a Type’s metadata is visible for the Type and all its instances. But when the metadata of an instance of any Type is updated, the update is seen only for that instance. Metadata of the Type and its other instances are unaffected.

Also note that TypeDescriptor and Reflection API do not interoperate. While TypeDescriptor gets the initial metadata information using Reflection (explained below), further changes to metadata using TypeDescriptor is not visible using Reflection. Only the TypeDescriptor API will let you inspect the new metadata and update it further.

Here is an example of metadata inspection using TypeDescriptor:

using System;
using System.ComponentModel;

namespace InspectWithTD
{
   
class Program
    {
       
static void Main(string[] args)
        {
           
Console.WriteLine("Attributes on System.Object are:");
           
AttributeCollection attributes = TypeDescriptor.GetAttributes(typeof(object));
           
foreach (Attribute attribute in attributes) Console.WriteLine("- " + attribute.GetType().Name);

           
Console.WriteLine("\nProperties of a string object are:");
           
PropertyDescriptorCollection properties = TypeDescriptor.GetProperties("A String");
           
foreach (PropertyDescriptor property in properties) Console.WriteLine("- " + property.Name);

           
Console.WriteLine("\nEvents on the AppDomain we are running in are:");
           
EventDescriptorCollection events = TypeDescriptor.GetEvents(AppDomain.CurrentDomain);
           
foreach (EventDescriptor @event in events) Console.WriteLine("- " + @event.Name);
        }
    }
}

In the following sections we will take a brief look at the architecture of the TypeDescriptor. Then we see what its capabilities are. And finally close the topic with an example. But before we get started, I suggest you open System.dll in Reflector, navigate to TypeDescriptor and disassemble the method implementations to get a deeper understanding.

Brief Architecture of TypeDescriptor

Internally, the TypeDescriptor creates (on demand) and maintains a stack of TypeDescriptionProvider (provider hereafter) for each target. Providers can be considered as plugins for the TypeDescriptor and they are the entities responsible for supplying the target’s metadata and for most other TypeDescriptor magic.

For Type targets, the TypeDescriptor can initialize the provider stack with a read-only provider that is built on Reflection API. Or with a delegating provider that is hooked up to delegate the calls each time to the latest provider for the Type’s base Type. This way the calls are handled by the latest provider for the Type’s base Type, if the Type doesn’t have provider stack by itself.

For instance targets, the provider stack can be the same as the provider stack of the instance’s Type. Or it can be initialized with a delegating provider that is hooked up to delegate the calls each time to the latest provider for the instance’s Type. This way the calls are handled by the latest provider for the instance’s Type, if the instance doesn’t have provider stack by itself.

Once the provider stack is initialized, the TypeDescriptor lets you add/remove custom providers to/from the stack. Custom providers need to extend TypeDescriptionProvider and can have a parent provider. They can selectively change the TypeDescriptor behavior for a given target by selectively overriding the TypeDescriptionProvider methods. If a parent provider exists, the methods of TypeDescriptionProvider delegate the work to the parent provider. This is how we achieve TypeDescriptionProvider chaining. The provider one on the top of the target’s provider stack (latest provider hereafter) is returned by TypeDescriptor.GetProvider and is the one doing most of the TypeDescriptor’s work as we are going to see below.

To proceed further we must understand the ICustomTypeDescriptor (custom type descriptor or CTD hereafter). Click on the link and see its API – isn’t there a striking similarity with its methods and metadata inspection methods of the TypeDescriptor? Indeed – the purpose of a CTD is to return the metadata for the given target. The default implementation of ICustomTypeDescriptor, the CustomTypeDescriptor, takes in a parent ICustomTypeDescriptor (to enable CustomTypeDescriptor chaining) in its constructor. If the parent exists, each method of the CustomTypeDescriptor simply delegates the call to the parent – else the methods return empty metadata.

TypeDescriptionProvider.GetTypeDescriptor returns the CTD supplied by a target’s provider. For instance targets another method, TypeDescriptionProvide.GetExtendedTypeDescriptor also returns a CTD. But this CTD implementation returns the metadata contributed to the instance by the Extender Providers. The default implementation of this returns empty metadata if the instance is not a component or it is not sited. Otherwise if the target is sited and the site proffers the IExtenderListService, which lists the extenders. If the site doesn’t proffer the service, the extenders in the target’s container are listed. From the list of extenders, the extended metadata is collected.

The TypeDescriptor methods that return the target’s metadata are in fact wrappers around the corresponding methods of the CTD obtained from the target’s latest provider. For Types, calling GetTypeDescriptor on the latest provider obtains the CTD. However obtaining the CTD for an instance is a little involved since the instance itself may implement ICustomTypeDescriptor. If it does, then the CTD returned is a merge of the ICustomTypeDescriptor implemented by the instance and the CTD supplied by the latest provider of the instance – but with a preference to the ICustomTypeDescriptor implemented by the instance. If the target doesn’t implement ICustomTypeDescriptor then the CTD obtained is the one supplied by the instance’s latest provider. This means any arbitrary object, by implementing ICustomTypeDescriptor, can dynamically modify the metadata obtained from it by the TypeDescriptor. Of course many of the TypeDescriptor method take in a boolean argument – which when true, simply tells the TypeDescriptor to ignore the instance’s implementation of ICustomTypeDescriptor.

A few of the above methods that return metadata of instances are a bit more sophisticated. For instance the TypeDescriptor.GetProperties that takes in an instance, determines the set of properties to return in as many as 4 successive stages. The output of each stage is fed to the next stage.

  1. First stage: The CTD is obtained (as above) from the latest provider and it returns the initial set of properties.
  2. Second stage: The input set is merged with the set of properties contributed by the Extender Providers. To obtain this set, GetExtendedTypeDescriptor is called on the latest provider for the instance.
  3. Third stage: If the component is sited and the site proffers the ITypeDescriptorFilterService, the input set is passed to the ITypeDescriptorFilterService to be filtered. This forms the basis of the IDesignerFilter that allows a ComponentDesigner to add/change/delete properties from the set of properties of the component it is designing.
  4. Fourth stage: The input set is filtered based on a set of attributes. The rules used for this filtering is explained well in the Remarks section here.

Whew! That is sophistication for you. The power you wield is just awesome! The above is for GetProperties method with instance target. The logic is similar for TypeDescriptor.GetAttributes and TypeDescriptor.GetEvents methods with instance target. Of course, the metadata obtained from the above stages are cached into the IDictionaryService if the instance happens to be a component and its site proffers the dictionary service. This way, unless the cache is invalid, the subsequent call to obtain the metadata can work off the cache. Note that this is a per-instance cache, unlike the per-Type cache used by TypeDescriptor methods that work on Types. The necessity of a per-object cache is explained well in the Remarks section here. To obtain the per-instance cache, call the GetCache on the latest provider. To clear the per-type and the per-object caches call TypeDescriptor.Refresh. Once the caches are cleared, the TypeDescriptor will rebuild all of them on subsequent calls.

We have seen how we can write a TypeDescriptionProvider and a CustomTypeDescriptor for a given target and return any metadata we choose – say a completely different set from what is returned by Reflection. Now, what if someone wants to call some of the lower level Reflection API on the target to get more information about the target? E.g. say the target is object1 and using TypeDescriptor.GetProperties(object1) we get the Pubic, DeclaredOnly and Instance properties of object1. This set of properties is different from the set obtained by object1.GetType().GetProperties() with the above BindingFlags. We have an inconsistency a.k.a. confusion and this is not a happy situation!  To avoid such scenarios, you should use TypeDescriptor.GetReflectionType(object1).GetProperties() with the above BindingFlags. Internally this method will return the System.Type implementation returned by the GetReflectionType call on the object1’s provider. So if you are greatly mucking around with the metadata, make sure your provider’s GetReflectionType returns a System.Type implementation that returns metadata consistent with the provider’s custom type descriptor. This is exactly what happens in the VSD designers where we are actually using a desktop type (e.g. desktop WinForms Button) to represent the corresponding device type (the NETCF Button) on the design surface. We will delve deeper into this in a future post. Also for another illustration, refer to Brian Pepin’s post on designing abstract forms.

As if this much power isn’t enough, the TypeDescriptor.CreateInstance, takes all of the above a step further. CreateInstance first checks if the ServiceProvider (first argument) proffers a TypeDescriptionProvider and if so, it delegates the call to this provider. Else the provider for the target is located the call is delegated to its CreateInstance. The provider can create and return any instance of any other Type (this is called Instance Substitution). Refer to Brian’s post mentioned above to see application of this. Another application of this is in the VSD designers, where we implement a provider that overrides the CreateInstance calls to hook in Type Filtering into the designtime – this is the trick that filters a desktop Type such that on the design surface it appears as the corresponding device (.NETCF) Type.  More about these in the same future post I promised above.

Capabilities of the TypeDescriptor

The purpose of all of the above blah, blah, yada, yada... is so that you can better understand the capabilities of the TypeDescriptor. A great set of documentation on is already provided on MSDN2 that lists and explains all of them – so read it carefully and understand it and let me know if some parts are still not clear. Understanding the TypeDescriptor is critical to understanding my future posts on Designers. To restate, the following are the capabilities of the TypeDescriptor:

Capability Description
Instance substitution Enables an arbitrary type to be created when another type is requested.
Metadata substitution Enables an object's metadata to be modified.
Attribute redirection Enables attributes to be specified dynamically.
Target substitution and shadowing Enables one object to stand in for another.
Extended type descriptor support Enables access to object properties added by other objects.

A simple illustration...

Let us now close this discussion with a small example. It doesn’t do anything useful other than illustrating that you can add your choice of metadata to any given .NET object/Type. We ask TypeDescriptor to register a custom provider the System.Object. And from the above you can see why this provider automatically becomes the provider for all .NET objects and Types. Our custom descriptor than returns a custom type descriptor which when asked for properties, just creates a custom property and returns it along with the original set of properties. Note that both our custom provider and type descriptor selectively override 1 method each and using chaining they delegate the remaining calls to their respective parents.

using System;
using System.ComponentModel;
using System.Collections.Generic;

namespace ExtendWithTD
{
   
class Program
    {
       
static void Main(string[] args)
        {
           
Console.WriteLine("Properties of an arbitrary object: Console.Out:");
           
PropertyDescriptorCollection properties = TypeDescriptor.GetProperties(Console.Out);
           
foreach (PropertyDescriptor property in properties) Console.WriteLine("- " + property.Name);

           
Console.WriteLine("\nAdd our useless provider System.Object and chain it with the original one...");
           
TypeDescriptor.AddProvider(new UselessTypeDescriptionProvider(TypeDescriptor.GetProvider(typeof(object))), typeof(object));

           
Console.WriteLine("\nProperties of an arbitrary object: Console.Out:");
            properties
= TypeDescriptor.GetProperties(Console.Out);
           
foreach (PropertyDescriptor property in properties) Console.WriteLine("- " + property.Name);
        }
    }

   
/// <summary>
    /// This is our custom provider. It simply provides a custom type descriptor
    /// and delegates all its other tasks to its parent
    /// </summary>
    internal sealed class UselessTypeDescriptionProvider : TypeDescriptionProvider
    {
       
/// <summary>
   
    /// Constructor
        /// </summary>
        internal UselessTypeDescriptionProvider(TypeDescriptionProvider parent)
            :
base(parent)
        {
        }

       
/// <summary>
        /// Create and return our custom type descriptor and chain it with the original
   
    /// custom type descriptor
   
    /// </summary>
   
    public override ICustomTypeDescriptor GetTypeDescriptor(Type objectType, object instance)
        {
           
return new UselessCustomTypeDescriptor(base.GetTypeDescriptor(objectType, instance));
        }
    }

   
/// <summary>
    ///
This is our custom type descriptor. It creates a new property and returns it along
   
/// with the original list
   
/// </summary>
   
internal sealed class UselessCustomTypeDescriptor : CustomTypeDescriptor
   
{
       
/// <summary>
        ///
Constructor
       
/// </summary>
       
internal UselessCustomTypeDescriptor(ICustomTypeDescriptor parent)
            :
base(parent)
        {
        }

       
/// <summary>
        /// This method add a new property to the original collection
        /// </summary>
        public override PropertyDescriptorCollection GetProperties()
        {
           
// Enumerate the original set of properties and create our new set with it
   
        PropertyDescriptorCollection originalProperties = base.GetProperties();
           
List<PropertyDescriptor> newProperties = new List<PropertyDescriptor>();
           
foreach (PropertyDescriptor pd in originalProperties) newProperties.Add(pd);

           
// Create a new property and add it to the collection
   
        PropertyDescriptor newProperty = TypeDescriptor.CreateProperty(typeof(object), "UselessProperty", typeof(string));
            newProperties
.Add(newProperty);

           
// Finally return the list
   
        return new PropertyDescriptorCollection(newProperties.ToArray(), true);
        }
    }
}

There is obviously the inconsistency I mentioned above with this example. Ideally, we should implement a System.Type and the custom type descriptor’s GetProperties method should be using our System.Type implementation. But hey! As an exercise, why don’t you come up with the implementation?

Let me know if something isn’t clear from above. I will be glad to explain further...

See you soon in my next post on VSD designers...

Posted by parthopdas | 9 Comments

Writing .NET CF 2.0 Designtime Code – I: An Overview…

Hi Folks...

I am back and it’d have been sooner if not for ELDS :( Never mind that. As many of you have already noted, with Visual Studio 2005 there have been radical changes in the way designtime code is written for .NET Desktop/CF 2.0. These changes in the .NET CF Designtime architecture do break backward compatibility: .NET CF 1.0 Designtime code will refuse to even compile as .NET CF 2.0 Designtime code. But at the same time we believe these changes will:

  1. Improve the experience of the designtime developer. Now you can have designtime and runtime code separated and modularized and your runtime code doesn’t need to have designtime code sprinkled all over it under #ifs.
  2. Better extensibility story. Writing designtime code for Visual Studio 2003/.NET CF 1.0 was a highly complicated and advanced scenario. It was targeted at ISVs to develop designers for the controls they shipped. With the current technology, if someone can write a control, s/he can, with less than or equal effort, write its designer.
  3. Enable better consistency between runtime and designtime. Controls at designtime will look much closer to what they actually look on the device at runtime.
  4. Provide a robust and powerful underlying designtime architecture/framework for you to build great designtime experience for your customers.

Also some of you have asked whether there are any plans for a v1.0 Designtime to v2.0 Designtime upgrade story. There is definitely something up in the air – we have started thinking about it! As of now I do not know exactly how/when it will shape up. But before we release it and so that you are able to take better advantage of the power you wield with .NET 2.0 Designtime, I highly recommend that you take a very detailed pass through Extending Design-Time Support. This is a thorough documentation of .NET 2.0 Designtime, complete with a variety of walkthroughs, samples and lot of other resources. Understanding this is the key to understanding .NET CF 2.0 Designtime. Believe me when I say that the .NET CF 2.0 Designtime is simply built on the .NET 2.0 Designtime. We do nothing special and neither do we get anything special from the .NET Framework Designtime team, just because they happen to sit next door to us :)

There seems to be a little bit of confusion all around on how folks ought to author designtime code for .NET CF 2.0. To help clear this confusion, I am going to temporarily suspend my currently running blog thread on .NET Framework Designtime. I will resume it once I am done laying out some information on how to tackle the .NET CF 2.0 Designtime and the VSD WinForms designer.

I am planning to make divide this series into 3 more posts. Next post will be on designtime assemblies (XMTA/AsmMeta) and why they are critical to the .NET CF 2.0 Designtime. In the post after I am going to build a .NET CF custom control and its designer. And in the last post, I will provide tips on debugging .NET CF 2.0 Designtime code. Let me know if you need me to cover other topics.

See you soon...

Posted by parthopdas | 3 Comments

Generics Terminology in .NET Framework

Hi Folks...

The other day I was investigating an issue that finally boiled down to incorrect usage of System.Reflection & System.Reflection.Emit APIs (SR & SRE hereafter) to analyze Generic types. We were using Type.IsGenericParameter instead of using Type.IsGenericType. During investigation, what made things worse is that each time I went through the first paragraph of their respective help pages, I got more confused. Then I decided to read the Remarks section of the help pages and I was hopelessly lost because it seemed to be written in a dialect of English that I didn’t speak! 

Do you know the difference between Type.IsGenericParameter and Type.IsGenericType? The difference between Generic Type Parameter and Generic Type Argument perhaps? How about Closed Constructed Generic Method and Open Constructed Generic Method? These are a part of some subtle terminology introduced into .NET Framework with Generics. Using SR & SRE without a clear understanding this terminology is only going to land you into trouble. You are going to introduce bugs into your code and you will have a really hard time debugging your own code! Just like me!

It is all there in MSDN but it is highly confusing at the first glance! So I decided to get done with this for good. I spent a good 3 hours going over Generics terminology in .NET Framework in MSDN. This is a gist of my understanding. I am all clear now. You should be too before you begin using SR and SRE.

Basic Terminology

  • Generics:
    • They are classes/structs/interfaces/methods (CSIM hereafter) that have placeholders for one or more of the types they store and/or use.
  • Generic Type:
    • Union of the set of all Generic Type Definitions and Constructed Types.
    • A System.Type object represents a Generic Type iff its IsGenericType property is true
  • Generic Method:
    • Must have a non-empty list of Generic Type Parameters.
    • A method being a Generic Method doesn’t have anything to do with the enclosing type being Generic Type or not
    • Union of the set of all Generic Method Definitions and Constructed Methods.
    • Generic Type Parameters can appear as the return type or as the types of the formal parameters
    • A System.Reflection.MethodInfo object represents a Generic Method iff its IsGenericMethod property is true
  • Generic Type Definition:
    • A CSI declaration that serves as a template, with placeholders for the types it can contain or use.
    • A System.Type object represents a Generic Type Definition iff its IsGenericTypeDefinition property is true
  • Generic Method Definition:
    • A method with two parameter lists: A non-empty list of placeholder types and a list of formal parameters
    • A System.Reflection.MethodInfo object represents a Generic Method Definition iff its IsGenericMethodDefinition property is true
  • Generic Type Parameter:
    • It is the placeholder type in a Generic Type/Method Definition.
    • It is represented using a System.Type object in SR and SRE.
    • A System.Type object represents a Generic Type Parameter iff its IsGenericParameter property is true
    • Example: Is T in Dictionary<T, int> in "class C<T> : Dictionary<T, int>" a Generic Type Parameter? Yes!
  • Generic Type Argument:
    • It is any type that is substituted for a Generic Type Parameter. It could be another Generic Type Parameter.
    • IsGenericParameter is false
    • A System.Type object represents a Generic Type Argument iff its IsGenericParameter property is false
  • Constructed Generic Type/Method:
    • A new type/method constructed as a result of specifying actual types for one or more of the Generic Type Parameters of a Generic Type/Method Definition.
    • Example: Is A<B<, >, > a Constructed Generic Type? Yes!
  • Closed/Open Constructed Generic Type:
    • A Closed Constructed Generic Type is the result of specifying Generic Type Arguments for all Generic Type Parameters of a Generic Type/Method Definition. An Open Constructed Generic Type otherwise.
    • A System.Type object represents a Closed Constructed Generic Type iff its ContainsGenericParameters property is false. Otherwise it is an Open Constructed Generic Type.
    • Only a Closed Constructed Generic Type can be instantiated
    • Is A<B<T, string>, int>.C a Closed Constructed Generic Type? No!
  • Closed/Open Constructed Generic Method:
    • A Closed Constructed Generic Method has no unassigned Generic Type Parameters *and* the containing type is a Closed Constructed Generic Type *and* all the Generic Type Arguments are Closed Constructed Generic Types. An Open Constructed Generic Method otherwise.
    • A System.Reflection.MethodInfo object represents a Closed Constructed Generic Method iff its ContainsGenericParameters property is false. Otherwise it is an Open Constructed Generic Method
  • Constraints:
    • They are limits placed on Generic Type Parameters.
    • The GenericParameterAttributes property of a System.Type object representing a Generic Type Parameter gets a combination of System.Reflection.GenericParameterAttributes flags that describe its covariance and special constraints
    • The GetGenericParameterConstraints() method of a System.Type object representing a Generic Type Parameter returns an array of System.Type objects that represent its constraints.

A few points to note

  • If a nested types doesn't have Generic Type Parameters of its own, the enclosing type determines the above for it.
  • Generic Type Parameters and Generic Type Arguments have the same relation as the parameters and arguments of a function.
  • typeof(Dictionary<, >).MakeArrayType() returns a System.Type object that is not a Generic Type but is an Open Constructed Generic Type. Same for pointers
  • In SR & SRE:
    • Generic Type Parameters are represented by System.Type
    • A Type/MethodInfo object representing a Generic Type/Method has an array of types containing the Generic Type Parameters and Generic Type Arguments

Examples to consolidate our understanding

Consider the followign code snippet. The invariant conditions are shown in the comments below it.

public class Base<T, U>
{
    public static T M1Base(U u) { return default(T); }
}

public
class Derived<V> : Base<string, V>
{
    public G<Derived<V>> F;
   
    public
class Nested
    {
        void M1Nested() { }
    }
   
    public
static void M1Derived<W>() { }
}

public
class G<T> { }

/*
Type: Derived<V> 
                 IsGenericType: True
       IsGenericTypeDefinition: True
     ContainsGenericParameters: True
            IsGenericParameter: False

Type: Base<string, V>
                 IsGenericType: True
       IsGenericTypeDefinition: False
     ContainsGenericParameters: True
            IsGenericParameter: False

Type: Array of Derived<int>
                 IsGenericType: False
       IsGenericTypeDefinition: False
     ContainsGenericParameters: False
            IsGenericParameter: False

Type: Type parameter T in Base<T>
                 IsGenericType: False
       IsGenericTypeDefinition: False
     ContainsGenericParameters: True
IsGenericParameter: True

Type: Field Derived<V>.F
                 IsGenericType: True
       IsGenericTypeDefinition: False
     ContainsGenericParameters: True
IsGenericParameter: False

Type: Nested
                 IsGenericType: True
       IsGenericTypeDefinition: True
     ContainsGenericParameters: True
            IsGenericParameter: False

Method: T Base<T, U>.M1Base(U u)
               IsGenericMethod: False
     IsGenericMethodDefinition: False
     ContainsGenericParameters: True

Method: void Derived<V>.M1Derived<W>()
               IsGenericMethod: True
     IsGenericMethodDefinition: True
     ContainsGenericParameters: True

Method: void Derived<V>.M1Derived<int>();
               IsGenericMethod: True
     IsGenericMethodDefinition: False
     ContainsGenericParameters: True

Method: void Derived<string>.M1Derived<int>()
               IsGenericMethod: True
     IsGenericMethodDefinition: False
     ContainsGenericParameters: False
*/

Some of the System.Reflection API for Generics

  • System.Type.*Generic* members:
    • [P] ContainsGenericParameters: True if the type is an Open Constructed Generic Type. False otherwise. 
    • [P] GenericParameterAttributes: Gets a combination of System.Reflection.GenericParameterAttributes flags that describe the covariance and special Constraints of a Generic Type Parameter
    • [P] GenericParameterPosition: Gets the position of the Generic Type Parameter in the Generic Type Parameter list of the Generic Type/Method that declared the parameter. 
    • [P] IsGenericParameter: True if the type represents a Generic Type Parameter. False otherwise. 
    • [P] IsGenericType: True if the type represents a Generic Type. False otherwise. 
    • [P] IsGenericTypeDefinition: True if the type represents a Generic Type Definition. False otherwise. 
    • [M] GetGenericArguments: Get the array of types representing Generic Type Parameter/Arguments of the type.
    • [M] GetGenericParameterConstraints: Returns an array of System.Type objects that represent the Constraints of a Generic Type
    • [M] GetGenericTypeDefinition: Gets the Generic Type Definition corresponding to the Generic Type.
    • [M] MakeGenericType: Construct Closed/Open Constructed Generic Type from the Generic Type Definition
  • System.Reflection.MethodInfo.*Generic* members:
    • [P] ContainsGenericParameters: True if the type is an Open Constructed Generic Method. False otherwise.
    • [P] IsGenericMethod: True if the MethodInfo represents a Generic Method. False otherwise.  
    • [P] IsGenericMethodDefinition: True if the MethodInfo represents a Generic Method Definition. False otherwise.  
    • [M] GetGenericArguments: Get the array of types representing Generic Type Parameter/Arguments of the MethodInfo
    • [M] GetGenericMethodDefinition: Gets the Generic Method Definition corresponding to the Generic Method
    • [M] MakeGenericMethod: Construct Closed/Open Constructed Generic Method from the Generic Method Definition

References

If you want to learn more about Reflection & Generics in .NET Framework here are some references:

Finally as an exercise:

Type.IsGenericType Property help page contains an error in the Example-Invariants table. Armed with the above knowledge find it out and let me know! :)

That's all for now... See you in my next post!

.NET Framework Designtime – I: An Introduction...

Hi Folks...

For a while now, I have been studying the .NET Framework Designtime Infrastructure. Not only because I now own the VSD WinForms and Data designers. But also because the design & architecture of this piece is quiet fascinating… What is the .NET Framework Designtime Infrastructure.? It is the guts of the Visual Studio designer that lets you rapidly design and build rich Windows Forms UI.

In this series, I will dump my understanding of the .NET Framework Designtime. We will also progressively build an application that can design Windows Installer UI. Our application will also persist the UI in a Windows Installer XML Source file (.wxs) that you can readily use in your WiX projects. The first part of the series will be an introduction to the .NET Framework Designtime. Nothing much explained but a lot outlined. So here goes…

First off...

The term designer used here is not the same as VS WinForms Designer that you see when you create a new windows application. What you see is really the design surface provided by VS acting as the designer host.

Earlier, in the MFC/ActiveX worlds, the logic for designtime behavior of a component was hardcoded into the tool (a.k.a. IDE) that provided the design surface – meaning, it was the IDE and not the MFC/ActiveX component that was responsible for providing its (the component’s) designtime behavior. This also meant that different IDEs offered their own designtime environments for components, and with different user interfaces and capabilities. And since an IDEs fixed feature set cannot account for the needs of all components, ActiveX/MFC components had only a limited level of custom designtime support.

In .NET Framework world though, the framework provides every component with its own runtime (obviously!) and designtime behaviors. And just like how the runtime behavior of a component can be customized/extended (e.g. subclassing), so can the designtime behavior of the component be customized & extended by IDEs. This means that every .NET Framework component contains logic for how it behaves and runtime and at designtime. The IDE simply extends, configures and exposes this designtime behavior to the developer. There are of course some services and behaviors that are very strongly tied to the IDE that hosts the design surface (e.g. exposing the design surface, context menus, toolbox interaction etc.). These services are implemented by the host IDE and the component uses these services at designtime after obtaining them from the host IDE through well defined interfaces.

Comparing MFC/ActiveX & .NET Framework Designtime: Click to enlarge

Designtime Runtime code seperation...

Of course when I say that every component provides both its runtime and designtime behavior, I don’t mean that every component needs to contain the code for its designtime behavior and the code for its runtime behavior in the same class or even the same assembly. Although this is possible, in most cases it is not practical. Doing so would simply bloat the runtime code which necessarily needs to be as slim and trim as possible. Note that every byte loaded into memory has a cost associated with it (working set, time to JIT etc.). As an example, consider System.Windows.Forms.Control. Its runtime code is in System.Windows.Forms.Control, System.Windows.Forms.dll and its designtime code in System.Windows.Forms.Design.ControlDesigner, System.Design.dll. Use Reflector to peek into the implementations of both and you will get an idea of the bloat. Just going by lines of code Control.cs is 19217 lines and ControlDesigner.cs is 3842 lines which is ~20% code bloat.

So more often than not, designtime and runtime code separation is employed. This basically means that the designtime code is located on a different class/assembly than the runtime code, like in the System.Windows.Forms.Control example. And the runtime code provides pointers to the designtime code using one or more attributes (e.g. DesignerAttribute). At designtime the .NET Designtime Infrastructure then just looks up the designtime attributes on the component, locates and executes the designtime code to allow us to design the component.

Things get a little bit trickier when you are designing .NET CF components. The .NET CF usually executes on memory constrained devices so it needs to be highly trimmed. Since attributes contribute to size of metadata and therefore overall size of the assembly, the usage of attributes in .NET CF code cannot be as lavish as that on the desktop. Because of this the .NET CF doesn’t support any of the designtime attributes. To work around this the VSD WinForms designers uses the XMTA and AsmMeta files. I will elaborate on the XMTA/AsmMeta files in a future blog but for now, in short, the pointers to the designtime code reside in the AsmMeta assemblies. So at designtime for a .NET CF component, the VSD WinForms designer locates the AsmMeta assembly for the component, reads the pointer to the designtime code from it, loads and executed the designtime code to allow us to design the .NET CF component.

An overview of the designtime infrastructure...

How do you typically use the IDE to design a windows forms application? For starters, you create a class that derivers from Form (or UserControl or Component). Then you open that class in design view and drag-n-drop other controls and/or components on it, edit the various properties and hook up handlers for the various events. While you are designing the application the IDE may look like the following:

Design surface items: Click to enlarge

As usual, the details hidden from the user are a lot more complicated. The following diagram will give you an idea of what is actually going on under the hood. A sound understanding of the pieces in the diagram is the key to understanding the .NET Framework Designtime Infrastructure. In this series I will be explaining each of these pieces.

Designtime Environment: Click to enlarge

How and where does the VSD WinForms designer fit in?

In the case of the desktop .NET Framework both designtime and runtime code is executed by the desktop CLR. But for .NET CF, the designtime code is executed by the desktop CLR (hosted inside Visual Studio) while the runtime code is executed by the device CLR. This implies that for .NET CF components the designtime code targets the desktop .NET Framework while the runtime code targets the .NET CF.

Also the architecture of .NET Designtime is such that during designtime the .NET Framework components (well controls really!) draw themselves! Meaning at designtime, in addition to the component's designtime code, its runtime code is also executing! Meaning during designtime, since Visual Studio has loaded only the desktop CLR, device controls that require the device CLR cannot run! All device components belong to the following two categories:

  1. Retargetable: On the desktop when you try and load a retargettable assembly (e.g. System.WinForms.dll), the desktop fusion will load the corresponding desktop assembly automatically
  2. Desktop Incompatible: Assemblies referring directly or indirectly to platform specific drawing/rendering code (e.g. through P/Invoke) For both above cases, you are forced to use the desktop components during design time!

The above is the basis for the .NET CF Designtime.

Moreover the set of Properties/Attributes/Events exposed by a .NET CF component is usually, but not always, a subset of the set exposed by the corresponding desktop .NET component. This needs to be reflected during the designtime – we cannot show a desktop only P/A/E while designing a .NET CF component. To achieve this VSD WinForms designer uses the XMTA/AsmMeta trick that is based on the P/A/E filtering natively supported by the .NET 2.0 designtime.

Central to the P/A/E filtering architecture is the TypeDescriptor. The v2.0 .NET designtime doesn’t use System.Reflection to obtain a component’s metadata. Instead the TypeDescriptor is used. Using it, you can remove all of the components native metadata (ones that you get by System.Reflection) and replace them with your choice of metadata! More on this in a subsequent post…

Thats all this post. Next post will dig deeper into the Designtime Environment diagram and start constructing our WiX UI designer.

See you later...

Posted by parthopdas | 9 Comments

Object Test Bench – IV: Please send us your feedback and vNext wish list...

Hi Folks…

It’s been a long long time since my last post. What have I been doing since then? Hmmm… apart from ELDS here is that list:

  1. Transitioned out of J#: Handed over my owner ship of J# Project system, Object Test Bench and other areas.
  2. Started on transitioning into VSD: Had a few meetings with Carlton.
  3. Got married to Anutthara and went for a 2 week post marriage vacation.
  4. Finished transitioning into VSD after numerous meetings with Xin, Carlton, Andrew and Brian Chamberlin: I now own the VSD Data designers, WinForms Designers and some parts of managed VSD IDE.
  5. Got rammed down by a spate of VSD bugs, wrestled with them hard and finally prevailed. We are now ready to ship Visual Studio 2005.

But these are of course very poor excuses for not coming up with a single post in 6 months. But I am going to change that esp. since engaging and spending time with you guys is one of the focus points for my next and future career reviews :-)

Getting back to the topic on hand…

Object Test Bench has been out for many months now. The final release will have very few changes from Beta 2. And we are looking forward to enhancing it for v2. Here is your chance to shape the v2 of Object Test Bench.

How have you liked OTB so far? What is the weirdest way you have used it? Which feature of OTB did you hate the most? What are the roadblocks you have hit? What enhancements would you like us to make in v2? What do you think it will take for OTB to be a killer component?

Personally, I would like to see the following in v2.

  1. Support for Generics
  2. Enhanced integration with the debugger (e.g. edit and continue and support for native C++)
  3. Built in support for Unit Testing

Basically OTB should graduate to a full blown feature helping in enterprise software development. But note that I am no longer in the team that owns OTB, so you should not be surprised if OTB v2 turns our completely different from what I think it should :-)

We are eagerly waiting for your feedback, criticisms, comments and ideas for v2. You may send them through any of the channels below:

  1. Make a suggestion or repro a bug on MSDN Product Feedback Center
  2. Send a mail to JSharp Feedback
  3. Contact and/or reply to the team members’ blogs (Brian Keller, Varun Gupta, Gaurav Seth, Jaiprakash Sharma)
  4. You may email your suggestions/bug directly to our product manager, Brian Keller
  5. Post your feedback and feature requests on the Visual J# Forum
  6. Make a post on the J# public newsgroup at microsoft.public.dotnet.vjsharp

Last but not the least you may contact me and I will personally go and deliver your message to the J# team. (They are sitting just next door to me anyways :-)

See you in the next post…

Posted by parthopdas | 3 Comments
Filed under:

Object Test Bench – III: Exploring Framework APIs...

Hi Folks…

How often have you surfed the web, wanting to learn about a new whiz-bang .NET Framework API that you just heard of? Often right? I for one have found myself do this very frequently…

Surfing the web usually comprises of running a web search on the whiz-bang keywords that got you in the mood in the first place. With a bit of luck, the search engine returns a host of sites that contain documentations/tutorials/samples/snippets on the API you are looking for. There are sites like The Code Project, DevX.com, DevHood that are doing a great job in introducing the API to audiences at all levels. Then there also sites like the one you are currently on – the blogs. After reading the documentation/comments on these sites you aren’t really done till you actually try out the code snippets/samples – right?

Here is when OTB can help you out! You no longer need to copy the sample, build it and then debug it before you can get a grip on what’s the API is really about. With as little work as starting the IDE and creating a Zero Impact project, OTB will help you understand the API visually. This is what I am going to show you today. And note that, the API need not really be an .NET Framework API. It could be any 3rd party API written for the .NET platform.

Searching on the web for Reflection.Emit…

One of the killer features that .NET Framework offers the compiler writers is the Reflection.Emit namespace. Now I want to learn about that. So I run a web search with “Reflection.Emit dynamically generated code”. And the first hit on one of the popular search engines is an excellent blog by my fellow Microsoftie, Mike Stall. That’s a good hit – apart from demonstrating how to generate code dynamically, Mike also demonstrates how to debug the dynamic code!!! Ain’t that really totally cool!!!

So what do we do next?
1.    Open launch Visual Studio 2005
2.    Create a Zero Impact project (I am creating a C# console application – you could create either that or any of the C#/J#/VB projects).
3.    Save this text file on your hard disk (say C:\temp\souces.txt)
4.    Open C:\temp\source.txt in VS and set a break point in the second line (i.e. the line with “xyz = "hello";”)
5.    Turn of Tools::Options::Debugging::General::Enable Just My Code (Managed only)
/*01*/  // Sample of emitting debugging information for dynamic modules
/*02*/  // Reflection emit example adopted from http://blogs.msdn.com/joelpob/archive/2004/01/21/61411.aspx
/*03*/  using System;
/*04*/  using System.Reflection;
/*05*/  using System.Reflection.Emit;
/*06*/  using System.Threading;
/*07*/  using System.Diagnostics.SymbolStore;
/*08*/ 
/*09*/  public class EmitHelloWorld
/*10*/  {
/*11*/      static void Main(string[] args)
/*12*/      {
/*13*/          // create a dynamic assembly and module
/*14*/          AssemblyName assemblyName = new AssemblyName("HelloWorld");
/*15*/ 
/*16*/          AppDomain appDomain = Thread.GetDomain();
/*17*/          AssemblyBuilder assemblyBuilder = appDomain.DefineDynamicAssembly(assemblyName, AssemblyBuilderAccess.RunAndSave);
/*18*/ 
/*19*/          ModuleBuilder module = assemblyBuilder.DefineDynamicModule("HelloWorld.exe", true); // <-- pass 'true' to track debug info.
/*20*/ 
/*21*/  &n