Meffifying Windows Azure

Magnus has been off doing some interesting work around integrating MEF with Windows Azure. The first question you might be asking is Why?

In his words, he set out to build a template for Windows Azure templates that:

In the post he shows how to take the RoleManger and expose it through MEF, thereby making it pluggable. He then creates a mock Role Manager for use in his unit tests, thus removing the dependency on all the Azure infrastructure.

I am guessing this is the first of many posts to come on MEF and Azure.

For more, check out Magnus post here.

Upcoming talks at NDC in Oslo and Poland

Next week, I am heading to Europe for two weeks to give several talks. My first stop is NDC 2009, where I’ll be delivering the following talks:

• Framework Design Guidelines
• Building Maintainable Enterprise Applications with Silverlight and WPF
• Building openly extensible applications with .NET Framework 4.0

NDC looks like an amazing event with a great lineup of speakers. I am hoping to be able to attend a few talks so I can soak up all their goodness. All my talks are in one day, so it’s going to be real wild ride!

Next is on to Poland for a 5 city user group tour in: Olsztyn, Toruń, Warszawa, Kraków, Katowice where I’ll be speaking about building extensible apps, or other topics if folks attending would like (comment to this post if you have other topics)

If you are attending NDC or will be in any of the cities mentioned, I look forward to chatting with you about how we can build better software!

Customizing container behavior part 2 of N - Defaults

Before I get into the post, let me start off with a word of advice for new bloggers. Never say in a post, this is the first of many to come. RESIST! You are just setting yourself up for never doing that second post. I know as the last post in this series was five months ago. ;-)

In the last post, I introduced the concept of ExportProviders as a way to customize how the CompositionContainer discovers exports. One of the primary use cases for doing this is to provide default exports, which is particularly important for cases where you are importing a single value for which there are multiple present. The common case I like to use is the overused Logger example. Let’s say you have multiple loggers in the system, which are used for different functions. One logger logs locally, while others can log remotely perhaps invoking a web service, or using MSMQ. These loggers are all pluggable, as different organizations that use the app may have different needs. Each of these loggers export an ILogger contract.

Now most of the time when logging occurs the local logger is sufficient. It’s only exceptions to the rule that require the special loggers. And this is where you run into a problem. How do you specify that you need to import that default logger, where there are multiple present? In the default configuration of MEF, the only way to really handle this if you are using imports is to either create a specialized contract for the default logger say IDefaultLogger which is imported, or to import the entire collection lazily using Export<ILogger> and then looking at the metadata for each to find the right one. You could also create a custom service called LoggerService, which imports all the loggers finds the default one, and then returns it when LoggerService.DefaultLogger is called. None of these are optimal though.

For example, you could do this….

[Export]
public class LoggerManager : ILoggerManager
{
    [ImportingConstructor]
    public LoggerManager(Export<ILogger,ILoggerMetadata>[] loggers)
    {
        var logger = loggers.Single(e => e.MetadataView.IsDefault==true);
        DefaultLogger = logger.GetExportedObject();
    }
    
    public ILogger DefaultLogger { get; private set;}

}

The problem is this means the LoggerManager has to have hard knowledge of which logger is the default. It also means the logger itself needs to somehow declare that it is the default, which is problematic because one logger might be the default in one app, but not in another. Separation of concerns anyone? Even worse you might have two loggers that say they are the default!

Now there are other mental hoops you can jump through to find a workable solution, but it’s a lot of pain when what you really want you want to do is tell MEF’s container somehow “This is my default logger for this application”. Not only that but you want to do it in a way that does not require the app to hardcode it, does not require explicit configuration, nor does it require the parts to somehow ‘tell’ MEF they are the default. Also you don’t want the parts that are importing ILogger to have to know or care about a LoggerManager or anything like that. Instead they should simply have an import of ILogger.

[Export]
public class Executor
{
    [ImportingConstructor]
    public Executor(ILogger logger)
    {
    }
}

The $25000 question is how? The answer is using our friendly ExportProvider. In my last post I described the role of ExportProviders, which is to simply return exports. Those exports can come from several different sources, including from catalogs.

Constructing the container with ExportProviders

If you look on the constructor of the container, you’ll see several overloads which accept an ExportProvider.

public CompositionContainer(params ExportProvider[] providers);public CompositionContainer(ComposablePartCatalog catalog, params ExportProvider[] providers);  

Any export providers passed in to the container are incorporated into the container’s query strategy. This means whenever anyone pulls on the container for an export, these providers will also get queried. Now catalogs have a special provider associated with them called a CatalogExportProvider. As a matter of fact the overload on the container that accepts a catalog is really just syntactic sugar. Behind the scenes we immediately create a CatalogExportProvider passing in the catalog. This means you can pass in an additional default catalog. If you’ve worked with MEF before, you might be shaking your head at this point and thinking “How is this any different than just having an AggregateCatalog and adding two catalogs?” And that IS the key question ;-).

The answer is that adding catalogs to an AggregateCatalog does not allow handling defaults, however using this technique with ExportProviders does. I’ll explain how next.

AggregateCatalogs vs AggregateExportProvider

AggregateCatalog is a catalog that contains a list of catalogs. It is a composite pattern, so querying the parent, queries all the children. This is especially useful when you have parts that are “built in” to the application which are added via an AssemblyCatalog, and other parts which are discovered through a DirectoryCatalog. You can add both catalogs to an AggregateCatalog and everything works nicely. It doesn’t help you though in a default scenario however. The reason is because if one of your catalogs contains your default ILogger and your other catalogs contain the pluggable ILogger implementations, then the aggregate catalog will return them all when it is queried. This means the Executor above will blow up as the constructor is expecting a single ILogger.

For the AggregateExportProvider however, we designed specifically to allow this defaults scenario to be addressed. The provider behaves differently depending on whether it is being queried for a single export or multiple. If the query is for multiple exports, then it works similar to the AggregateCatalog and collects all the exports it finds in any of it’s providers. If on the other hand the query is for a single export, then it uses a prioritization scheme. It will start at the top of the list of ExportProviders and query the first one for the single export (Logger). If it does not find one, then it keeps querying on to the next ExportProvider and so on. Once it finds an ExportProvider that returns one, then it will be returned, if it does not find one, then it will throw an exception.

This means you can have default ExportProviders in the chain (including CatalogExportProviders) that contain your defaults so that the Executor class always succeeds in creation. You can even configure the behavior so that the defaults are overridable or not. For example if you want the default ILogger to always be the logger that is returned for all single import queries no matter what, then put the defaults at the top of the chain. If however you want the default logger to be overridable by another single ILogger, then put it at the end.

Here’s an illustration of what I mean.

public void OverridableDefaults()

{

    var catalog = new DirectoryCatalog(@".\Extensions");

    var defaultCatalogEP = new CatalogExportProvider(new DirectoryCatalog(@".\Defaults"));

    var container = new CompositionContainer(catalog, defaultCatalogEP);

    defaultCatalogEP.SourceProvider = container;

}

In the configuration above, we are placing our defaults at the end of the chain of providers. Remember as  I mentioned above that the ‘catalog’ reference will get wrapped in a CatalogExportProvider as well. It will get added first, followed by the additional providers. We then have to set the SourceProvider to the container, I talked about why this is necessary today in my previous post, though it may be possible for us to infer the setting in the future. Anyway, now when a query happens for a single logger, the container will first check the parts in the Extensions folder, and if none are found it will revert to the defaults.

public void NonOverridableDefaults()

{

    var defaultCatalog = new DirectoryCatalog(@".\Defaults");

    var catalogEP = new CatalogExportProvider(new DirectoryCatalog(@".\Extensions"));

    var container = new CompositionContainer(defaultCatalog, catalogEP);

    catalogEP.SourceProvider = container;

}

In this example, we are passing in the default catalog first, and then passing the catalog (as an EP). This configuration now means that the defaults are NEVER overridable, as whenever the container is queried for a single logger, it will ALWAYS return the default.

Using Type or Assembly Catalogs for defaults

Now in the approach above we are using DirectoryCatalogs for specifying defaults. But remember, this is an option, you don’t have to, and in many cases might not want to. For example you might want to just specify your list of defaults in code, or even in a config file. TypeCatalog provides a great way to do this. (It accepts a params of Types, so you can add as many as you want).

public void OverridableDefaultsWithTypeCatalog()

{

    var catalogEP = new CatalogExportProvider(new DirectoryCatalog(@".\Extensions"));

    var defaultCatalogEP = new CatalogExportProvider(new TypeCatalog(typeof (ILogger)));

    var container = new CompositionContainer(catalogEP, defaultCatalogEP);

}

OK, so now that you sold me, what are the caveats?

As with all things, there are some. Really, I can only think of one primary one, and that is that there’s still a bit of indeterminism as to what will get returned on a single import depending on how you set up the provider chain. If you go with the approach of putting defaults first, then it is completely deterministic. You know that whenever a single import request is issued to the container, the exports in that catalog will be returned, period. If however you arrange the defaults at the end of the chain, then it really depends on what is in the middle. Let’s say you have 3 EPs, and the one in the middle has a part with the single export that you are looking for, while you also have a default. In that case the one in the middle will get returned. Another caveat might be that this does add a bit of complexity, however I would argue that’s a reasonable trasdeoff to having to import collections all over the place.

The code

You can download sample code that illustrates using ExportProviders in various fashions here

If you look inside, you’ll find much more than just what we covered here.  Including how to do do role based composition, or even wire MEF to a configuration store where config info is an import. All of this will get covered at some point.

Additionally:

1. You’ll find several test classes which are using a context/specification style of testing to illustrate how to use ExportProviders.  If you are not familiar with BDD/CS it may seem very foreign, but I think you’ll find it easy to follow once the initial shock wears off. Having gone through this experience made me a big fan of CS style, and that is for another post.
2. You’ll find several test helper classes I’ve written specifically for MEF. These are really useful for folks writing their own ExportProviders and who wish to mock / fake certain parts of our programming model. Specifically I am talking about FakeExportDefinition and FakeExportProvider. Additionally you’ll see some common classes I’ve used for my own CS experiments.

For stuff relating to what was covered in this post, go check out the Import_Logger_Specs.cs unit-tests.

Special thanks to Scott Bellware as well as Scott C. Reynolds and David Foley who invested a lot of time and effort in getting me off the ground in using this style of testing. Without their help, I would have been stranded in a pool of confusion. This post is really not doing CS justice, and I really need a separate series. I can’t guarantee that I’ll get to it, but I’ll try.

What’s next

In the next post we’ll talk about establishing parent-child containers hierarchies. You may be asking how does this relate to ExportProviders? I’ll leave you with this thought to ponder, our container IS an ExportProvider.

“View Model” - the movie, cast your vote.

http://www.flickr.com/photos/richardmoross/3064808115/

No we’re not actually making a movie about View Model. If we were I am sure we’d get a low turn-out. :-) However it did get your attention.

What we are doing is some serious exploration into how we can help from a platform perspective folks that are implementing ViewModel (AKA Model-View-ViewModel  AKA Presentation Model)  in their Silverlight and WPF applications.

In the sense of a movie production, we’re in the script-writing phase. Here’s the general background.

Why is ViewModel important?

There have been many a post on the importance of ViewModel as a pattern and the benefits it brings. In that there is little disagreement that it benefits both developers and designers.

For developers it allows them to have clean separation of presentation logic from the UI rendering, thus making the app easier to maintain. It removes code from the code-behind that often makes the UI logic difficult to test.

For designers it provides a way for a designer to completely change the look of the UI in a tool such as Expression Blend, without breaking the application functionality. Furthermore it gives the designer freedoms beyond simple look and feel, to actually decide which data elements are exposed, as well as defining the interactions between the UI elements through commands and attached behaviors.

The debate over roles

Although the benefits of ViewModel are known, there’s quite a lot of debate both inside and outside Microsoft on the roles the developer and designer play in developing applications that use the pattern.  This runs the spectrum from the designer having complete control of every aspect of the UI,  to the development team having more of a tight handle on the UI components, and the designer being responsible for skinning. In either case there are tradeoffs.

What is important to you?

What is your experience with ViewModel today?

1. Where does the platform help,? Where does it hinder?
2. What can we do in the platform to make life easier?
3. What role do developers / designers play in your application of ViewModel.

Like any movie, we need to touch on the right nerves in order to make it a box-office hit. We need your feedback so we can complete the script.

Thanks for your help

MEF Preview 5, changes and enhancements

We recently shipped MEF preview 5 on Codeplex, an exciting release.

In the latest release, you’ll find we’ve made quite a few changes, and some really powerful enhancements to our previous codebase. In this post, I’ll talk about those changes in detail, and why we made them. I’ll also discuss some changes we will be making in the future, in order to give you a heads up, so that you can adjust your code bases now.

Throughout the development of preview 5, we’ve done our best to minimize the impact to you. There are some breaking changes, primarily in terms of namespace refactoring, most of which do not effect part authors, but will effect MEF hosters, and authors of custom programming models or export providers.

Here’s a summary of these changes for those who want to just cut to the chase. For those who want the full skinny, that comes after the summary.

MEF Studio

MEF Studio is a designer hosting sample. It has an IDE that hosts various designers like Windows Forms, UserControl, etc. One can add controls on to these designers from the toolbox, set properties using the property grid, view generated code, etc. It provides design time experience for the designers and has extension points for extending the adding custom parts.

Migrating from preview 4

When you upgrade your existing apps, you will likely run into a few issues. Wes put together the following useful list of common errors that you might run into when migrating.

• System.ComponentModel.Composition.Container does not exist -> Add reference to System.ComponentModel.Composition.Hosting namespace
• CompositionContainer does not container method AddPart or Compose -> Need to start using CompositionBatch, or one of the helper extension methods ComposeParts or ComposeExportedObjects
• CompositionOptionsAttribute does not exist -> For CreationPolicy use PartCreationPolicyAttribute
• INotifyImportCompleted does not exist -> Use IPartImportsSatisfiedNotification interface and change method from ImportCompleted to OnImportsSatisfied

Part Lifetime changes

Previously, the CreationPolicy property of the CompositionOptions attribute was used to declare a part’s lifetime. We found this was less discoverable for customers as they have to know about the existence of the attribute. We also found that CreationPolicy was ambiguous in name, as it does not make clear whether or not the policy applies to the part or the export.

With the new changes, you will use the new PartCreationPolicyAttribute. CompositionOptions has been removed. Having part in the name of the attribute makes it easier to discover, and also removes ambiguity as to it’s scope.

Below the Logger class is declared as shared:

[PartCreationPolicy(CreationPolicy.Shared)]
[Export(typeof(ILogger))]
public class Logger : ILogger
{
}

Part Discovery / Inheritance

In MEF Preview 4, a part by default would be discovered if there was an export on the most derived class. Additionally a part author could decorate the part with the CompositionOptions attribute, and set the DiscoveryMode property to change the behavior. Regardless of which mode was chosen, we would always look at all exports in the inheritance chain and discover them once we determined a part was a part.

We heard quite a bit of feedback both internally and externally around both the naming of the different enum values for DiscoveryModel, and the resulting behavior.

Default Behavior

By default we will only look only at the most derived type to discover exports. If exports are found, then we will do a deep dive to discover all imports wherever they live in the hierarchy, including interface members. This is because imports are an implementation detail of a part, and the part many not function without them.

In the example, we will discover a single export of IOrderScreen, and the OrderList import.

[Export(typeof(IOrderScreen))]
public abstract class OrderScreen : IOrderScreen
{
 [Import]
 public IOrderListView OrderList { get; set; }
}

[Export(typeof(IOrderScreen))]
public class NorthwindOrderScreen : OrderScreen
{
}

Below, we will not discover anything, as the export is on the base.

[Export(typeof(IOrderScreen))]
public abstract class OrderScreen : IOrderScreen
{
}

public class NorthwindOrderScreen : OrderScreen
{
 [Import]
 public IOrderListView OrderList { get; set; }
}

Inherited exports behavior – using the PartExportsInheritedAttributed

The PartExportsInherited attribute allows declaring on a base class or an interface that it’s exports should be discovered by inheritors / implementers. This replaces the previous [CompositionOptions(DiscoveryMode = DiscoveryMode.Always)]. This basically allows you to define a template for a part, and is useful hiding MEF’s implementation details from a part author. Implementers can further extend on what has been provided, such as adding their own exports and imports.

There’s a few guidelines / caveats to using this feature.

• PartExportsInheritedAttribute only makes the exports of the type it decorates discoverable by inheritors. It does not make exports of derivers of that type discoverable. The deriver itself needs to be decorated in order to make its exports discoverable.
• We will ignore loose ExportMetadata attributes (and custom metadata attributes) that do not have an associated export.
• If an inheritor overrides a member from the base which has an export, and decorates the overridden member with an export, two exports will be discovered, even if the contract names are the same. This means if you expect implementers to add their own metadata, this might not be the best option, or at least you will need to provide special handling. We are looking into changing this behavior to have the inheritor override rather than be additive.

In the example below, NorthwindOrderScreen will automatically export IOrderScreen.

[PartExportsInherited]
[Export]
public interface IOrderScreen
{
}

public class NorthwindOrderScreen : IOrderScreen
{
}

Below NorthwindOrderScreen exports 2 instances of IOrderScreen, one with metadata and one without. The explicit export of IOrderScreen by NorthwindOrderScreen is in addition to the existing export.

[PartExportsInherited]
[Export]
public interface IOrderScreen
{
}

[ExportMetadata("Customer", "Northwind"]
[Export(typeof(IOrderScreen))]
public classNorthwindOrderScreen :IOrderScreen
{
}

Ignoring part discovery – Using the PartNotDiscoverableAttribute

This attribute replaces replaces [CompositionOptions(DiscoveryMode.Never).  It is useful for when a part author has parts that they do not want to be automatically discovered by catalogs and .  The scenarios where this make sense are where you either have a part which you will manually compose outside of a catalog, or you have a part that you intend for inheritors to use in a catalog but which cannot be made abstract.

In the sample below,  class OrderScreen will be ignored by the catalog. However, when class NorthwindOrderScreen is scanned, we will discover IOrderScreen, and will satisfy the OrderList import.

[PartNotDiscoverable]
[PartExportsInherited]
[Export(typeof(IOrderScreen))]
public class OrderScreen : IOrderScreen
{
 [Import]
 public IOrderListView OrderList { get; set; }
}

public class NorthwindOrderScreen : OrderScreen
{
}

Changes to collection imports

ImportMany attribute

We’ve added a new [ImportMany] attribute. This attribute is to be used for specifying that an importer is expected to import all parts that match the contract. This attribute is equivalent to using [Import] on collections today. By our next release however, [Import] will change to mean import an exported collection.

Once we make the switch, the [ImportMany] attribute will be required on both properties and importing constructor parameters in order to get the same behavior as today.

As an example let’s take an importer of a collection of IOperation contacts.

With the new build the following will have identical results.

[Import]
public IEnumerable<IOperation> Operations { get; set; }

[ImportMany]
public IEnumerable<IOperation> Operations { get; set; }

In the next drop,  the behavior of the first will change. Instead of importing all exports of IOperation, it will instead be looking for an exporter of IEnumerable<IOperation>. The main reason for this is to support custom collections. Today for example the following code won’t yield the expected results if IRepository<T> is an ICollection<T>.

[Import]
public IRepository<Customer> Customers { get; set; }

What the part author is intending here is to import a repository of customers. The repository as a whole is the import. Instead what will happen today is the container will set Customers to a new instance of Repository<Customer>. It will then query for all the Customer exports and add them to the Repository.

Array imports

In this release we’ve added support for array imports.  This allows the following syntax to be supported.

[ImportMany]
public IOperation[] Operations { get; set; }

Future Deprecation of ExportCollection<T> and ExportCollection<T,M>

In the next preview (not this one) we will be removing ExportCollection from MEF. The reasoning is that ExportCollection does not introduce any additional functionality, and we don’t want to introduce a collection which is solely there for syntactic sugar. The alternative recommended syntax for doing ExportCollection will be to use the new array support. Using the new syntax is significantly more compact than using ExportCollection, and is also less cryptic as arrays are very clear as to how they function.

[ImportMany]
public Export<IOperation>[] Operations { get; set; }

[ImportMany]
public Export<IOperation, IOperationMetadata>[] Operations { get; set; }

Additionally we support custom derived collections, thus you can easily create your own collections that provide the sugar.

Typed Import/Exports

In Preview 4, MEF discovers exports based on contract name, and not based on the type of the exporter. This means that an export can actually have an incompatible type with the importer. At the time when the export is set on the import, if the cast fails, then an exception is thrown.

This new feature in Preview 5 enables MEF to match imports and exports on contract AND type. When a part imports a contract, it additionally specifies the specific type that it is looking for. Only those exporters that also match on the importer type are matched. This means instead of failing, if the exporter’s type is incompatible, then it will not be matched.

In most cases, the type specification is done implicitly, and does not require any work on the part of the part author. The overloads of Export and Import that used to take Type as param, will now in addition to converting to the string contract representation, also embed the type information.

For example the code below works as it would have before.

[Export]
public class Editor
{
 [Import]
 public ISpellChecker SpellChecker { get; set; }
}

[Export(typeof(ISpellChecker))]
public class SpellChecker : ISpellChecker
{
}

However code that uses pure string contracts, will yield a different result than it would have before. The SpellChecker below will not be imported into the Editor.

[Export]
public class Editor
{
 [Import("SpellChecker")]
 public ISpellChecker SpellChecker { get; set; }
}

[Export("SpellChecker")]
public class SpellChecker : ISpellChecker
{
}

[Export("SpellChecker",typeof(ISpellChecker))]
public class SpellChecker : ISpellChecker
{
}

[Export]
public class Editor
{
 [Import("EditorContracts.SpellChecker",typeof(object))]
 public EditorContracts.ISpellChecker SpellChecker { get; set; }
}

public delegate bool ValidatorDelegate<T>(T value);

public class CustomerValidator
{
  [Export(typeof(ValidatorDelegate<Customer>))]
  public bool Validate(Customer customer)
  {
    return customer.Name != null && customer.Name != String.Empty;
  }
}

[Export]
public class CustomerView
{
  [ImportingConstructor]
  public CustomerView(ValidatorDelegate<Customer> validator)
  {
  }
}

public class Application
{
  private DirectoryCatalog catalog;

  //triggered by the user clicking the Refresh button on the addins page
  public void Refresh()
  {
    catalog.Refresh();
  }
}

Removal of caching / new infrastructure

We found several issues with our previous caching design which was black box. We’ve completely removed the old caching apis. We’ve added in a new API that can be used for caching, however we are not including a default caching implementation within MEF itself. We will however be shipping a caching API sample which illustrates how to use the APIs. The new APIs provides the further benefit of making it much easier to build new programming models that utilize reflection for are reflection based.

We’ve now introduced a new set of classes in System.ComponentModel.Composition.ReflectionModel that can be used by cache implementers. ReflectionModelServices provides a set of static helper methods that can be used for manufacturing part definitions from a cache store.

Creating a functional programming model for MEF

Disclaimer: This is a prototype, not production ready code. It is more illustrative of what you can do with MEF, and how things work under the covers.

The other day I was working one of our internal partners who is looking to integrate MEF into their framework stack. One of the interesting ideas that came out of that discussion was around a need they had to add POCO instances directly to a catalog (in addition to using our standard attributed part).

Now MEF supports the ability to add existing instances to the container through the AddExportedObject method, but that did not suit their need.  Instead they wanted these instances to actually be parts. Primarily because the types they were instantiating were not attributed MEF parts, and they did not want to incur the reflection cost.

I started thinking down the path of creating a programming model where the part definitions are actually wrapping instances. What I mean by programming model, is teach MEF a new way to represent parts, exports and imports. In MEF parlance, a programming model is almost like a type system, though far less complex. For more on programming models themselves, i'd refer you to Daniel's nice post on programming models themselves here.

Out of the box, MEF ships with an attributed programming model, however others can be created by implementing on top of a set of abstractions we call the primitives. What’s really nice about MEF is that all of the models can safely interact with one another in the container.

Building an instance model however is problematic in that parts in MEF should be lazily instantiated. I realized (probably due much influence of my Autofac workmate Nick Blumhardt) that the right way to go was not build an instance model, but rather build a model for representing functions as parts. And so, the FuncCatalog was born.

FuncCatalog

The FuncCatalog allows you to add functions directly to itself, and to associate those functions with a type to be exported. Each function gets a single parameter which is an ExportProvider. This allows you to build up an instance within your function, that can have constructor parameters pulled from the EP, similar to the registration style in Autofac.

Below is a sample acceptance test that illustrates how this works. In the example I am adding a function to funcCatalog which creates an OrderService. In the OrderService’s constructor, I am passing in an ILogger that I retrieved from the Export Provider.

[TestMethod]
public void FuncPartCanImportFromTheContainer()
{
    var catalog = newAggregateCatalog();
    catalog.Catalogs.Add(newTypeCatalog(typeof (AttributedLogger)));

    var funcCatalog = new FuncCatalog();
    funcCatalog.AddPart<OrderService>(
        ep => new OrderService(ep.GetExportedObject<ILogger>()));
    catalog.Catalogs.Add(funcCatalog);
   
    var container = new CompositionContainer(catalog);
    var batch = new CompositionBatch();
    batch.AddExportedObject<ExportProvider>(container);
    container.Compose(batch);
   
    var service = container.GetExportedObject<OrderService>();
    Assert.IsNotNull(service.Logger);
}

In terms of what the code is doing, I am first creating an aggregate catalog (a catalog of catalogs). Then I am adding to it a type catalog which contains an AttributedLogger (that is a logger with an ExportAttribute on it). Next I am creating a FuncCatalog which I am adding the new func part to. Notice how the lambda passes ep.

        ep => new OrderService(ep.GetExportedObject<ILogger>())

Next I am creating the container, and then adding itself to itself (via the batch) with ExportProvider as the contract. This is to allow the FuncPart to import the ExportProvider so it can use it to resolve other dependencies.

Finally I am asking the container to grab me an OrderService, which I am then checking to see if there is a logger present. Remember the logger actually was an attributed part that was added through MEF’s normal catalogs. So right here we have an illustration of multiple models living side by side.

How is it done?

As I mentioned above, I created a custom programming model. Depending on the model, this can be a lot or a little work. Andreas Haakanson has a project on CodePlex that simplifies creating such programming models, but that is for another overdue post :-)  Below I’ll detail the approach I took in order to give you a view into how programming models work.

Programming model players

Below are the main abstractions you  need to implement for building your own programming model which live in the System.ComponentModel.Composition.Primitives namespace.

Here’s  a quick summary of the responsibilities for each of these classes.

• ComposablePartDefinition – Schema for a part, analogous to a type in many respects. As a matter of fact in our default Attributed Programming Model (APM), the Composable Part Definition wraps a type. Aside from being schema, the other main responsibility for CPD is to create a composable part.
• ImportDefinition – An import definition is carries information about a contract that a part needs, it’s import. In the APM, import definitions are created for properties decorated with the ImportAttribute, or constructor parameters if the constructor has an ImportingConstructorAttribute.
• ExportDefinition - An export definition carries information about a contract that a part offers up, it’s export. In the APM, an export definition is created for each member that is decorated with the ExportAttribute. This includes the class itself, or methods / properties.
• ComposablePartCatalog – The catalog aggregates part definitions, which the container will query during composition in order to find available exports. The catalog does not create any parts, it only returns requested definitions. In our APM, the catalogs read assemblies and types in order to create these definitions.
• ComposablePart – The part itself. If the CPD is the schema, then the part is the instance. This does not mean it is the actual instance that is being returned from the container. No. The part contains an instance which will get created only at the time when the first exported object actually pulled from it. There’s a difference between an export and an exported object, but I’ll resist to go deeper and try to explain it for now.

In my new model I implemented FuncCatalog, FuncExportDefinition, FuncPart, and FuncPartDefinition. I did not need a new ImportDefinition as the standard one was fine. Also my parts only support a single import, namely an ExportProvider.

FuncExportDefinition

Ok, so first stop is to build a FuncExportDefinition. In my case I need a definition that carries a custom function. So here it is.

public class FuncExportDefinition : ExportDefinition
{
    public FuncExportDefinition(string contractName, 
        Func<ExportProvider,object> factory) :
        base(contractName, null)
    {
        Factory = factory;
    }

    public Func<ExportProvider, object> Factory { get; private set; }
}

Pretty simple, I basically allow associating a contract and an additional func which accepts an ExportProvider as a parameter and returns object. Now notice the second parameter I pass to the base class is null. That represents metadata. I decided for simplicity to not allow support of metadata, though that could easily be added back.

FuncPartDefinition

Now that I can define Funcs as exports, I need a PartDefinition which can carry that information, and manufacture a part. This one was a bit tricky, below are the important pieces.

public abstract class FuncPartDefinition : ComposablePartDefinition
{}

public class FuncPartDefinition<TContract> : FuncPartDefinition 
{
    private static ContractBasedImportDefinition 
        _exportProviderImportDefinition;


    static FuncPartDefinition()
    {
        string importContractName = CompositionServices.GetContractName
            (typeof(ExportProvider));
        _exportProviderImportDefinition = new 
            ContractBasedImportDefinition(
            importContractName, null,
            ImportCardinality.ZeroOrOne, false, false,
            CreationPolicy.Any);
    }

    public FuncPartDefinition(Func<ExportProvider, object> factory)
    {
        _exportDefinitions.Add(new FuncExportDefinition
            (CompositionServices.GetContractName(typeof(TContract)),
            factory));
        _importDefinitions.Add(_exportProviderImportDefinition);
    }

    public FuncPartDefinition()
        : this((ep)=>(TContract) Activator.
            CreateInstance(typeof (TContract)))
    {
    }

    public override ComposablePart CreatePart()
    {
        return new FuncPart<TContract>(this);
    }
}

As I mentioned earlier, my func parts need to access an ExportProvider so they can retrieve things from the container. The question was how to do this. I could make the ExportProvider a parameter on the constructor of the part. Then however, I would have to pass it in to the catalog, which would pass it in to the part. This is further problematic as catalogs can actually be passed to several containers. So I decided the easiest way to do it, was to have the part actually import the ExportProvider through MEF’s composition, that is similar to the way I would import on an attributed part through a constructor param.

In order to do this, all I need is to add a single ImportDefinition to every part instance. The static constructor of FuncPartDefinition does the work of creating this definition. The definition never changes, and it is cheaper to create it once, which is  why I made it static.

The instance constructor accepts the  func. It then goes and creates a FuncExportDefinition which it adds to the collection of export definitions. Next it adds the export provider import. I added an additional overload just as a convenience which simply creates the type. Probably just being gratuitous ;-)

Finally the CreatePart method goes and creates a FuncPart passing in the definition (the part’s schema) which leads us to our next suspect.

FuncPart

This guys job is to pass back export definitions of our func, to the container on request. Those definitions can later be activated to invoke the actual func.

public class FuncPart<TContract> : ComposablePart 
{
    private FuncPartDefinition<TContract> _definition;
    private ExportProvider _provider;

    public FuncPart(FuncPartDefinition<TContract> definition)
    {
        _definition = definition;
    }

    public override object GetExportedObject(ExportDefinition definition)
    {
        var funcExportDefinition = definition as FuncExportDefinition;
        return funcExportDefinition.Factory(_provider);
    }

    public override void SetImport(ImportDefinition definition, 
        IEnumerable<Export> exports)
    {
        _provider = exports.First().GetExportedObject() as ExportProvider;
    }

    public override IEnumerable<ExportDefinition> ExportDefinitions
    {
        get { return _definition.ExportDefinitions; }
    }

    public override IEnumerable<ImportDefinition> ImportDefinitions
    {
        get { return _definition.ImportDefinitions; }
    }
}

When the part gets constructed, it stores a copy of it’s definition. The ExportDefinitions and ImportDefinitions delegate directly to the part definitions members. In the SetImport method I have got a bit of a hack going on. This method will get called by the composition engine to set the imports. In the method, I am grabbing the first export passed in and then assuming it is the export provider. This is actually an OK assumption in my case, since there is only a single import that I allow in my programming model. However, I really should be verifying that it is passing me the right definition.

And just when you thought I was done with hacks, another one :-) GetExportedObject takes the definition passed to it and assumes it is a FuncExportDefinition. Again, this is an ok assumption in the cases where I am using it, but it should be more bullet proof and not make those assumptions. Anyway, once it has the definition  unwrapped, it grabs the factory and invokes it passing in the provider.

FuncCatalog

Last stop on our journey is the func catalog.  He’s actually pretty straightforward. I added a few convenience methods on him so that you don’t have to create parts a head of time.

public class FuncCatalog : ComposablePartCatalog,     
    INotifyComposablePartCatalogChanged
{
    private List<ComposablePartDefinition> _parts = new List<ComposablePartDefinition>();

    public FuncPartDefinition AddPart<T>(
        Func<ExportProvider, object> factory)
    {
        var addedParts = new List<FuncPartDefinition>();
        var definition = new FuncPartDefinition<T>(factory);
        _parts.Add(definition);
        
        addedParts.Add(definition);
        OnChanged(addedParts);
        return definition;

    }

    public void AddParts(params FuncPartDefinition[] parts)
    {
        _parts.AddRange(parts);
        OnChanged(parts);
    }

    public void RemoveParts(params FuncPartDefinition[] parts)
    {
        _parts.RemoveAll(c => parts.Contains(c));
        OnChanged(parts);
    }
    
    public override IQueryable<ComposablePartDefinition> Parts
    {
        get
        {
            return _parts.AsQueryable();
        }
    }

    private void OnChanged(IEnumerable<FuncPartDefinition> parts)
    {
        var args = new ComposablePartCatalogChangedEventArgs(
            parts.Cast<ComposablePartDefinition>());
        Changed(this, args);
    }

    public event EventHandler<ComposablePartCatalogChangedEventArgs> 
        Changed = delegate { };
}

First thing is the catalog implements INotifyComposablePartCatalogChanged. This is to allow the catalog to notify the container when parts are added or removed from it. Implementing a catalog only requires to to override one property, namely the Parts property, which as you can see returns a queryable of Parts. The rest of the methods are concerned with the mutability of the container. Now catalogs do not have to be mutable, as you can handle loading up all the part defs in the constructor. However, for usability sake, I wanted mine to  be mutable. This means that I need to have methods for adding, removing, and I need to support notification.  The code should be pretty easy to follow.

So what have we learned?

Well first thing we learned that MEF supports alternative programming models other than the attributed model we ship in the box. Next, we learned about the key concepts involved in authoring your own programming model. In this case the model was very, very simple, which is why I was able to cover it in a single post. I am not recommending every one go out an author their own model. Lastly we saw learned that parts can be functions :-)

If your interested in other applications of programming models in MEF, I recommend you check out Nick Blumhart’s posts on Ruby. Also don’t forget to check out Andreas’s provider based programming model on MefContrib.

Why doesn’t MEF support open-generics for exports? Because MEF is not type based.

I get this question all the time. Recently it popped it’s head on the forums in this thread (which I referred to in my last post)

In that thread, Andrew was asking why the following does not work?

[Export(typeof(IDomainController<>)), CompositionOptions(CreationPolicy = CreationPolicy.Shared)]

public class DomainController<T> : IDomainController<T>

{

  /* some code here */

  [ImportingConstructor]

  public DomainController(IRepository repository)

  {

    _Repository = repository

  }

} 

That is he wants to export a generic IDomainController with the idea the closed generic versions like IDomainController<Customer> could be imported.

He also rightly observed that many of the IoC containers that exist support this functionality including Unity. (I chuckled when I heard that because I was in p&p when we first wrote Unity) However, it is a valid question, why don’t we support it? This is one of those questions that I, Hammett and Nick all asked when we joined the team.  We do support closed generics without a problem, but not open. Why?

As you dig into the details, it quickly becomes clear. The root of the answer is that MEF is not type based, it is contract based. Below are the details from my response…

MEF parts relate on contracts which are strings, not types. To illustrate, see the code below.

namespace Orders {

  public interface IOrderRepository {}

  [Export(typeof(IOrderRespository))]

  public class OrderRepository : IOrderRepository {

  }

}

Although I have passed typeof(IOrderRepository) to the export, that is just a convenience that gets converted to a contract name of "Orders.IOrderRepository". The same applies to the importer...

[Export]

public class OrderService(

  [Import]

  public IOrderRepository Repository {gets;set;} 

)

The import on IOrderRepository also gets converted to a contract name of "Orders.IOrderRepository". During composition, the container is able to satisfy the import as the 2 contracts match. In the same way we support closed generics, so this code....   

public interface IRepository<T> {}

[Export(typeof(IRepository<Order>))]

public class OrderRepository : IRepository<Order> {

}

[Export]

public class OrderService(

  [Import]

  public IRepository<Order> Repository {gets;set;} 

)

will work because the OrderRepository is exporting the contract name "Orders.IRepository<Order>" and the OrderService is importing the same contract.

However, this is what it looks like if we try the same with open generics.

public interface IRepository<T> {}

namespace Utils {

  [Export(typeof(IRepository<>))]

  public class Repository : IRepository<T> {

  }

}

[Export]

public class OrderService(

  [Import]

  public IRepository<Order> Repository {gets;set;} 

)

Now the contract names will be different. The exporter will have a contract of  "Utils.IRepository<>" and the importer will have a contract of "Utils.IRepository<Order>".

It is a simple match up, that breaks down in the open-generics case. This is because fundamentally, MEF is not matching on type.

There are theoretical ways to address it, but they are all very ugly, require a lot of string parsing and really go against the grain of MEF’s core design.

Recomposition and constructors

Yesterday on twitter I was expressing how far behind I am  on blogging, and how I don’t know where to start. Ayende (Oren) responded by saying “Just start bloggging!” So here goes.

Ben Hall pinged me over IM yesterday asking about recomposable imports on constructors, and whether or not they work? Natively MEF does not support this, but I did find a workaround. More about this later as the answer won’t make any sense without first understanding what in the heck recomposition is.

Recomposition

Recomposition is an opt-in feature of MEF that allows imports to be updated whenever there are changes in the container that relate to the availability of exports of that contract, you can think of this as live imports.

Recomposition is one of those capabilities that is useful for systems that dynamically change after the app is running. The change does not only have to be due to new assemblies appearing, it could due to contextual changes in the application such as a user updating their profile, or the application moving into a different state.

For example below I have a part that imports a collection of ILoggers.

public class LoggerManager {

  [Import(AllowRecomposition = true)]

  public IEnumerable<ILogger> Loggers {get;set;}

}

Notice the import is marked with AllowRecomposition = true. This means that after initial composition, the Loggers collection should be updated should new loggers either be manually added (or removed) to the container, or appear in a catalog.

Recomposition is not only useful for collections, it can also be for single imports, which was Ben’s case. That is I could have a single logger import which is also recomposable. That means that if someone replaces the single logger in the system, then the part which imports will automatically be updated.

public class OrderProcessor {

  [Import(AllowRecomposition = true)]

  public ILogger logger {get;set;}

}

Now the OrderProcessor expects to see a single ILogger. That logger however is recomposable such that when the logger is replaced, it will get automatically updated.

Constructors

Now that we understand what Recomposition is, we can get to Ben’s question. Are constructors parameters supported? The answer is no (at least not without help). The reason is that recomposition on MEF is connected to the part,. Whenever a part has imports that are recomposable, we keep a pointer to the associated ComposablePart within the container. We then monitor changes on that particular contract that the  part is importing. Whenever changes occur, we then update the imports on the part which results in replacing the value of the corresponding property. This works because we can replace the reference. In the case of constructor parameters however, we cannot grab your parameter reference and change it.

OK, so we’re out of luck then?

Initially I thought yes, but then as I thought about it, I realized there is a workaround. That is create a generic Recomposable<T> which itself has an import of T that is recomposable. The class is so simple, it’s not even funny, and here it is.

[Export]

public class Recomposable<T>

{

    [Import(AllowRecomposition=true)]

    public T Value { get; private set; }

}

Now you can do  an import such as the following:

    [Export]

    public class UsesLogger

    {

        [ImportingConstructor]

        public UsesLogger(Recomposable<ILogger> logger)

        {

            _recompLogger = logger;

        }

        private Recomposable<ILogger> _recompLogger;

        public ILogger Logger { get { return _recompLogger.Value; } }

    }

The logger in UsesLogger is now recomposable.

Now there is one caveat. Because MEF does not support open generics (see this thread for more on that), you need to either create closed versions of Recomposable<T> which export the correct contract and have them discovered in a catalog:

[Export(typeof(Recomposable<ILogger))]

public class RecomposableLogger : Recomposable<ILogger> {}

Or you need to use a TypeCatalog and manually add each type that you want to be able to import:

var catalog = new TypeCatalog(typeof (UsesLogger), typeof (Recomposable<ILogger>));

The first approach is what extenders need to do if they were introducing new types that the main app does not now about.

With great power comes responsibility, proceed with caution!

Some of you might be rolling your eyes at this post and thinking, does he expect me to believe that this will just automagically work? And the answer is……NO!

Recomposition does not come for free. I mean you can turn it on for free, but you have to design for it. When we recompose, we completely replace the property reference. There is no inherent thread-safety in this, it just happens (if you opt-in), and you need to prepare for it, and design with recomposition in mind. This means very carefully deciding where to use recomposition, and making sure where it is used, the proper safeguards are put in place. And that could probably be a whole other post.

See it in action

For a sample of recomposition with an overriding logger, see this sample code.

ALT.NET Seattle Day 1 reflection – Learn, Share, Grow

I am sitting here at the opening of ALT.NET Seattle. I look around and see the words "Learn, Share, Grow” on a poster, words I first wrote on the ALT.NET wiki about two months ago. At the time I when I wrote it, I sat back and thought to myself “What is ALT.NET about?”. I had a flurry of thoughts, some positive, and some negative.

At the end of the day I boiled it all down to three fundamentals.  ALT.NET is about learning from others, sharing experiences and growing in the process. Now you may think this sounds all peachy clean and blue sky. Learning and sharing, doesn’t mean we always agree and are on the same page. It doesn’t mean we won’t get frustrated with each other about not getting across our viewpoints, or that our ideas are not being accepted, we will.

What it means is that we recognize the value of the sum total of our experiences. We recognize that we all have something to give, and to take.  Benefitting, doesn’t come for free, but the cost is a worthy investment.

OK, I am ready to make mine. How about you?

Event Aggregation with MEF (with and without EventAggregator)

The title probably sounds like an oxymoron, but it is not. Recently there was a question on our CodePlex forums from Denis Vuyka about whether or not MEF supports anything like EventBroker for pub/sub type communication. Asking such a question makes a lot of sense, as if you are building open-ended systems of extensions, you will run into cases where you need a loosely coupled way to communicate between the parts. For example you might have a dynamically populated navigation bar. like the one in Outlook, where you publish an event to indicate the item selection. The individual subsystems (contacts, mail, etc) all subscribe to that event to receive notifications.

In as much as it make sense, we don’t currently have a specific facility for this, mostly due to the fact that we had bigger fish to fry. However using p&p’s EventAggregator for Prism is a great option. Of course I am biased, as I came from the Prism project before joining the MEF team. EventAggregator is a very simple service that allows writing up publishers and subscribers in a loosely coupled way. It also provides benefits over events in that it is delegate based and supports a weak delegate notion, thus allowing subscribers to not be held-alive by the publisher. Additionally EventAggregator supports providing lambda predicates for subscription, thus allowing subscribers to conditionally handle the notification based on custom filters against the payload. Finally it is thread safe, and can marshall between threads through using overloads on the event Subscribe() method. The EventAggregator also doesn’t have any real dependencies on Prism itself. Because it is so simple, it is very easy to integrate with MEF, thus allowing MEF parts to benefit from it’s capabilities. Sure enough, Denis went and wired up EA to MEF without a hitch, which he posted about here.  Nice job Denis!

Event Aggregation without EventAggregator

Around the same time that Denis was looking at EventAggregator I started thinking about the problem as it relates to MEF. The question I kept toying with was did we really need EventAggregator at all, or could we simply remove it, and instead just expose the events directly. After a bit of chatting and pairing with Julian Dominguez, my former mate from patterns & practices, we realized we actually could. Instead of adding EventAggregator to the container at all, you can simply add Prism events as exports through a catalog which is passed to the container. Once you do, Publisher and Subscriber can easily import the event and access it in a similar fashion. Below are the steps you need to follow, and for which the code is in the attached zip (along with unit tests)

Creating the Event

First create your custom event class by deriving from CompositePresentationEvent and passing your custom args class. Add an Export to the event, and mark it as a shared CreationPolicy, this way all publishers / consumers share the same event instance. For example below I defined a CustomCompositionEvent.

[Export]
[CompositionOptions(CreationPolicy = CreationPolicy.Shared)]
public class CustomCompositionEvent : CompositePresentationEvent<CustomArgs> {}

public class CustomArgs{}

Registering

Next register the event in a catalog. You can use any catalog of your choice, in this case because I am in a unit-test (really an acceptance test as I am just using MEF), I registered manually with a TypeCatalog in a Setup() method. However in a real app, you would probably use a Directory / Assembly catalog.

public void Setup()
{
    var catalog = new TypeCatalog(typeof (Subscriber), typeof (Publisher), typeof(CustomCompositionEvent));
    _container = new CompositionContainer(catalog);
}

Subscribing / Publishing

Once you have registered the event, you can easily import it into any of your parts. Below are my fake publisher and subscriber that I am using for my tests.

  [Export]
  public class Publisher
  {
      [Import]
      public CustomCompositionEvent CustomCompositionEvent { get; set; }
  }

  [Export]
  public class Subscriber
  {
      [Import]
      public CustomCompositionEvent CustomCompositionEvent { get; set; }
  }

To subscribe, call the Subscribe() method on the event, passing in a lambda for the subscriber method. To publish, call the Publish() method on the event and pass the args. See the p&p documentation for more on the params available to both methods.

Both can be seen below in my acceptance test.

[TestMethod]
public void When_event_is_fired_subscriber_gets_notified()
{
    Setup();
    bool eventRaised = false;
    var subscriber = _container.GetExportedObject<Subscriber>();
    var customEvent = _container.GetExportedObject<CustomCompositionEvent>();
    subscriber.CustomCompositionEvent.Subscribe(a => eventRaised = true);
    customEvent.Publish(null);
    Assert.IsTrue(eventRaised);
}

In the code above, I am grabbing a Subscriber instance from the container, which imports the event. I am then grabbing the event directly. I could have actually used the instance of the event that was on the subscriber, but that might look confusing in code, so for clarity, I imported it again directly. This also drives home why it needs to be shared. Next the subscriber subscribes to the event. In this case in my unit test, I am simply setting a boolean when the event is raised, so I can verify it in the test. Finally i am calling publish. In live code, you would probably have the part wire itself up to subscribe in the property setter.

Conclusion

Prism’s EventAggregator infrastructure provides a great way to facilitate loosely coupled communication between parts in your MEF app. You can either use the EventAggregator itself in the container, or you can import CompositePresentationEvents directly. Either approach works well.

CommonServiceLocator for MEF, a service is a service.

Today, I finally got around to uploading a CommonServiceLocator adapter for MEF.

The code is actually quite simple thanks to Chris Tavares providing ServiceLocatorImplBase. (Updated thanks to feedback from several folks)

public class MefServiceLocator : ServiceLocatorImplBase
{
    private ExportProvider _provider;

    public MefServiceLocator(ExportProvider provider)
    {
        _provider = provider;
    }

    protected override object DoGetInstance(Type serviceType, string key)
    {
        IEnumerable<Export<object>> exports;
        string contract = CompositionServices.GetContractName(serviceType);
        exports = key == null ? _provider.GetExports<object>(contract) : _provider.GetExports<object>(key);

        if (exports.Any())
            return exports.First().GetExportedObject();
        
        throw new ActivationException(string.Format("Could not locate any instances of contract {0}", key));

    }

    protected override IEnumerable<object> DoGetAllInstances(Type serviceType)
    {
        var exports = _provider.GetExportedObjects<object>(CompositionServices.GetContractName(serviceType));
        return exports;
    }
}

One thing you’ll notice is that in DoGetInstance, I am calling to GetExports and then manufacturing only the first export that I return. The reasoning for this is that MEF doesn’t have an internal notion of default. If I call GetExportedObject and more than one is returned, it will blow up with a composition exception. The reason is because we have no way of knowing which is the default as we simply grab parts from catalogs (In Preview 3 we actually provided a way to assign defaults through usage of Export Providers however that’s another story for another overdue post :-) ). So instead of calling for a single, I call for multiple which will succeed in all cases.

To make the adapter satisfy the needs of the interface, I decided to apply Krysztof’s rule of thumb which is to take the first one that is returned regardless, as he says, if I walk into a car dealership and say give me a Saab, I don’t care which. Calling for an export instead of an exported object allows me to only instantiate (and load if I am caching) the one that is being returned.

Another thing you will notice is that i am passing object for the type as DoGetInstance only passes a type. The methods for retrieval on MEF’s Export Provider all allow you to pass object as the type, thus allowing me to do what I need to do.

Finally you’ll notice that I have passed in an ExportProvider to the constructor rather than a container. MEF’s composition container is an ExportProvider. ExportProviders are simply for retrieving exports, which matches very well with ISL’s needs and makes it a good fit.

You can download it here along with unit tests. Thanks to Rob Eisenberg for the initiative which encouraged me to actually get it done.

On to CommonServiceLocator itself…and MEF usage

A while ago , I posted about the new CommonServiceLocator library (and IServiceLocator interface) a bunch of us co-designed and p&p developed. The purpose of this library was to allow applications to use a standard interface for accessing services without being bound to a specific service provider / container.

Once the library launched on CodePlex it caused a healthy amt of debate. Plenty of folks were wondering “Why Service Locator?”. Was this an encouragement to throw away DI containers, and take a trip back in time to the good old days of the Service Locator pattern?  Was this the one abstraction to rule them all?

The answer on both accounts was no. In terms of the first question, the primary place where we thought CSL would be used was for actually accessing a DI container.  That being said, nothing on the interface had anything particularly DI-ish about it. Instead it simply contained interfaces for querying for a service. Now, the thing that provides the service which sits behind the interface might very well be an IoC container which constructs components and  performs dependency injection. However, there is nothing about the interface that enforces that so we felt we were staying true to the api itself.  Furthermore when you access an IoC container and call a Resolve / Get method you ARE performing Service Location. As I like to say, ServiceLocator is the Gateway to an IoC container.

Additionally, wee did not intend IServiceLocator  calls to be scattered throughout the application thereby creating a ton of static dependencies. Instead, the intent was that ISL should be used minimally at the root of an application to compose a root graph, or low-level in framework libraries that were reused across applications.

Now on to the second question. IServiceLocator does one thing, resolve services. We deliberately kept the interface focused around the common set of functionality that service locator’s provide. We kept out anything that was container specific, and also kept away from anything other than retrieval. The reasoning for this was so that this interface did not become some uber generic abstraction with a lot of leaky abstractions. Instead we kept short and sweet and focused on a single concern, resolving.

So that leads to MEF. Why should one want to use ISL with MEF? Quite simply because in the same way that IoC containers can provide services, MEF also provides services. How it find them and how they are created is quite different, but in the end a service is a service.

Managed Extensibility Framework Preview 4, a grab bag of goodies.

If you haven’t seen Krys and David’s announcement,  we just released another drop of MEF (I need to hold myself from breaking out in hysterics when I say that ) on CodePlex.

This release contains a whole grab bag of goodies, some of which may initially taste like sour candies (breaking changes), but are sweet in the middle. Most importantly, we’ve done a significant set of enhancements to the MEF codebase around Lifetime, Diagnostics, and Debugging.

Summary of the breaking changes:

• AllowNonPublicCompositionAttribute was removed. It is no longer needed MEF will always look at publics and non-publics.
• ComposablePartCatalog was moved from System.ComponentModel.Composition to System.ComponentModel.Compositioni.Primitives.
• AttributedTypesPartCatalog was renamed to TypeCatalog
• AttributedAssemblyPartCatalog was renamed to AssemblyCatalog
• DirectoryPartCatalog was renamed to DirectoryCatalog
• AggregatingComposablePartCatalog was renamed to AggregateCatalog
• Catalog Caching extensibility API’s have been made internal.
• Mutability APIs on the container have been changed to accept a batch. AddPart and RemovePart methods were removed from the container, and you know must pass a CompositionBatch. The advantage of the batch is that you can now replace a single part in the container with a new part in a single operation. You can also add or remove multiple parts in a single operation. This comes in very handy when you have parts that are recomposing.

New features:

New wiki pages

We’ve (well really Hammett) fleshed out all the TBD topics, plus added a ton of new ones both in our programming guide and the arch section.

Diagnostics and debugging

Since joining the team once thing that I’ve heard over and over is just how bad our debugging experience was. Not only did i hear it…I experience it first hand, some times live on stage :-) Well fortunately Nick and his feature crew (David and Jad/Zhen) spent the last several months focused on righting this wrong.

We’ve made significant improvements in the debugging experience to address some problematic scenarios and deliver a foundation for diagnostics in all MEF features and extensions.

The differences to note are:

• Multiple composition errors are structured in numbered groups each relating to a single root cause
• The ‘causal chain’ (Resulting in: …) traces an issue all the way back to the root action that the application was trying to perform
• The ‘origin path’ (Element: …) describes how each object involved in the scenario came to be in the composition in the first place
• All of this information can be retrieved programmatically from the exception types if necessary

Another difference will be the visualization of exceptions when they occur.

Previously you would see something like the following, which was difficult to navigate.

With the changes, you will now see.

Lifetime Management and Creation Policy

Another concern we heard from customers was around how MEF handles lifetime management for parts. In previous releases we allowed the exporter to declare that a part has either a singleton or factory (transient) lifetime.

• If a transient part implemented IDisposable, it’s resources would be released only when the container was disposed. Singletons would also be disposed in a similar manner. This forces transient parts which could container scarce resources to stay alive. For example if a part contains an unmanaged resource such as a database connection, it will stay open and not get recycled.  This is particularly problematic in server environments such as ASP.NET, where you have a limited number of resources shared by many active sessions. Not having the connections recycled could lead to starvation.
• Another challenge of the previous approach was that the Part had to determine its lifetime policy. As parts are reusable across different apps, it is very likely that a part in one app is a singleton, while in another it needs to be transient. In the same app there are even cases where the same part needs to be transient in one case and singleton in another. In our previous bits, there was no way to address this.

Hammett and team (Wes, Daniel) have been working tirelessly to come up with a solution to this one. Hammett personally was no stranger to this problem form all the work he’s done on the castle stack. Upon joining the team this is one area that he was relentless about finding a solution to. I am thankful for his passion to see this through as there were several times we hit road blocks that seemed insurmountable.

In the new bits we have provided solutions to both of these problems.

• We’ve renamed Singleton and Factory creation policy to Shared / NonShared.
• Parts and Imports can now both declare creation policy. We’ve also added a new policy called Any, which allows a Part or an Import declare that it can work with either policy, i.e. a Part could have a policy of Any, where one  importer of the part’s exports has a policy of NonShared, while another importer has a policy of Shared. The grid below indicates the behavior in different scenarios.

An importer specifies the required policy through the use of the new RequiredCreationPolicy as can be seen below.

[Export]

public class Window : System.Windows.Forms.Form

{

    [Import(RequiredCreationPolicy = CreationPolicy.NonShared)]

    public ExportCollection<IShape> Shapes { get; set; }

    ...

}

• Parts can now be explicitly released from the container by using the Container.ReleaseExport method. This method takes the export and traverses all its references that are non-shared and disposes of them if they implement IDisposable. This prevents what we call viral disposability where every part in the chain must implement IDisposable in order for the deepest child to be disposed. This is an extremely difficult problem for you to manage, and without this feature the likelihood of memory leaks is high.

For example, below you can see where the jobExport is explicitly released from the container.

var container = new CompositionContainer(...);

var jobExports = container.GetExports<IJob>();

foreach(var jobExport in jobExports)

{

     var jobProcessor = jobExport.GetExportedObject();

     jobProcess.Process();

     container.ReleaseExport(jobExport);

}

• A Non-shared part that has a recomposable import will be held conditionally based on the lifetime of its exports. If  the exports are gc’d the part will be released as well.

Be sure to check out the new wiki page on Lifetime for a much more detailed explanation.

This should be the last release where we introduce major API changes. We’re getting into the home stretch now, with Beta 1 around the corner. There’s still time for us to make fixes though so please get your feedback in.

Special thanks to Hammett for making sure this release got out the door and for the wiki page work. Also thanks to David and Daniel for their supporting efforts on the site. And thanks to the whole team for the awesome work they are doing! Lastly thank to you for your feedback!

ALT.NET Seattle 2009 is happening.

If you attended ALT.NET Seattle least year, you might be wondering whether or not it's happening this year.

While at Kaizenconf, a bunch of us wondered the same thing. David Laribee, Chris Bilson, Kelly Leahy and I chatted about hosting another ALT.NET Seattle conference. David basically told us it was time to pass the reigns, if we wanted it to happen, we had to make it happen. Well after returning from the conference, we started brainstorming on the idea at our local ALT.NET monthly open-space and thanks to a bunch of efforts from our local guys....

I am excited to announce that we are actually going to host an ALT.NET Seattle conference this year.

• What: ALT.NET Seattle 2009. (The site is still being worked on. Thanks to Justin Bozonier and Shoshanah Bain for getting the site up).
• Where: Digipen (thanks to Jeff Tucker for lining up the space)
• When: Evening February 27th through March 1st.
• Registration: Opens this tuesday evening at 6PM PST. We're holding on registration to allow the word to spread. There will be a max of 150 attendees. Check the wiki on Tuesday for details.

If you've never been to an Open Space event all I can tell you is it will defy your expectations. This is not simply sitting in a room listening to speaker after speaker. This is an organic conversation that emerges based on an agenda that attendees put together on site. At open-space everyone is a speaker, and everyone is a listener. You just have to experience it.

If you haven't heard about ALT.NET or what you have heard is not appealing, let me tell you from first hand experience what it's about. It's about you. It's about being  a better software developer. It's about learning and sharing. It's about  using all the tools (patterns, practices, frameworks, etc) at your disposal to get the job done. It's about growing. 

The event will be immediately before the MVP summit, rather than after.  We decided to do it earlier for several reasons. First several folks gave us feedback that having it after the MVP summit was tough due to being worn out from the week. Second as a lot of folks are traveling from a-far, having it after meant that we have to cut Sunday pretty short to allow for travel.

Things will be a little different last year than this year.

1. David and Scott are off the hook for organizing it :-)
2. Two full days of sessions (preceeded by an evening kick-off on Friday)
3. We're opening the doors  on attendance, and severely limiting the reserved slots. We want the attendance to be as diverse as possible.  You don't need a secret decoder ring to get in :-)
4. Unfortunately Stephen List (Doc) won't be facilitating as he has a conflict. We're looking for a top-notch facilitator to fill his place.
5. We're looking into having some kind of evening coding challenge. The point here is to do something that's fun and learn at the same time.
6. We're looking into telecasting some of the sessions over livemeeting for folks that can't attend.

If you've been to previous ALT.NET events and have any suggestions, please let us know.

We look forward to seeing you in Seattle (well technically Redmond)  the end of February!

PS: We're also looking for sponsors for the event who are willing to contribute financially. If your company is interested, contact Bobby Johnson who I am sure will be posting on this shortly.

Container-managed applications

OK, so you've past the point of deciding whether or not you will use an IoC container. Then you find out that is just the beginning of the road. How does that container fit in your application design? Should you pass it around, should you have a static accessor, or should all your components be completely free of any dependency on it's existence? How can you create new components dynamically at run-time? How should you design your system to best take advantage the container without becoming coupled to it? This are questions you need to answer irrespective of which container you choose.

My colleague Nick has started yet another blog series to explore these questions and more. He is sharing his experiences based on his work with IoC containers, including designing Autofac. In his first post he explores the place of the container within the design. If you are using an IoC container or thinking about it, I highly recommend it.

http://blogs.msdn.com/nblumhardt/archive/2008/12/27/container-managed-application-design-prelude-where-does-the-container-belong.aspx

Using ExportProvider to customize container behavior Part I

In our last drop of MEF, one of the significant changes was the introduction of the ExportProvider. In this next series of posts, we'll take an in-depth look at what it is, how it's used by the CompositionContainer, and how you can use it to customize the behavior of the container.

Disclaimer: These are not intro to MEF posts, if you don't have a good grasp of MEF concepts, then these posts may be a hard to follow.  Please refer to the MEF Wiki on our Codeplex site, or to community posts (we have some bloggers links on our site) if you need further reference.

Just to give you a preview of what's to come, here are some of the scenarios you can address using Export Providers.

1. Providing a filtered view over exports that come from other providers. This may be a fixed filter, or something more dynamic such as based on application state or a user's role.
2. Providing default exports such as a logger which will get returned whenever a single logger is imported if there are multiple loggers present.
3. Creating a hierarchical model with parent / child containers.
4. Injecting configuration information into parts during construction.
5. Integrating with legacy systems, or other component providers such as an IoC container.

ExportProvider

Parts in MEF carry exports and imports. During composition, the container composes parts and satisfies imports. In order to do this, it queries a series of export providers as can be seen in the diagram below.

ExportProviders have one purpose, drum roll please. They provide exports. Where those exports come from is unimportant to the caller, the main thing is that the exports that returned match the request. 

In the box

As can be seen from the diagram above, we make heavy use of export providers with MEF, and we include several in the box. You can also create your own.

CompositionContainer - The container itself is an Export Provider. This comes in very handy when building topologies of containers such as Parent / Child, or even Two Parent / Child  (yes it is possible).

MutableExportProvider - This provider is primarily an implementation detail of the container. Whenever you manually add parts to the container, those parts are added to an internal part collection within the MutableExportProvider. Whenever the MutableEP is queried, it then queries its parts to find matching exports. It contains a CompositionEngine, which is responsible for satisfying the imports on any of the parts that are added to it.

AdaptingExportProvider - MEF supports contract adapters . This provider queries a set of exports, and then invokes an adapter manager to adapt the exports returned to a different contract. More on this in another post.

ComposablePartCatalogExportProvider - This provider is responsible for retrieving exports from part catalogs. Like the MutableExportProvider, it too contains a collection of parts. However the parts in it's collection are created from the PartDefinitions it queries rather than being explicitly added to it. Also similar to the MutableEP, it too contains a CompositionEngine for satisfying the imports on it's parts.

AggregatingExportProvider - This provider is a composite of other providers that it contains, and is used for providing a topology of EPs. Whenever this provider is queried, it will query the providers within. The internal query behavior varies depending on the cardinality of the ImportDefinition (more on that below) that is passed in.  For now it is sufficient to know that it queries it's children. We will talk more about the behavior in a future post. The container uses an AggregatingEP internally which contains a MutableEP, a CatalogEP and/or a custom provider if one was passed in during its construction. AggregatingEPs can also be nested without a problem.

Under the covers

If you take a look at the ExportProvider API you'll see the following.

At first glance you may be thinking, Wow that looks anything but simple. Majority of these methods are different ways for specifying a set of exports to retrieve, a format to return them, and whether or not it is a single item or a collection that is returned. The GetExport / GetExports methods return lazy instantiated instantiated objects which are of type Export.  The GetExportedObject / GetExportedObjects methods returns the actual instances that  the Exports create.

Fortunately about 95% of the methods are syntactic sugar around one core method which is the only method you need to implement when authoring a custom ExportProvider.

That method takes an ImportDefinition and returns a collection of Exports.

ImportDefinition

You can think of the ImportDefinition as similar to a SQL where clause. It  specifies a filter for which Exports to return. The ImportDefinition has two main components. The constraint is an Expression<Func<ExportDefinition, bool>> and represents the export filter. Cardinality is an enum which specifies the cardinality of the exports, it can be ZeroOrOne (one max, but zero is allowed), ExactlyOne, or ZeroOrMore ( a collection of 0 to N). We'll hold of on talking about the other params for now.

These definitions come from several places. Parts carry import definitions, for example when you decorate a Part with one or more Import attributes, it will have  ImportDefinitions created for each Import as it is picked up by the catalog. The ExportProvider has several public methods that accept definitions as parameters. As the container is an ExportProvider this means definitions may be passed in directly through it's methods. Finally, if any of the overloaded GetExport(s)/GetExportObject(s) methods  on the ExportProvider that do not accept an ImportDefinition are callled, internally an  ImportDefinition will be created.

For example, the snippet below illustrates creating a definition that matches on all exports that use the convention "Service" as a suffix.

This is a very simple constraint, but you can let your mind run wild as to what you can do through an expression.

ContractBasedImportDefinition

As you can see above, the ImportDefinition takes a constraint as an Expression. This is great for cases where we need to support free-form queries such as you can specify with a lamdba. However an expression is essentially opaque unless you want to do some deep parsing. Also expressions are overkill for the simple cases such as declarative imports and exports on a part. This is where the derived ContractBasedImportDefinition comes in. 

This definition allows you to specify both a contract name, and an optional set of metadata keys that should be matched on (keys not values).

Our attributed part model creates these definitions for each Export and Import attribute we find. This is ideal for many cases and it provides the added benefit of making it easier to author an ExportProvider as you don't have to work with expressions for 95% of cases.

Below is an illustration of creating a ContractBaseImportDefinition to match on a logger.

ExportDefinition

Earlier was saw that the ImportDefinition takes a constraint on an ExportDefinition. Whereas the ImportDefinition defines what kind of exports are needed, the ExportDefinition defines the actual export itself.

As you can see it takes two parameters. The contract and a dictionary of metadata.

Contracts are strings

You might be surprised at the fact that contract appears as a string though the ExportAttribute allows you to pass a type. Yes folks, MEF doesn't care about types, under the hood it's all strings. When you pass a type, that type's full name gets pulled out (without the assembly info) and this is the contract. This is a subtlety that actually indicates one of the real powers behind MEF. MEF imports can come from anywhere, Dynamic Languages, XAML, or even a database. The only thing that matters is that the Importer is of a compatible type to cast the Export. However that Export did not at all have to be retrieved based on it's type.

Where does the metadata came from?

Part exports carry metadata. In our attributed part model, metadata is specified in one of two ways. One way is by annotating an Export with an ExportMetadataAttribute, which contains a name, value pair. A second approach is to use a custom metadata attribute.  Below you can see InMemoryCache is exporting metadata defining that it is a non-persistent cache.

Passing in Export Providers

Export Providers are passed in to the container during its construction. There are two overloads on the constructor which accept an EP.

Behind the scenes, an AggregatingExportProvider is created and each of the providers that were passed in is added to it. If a catalog is passed in, then a CatalogExportProvider is created and it is added to the collection of providers. In other words the container doesn't know anything about catalogs, only Export Providers. If you don't want to use catalogs, you don't have to.

Source Provider

Some export providers need to have access to other EPs in the outside world. In particular these are the EPs that need to satisfy their imports, such as the CatalogExportProvider and MutableExportProvider. For example, when the CatalogExportProvider is queried for an export such as a ContactView, and that view imports a NotesView, how can the CompositionEngine find the NotesView. If it only looks within itself, then it won't find NotesView exports that may exist in another catalog, or in a different provider such as the MutableEP.

For this reason these providers have a property called SourceProvider which is set to a provider that gives them that access. 9 times out of 10, this property is either set to the container itself, or to the container's AggregatingEP. It is important to remember that when you create CatalogExportProviders yourself and pass them in to the container, you need to set the SourceProvider to the Container after the Container has been created.

What's next?

Now that we've got the basics down pat we can move on to seeing different ways we can author and use Export Providers. In the next post, we'll take a look at how to create a filtered EP. We'll also look at how to design EPs in a test-driven matter, including a few APIs I whipped up to aid in testing EP functionality using tools such as Rhino Mocks.