David Ebbo's blog

Fun with T4 templates and Dynamic Data

T4 templates have been a pretty popular topic lately.  If you have no idea what they are, don’t feel bad, I didn’t either only a couple weeks ago!  In a nutshell, it’s a simple template processor that’s built into VS and allows for all kind of cool code generation scenario.  For a bunch of information about them, check out the following two blogs:

• GarethJ’s blog
• Oleg Sych’s blog

The basic idea is that you can drop a hello.tt file into a project, and it will generate a hello.cs (or hello.anything) file via its template.  The template syntax of .tt files is very similar to ASP.NET’s <% … %> blocks, except that they use <# … #> instead.

For a cool example of what you can do with T4, check out this post from Danny Simmons, which  shares out a .tt which generates all the Entity Framework classes automatically from an edmx file.  What’s really nice is that you can easily customize the template to generate exactly the code that you want, instead of being locked into what the designer normally generates.

So how does that have anything to do with Dynamic Data?

One weakness we currently have with Dynamic Data is that we don’t have a great way to get started with a custom page.  Suppose you start out with a web app in full scaffold mode, and find the need to customize the Details page for Products.  The ‘standard’ procedure to do this is:

• Create a Products folder under ~/DynamicData/CustomPages
• Copy ~/DynamicData/PageTemplates/Details.aspx into  it
• Modify the copied file to do the customization you need

While this generally works, it has one big weakness: you need to start your customization from a ‘generic’ file which knows nothing about your specific table.  e.g. you start out with a <asp:DetailsView> control that uses AutoGenerateRows to generate all the fields.  So if you want to do any kind of real UI customization, you’ll likely have to do a bunch of repetitive steps, e.g.

• Get rid of the DetailsView and replace it with a FormView (or ListView) so you can get full control over the layout
• In the FormView’s item template, you’ll then need to add one <asp:DynamicControl> for each field that you want to display, along with formatting UI between them (e.g. <tr>/<td>).

A T4 template to the rescue

In a few hours, I was able to put together a quick T4 template that replaces most of those repetitive steps.  The steps using this template become simply:

• Create a Products folder under ~/DynamicData/CustomPages (as above)
• Drop Details.tt into it, and instantly it generates the aspx file, all expanded out with a FormView and all the DynamicControls! This .tt file is attached at the end of this post.

While this is cool, there is one thing that is a bit strange about it: you’re left with a .tt file you don’t want in your project:

And since details.tt file generates details.aspx file, you can’t really change the aspx until you delete the .tt file.  Unfortunately, VS makes this more difficult than you would think because the two files are linked to each other.  The best steps I found to delete it are:

• Right click the .tt file and choose Exclude From Project
• Right click the Products folder in open it in Windows Explorer
• Delete the .tt file in the explorer
• Back in VS, click Show All Files in the solution explorer.  You should see the aspx
• Now right click it and Include it in the project

I know, that’s painful!!  Ideally, what we really want to do is use the .tt do to a one time transform, but never actually have it be part of the project.  At this point, I’m not yet sure how to do this, but I’m hopeful that there is a way.  Suggestions are welcome! :)

How does the template work?

In order to perform its task, the template needs to figure out what all the entity type’s fields are for the custom page.  Doing this is a bit non-trivial, and what I do here is not as clean as it could be, as this is just a quick prototype.  Here is a brief description of what it does:

• Based on the folder name, it knows what the Entity Set name is (e.g. Products)
• It then locates the app’s assembly in bin (caveat: this only works in Web Apps, not Web Sites!)
• It copies it to a temp location and loads it from there, to avoid locking the bin assembly
• In the assembly, it finds the DataContext/ObjectContext (it handles both Linq To Sql and Entity Framework)
• From the context and the entity set name, and can know what the Entity Type is
• Once it has that, it knows the fields and can generate the page!

If you look at the t4 files, you’ll see all this logic at the end.  That part is certainly quick and dirty, and could use some rewriting.  But what comes before it is a pretty clean template that’s easy to tweak to your needs.  e.g.

                    <table class="detailstable">
<# foreach (var prop in data.Properties) { #>
                        <tr>
                            <th>
                                <#= prop.Name #>
                            </th>
                            <td>
                                <asp:DynamicControl DataField="<#= prop.Name #>" runat="server" />
                            </td>
                        </tr>
<# } #>
                    </table>

As you can see, it looks very much like inline code in an aspx file, except that instead of executing at runtime on the web server, it executes in Visual Studio to generate a file that becomes part of the project.

What about all the other views?

So I’m only talking about Details.tt/Details.aspx.  What about the other Dynamic Data views: List, Edit, Insert?  Well, those would all work in pretty much the same way, and I simply haven’t had a chance to get to them.  For now, I just wanted to get this first T4 file out as a proof of concept that shows the kind of things that can be done.  Contributions are welcome! :)

Conclusion

Clearly, a lot of what I describe here is pretty raw, and it’s only a very early prototype.  But I think it should be enough to convey the potential power of design time file generation via T4 templates.  Later, I’d like to get better integration with VS.  e.g instead of dropping the .tt file in the project and later having to delete it, it’d be nice to get a custom action by right clicking on the CustomPages folder, that would let you generate the aspx file.  It would still use T4 under the cover, but you wouldn’t have to see it unless you want to.

Hopefully, I’ll  be writing more about this in the future if there is interest.

Using User Controls as Page Templates in Dynamic Data

Dynamic Data has the concept of Page Templates, which are pages that live under ~/DynamicData/PageTemplates and are used by default for all tables.  Recently a user on the forum asked whether they could use User Controls instead of Pages for those templates.

To me, this means potentially two distinct scenarios, and I tried to address both in this post:

1. Using routing: in this scenario, you still want all your requests to go through the routing engine, but have the user control templates somehow get used.  The URLs here would still  look identical to what they are in a default Dynamic Data app, e.g.  /app/Products/List.aspx (or whatever you set them to be in your routes).
2. No routing: in this scenario, you want the URL to go directly to a specific .aspx page, and then have that page do the right thing to use Dynamic Data through the User Control templates.

Creating the User Controls

First, let’s look at what will be the same for both cases.  Under ~/DynamicData/PageTemplates, I deleted all the aspx files (to prove that they’re not used), and instead created matching ascx files (i.e. User Controls).  Those user controls contain essentially the same things that were in the pages, minus all the ‘outer’ stuff (e.g. things that relate to the master page).

Making it work using routing

Now let’s look specifically at case #1 above, where we want to use routing.  First, we need to make a small change to the route to use a custom route handler:

routes.Add(new DynamicDataRoute("{table}/{action}.aspx") {
    Constraints = new RouteValueDictionary(new { action = "List|Details|Edit|Insert" }),
    Model = model,
    RouteHandler = new CustomDynamicDataRouteHandler()
});

Note how we set RouteHandler to our own custom handler type.  Now let see what this type looks like:

public class CustomDynamicDataRouteHandler : DynamicDataRouteHandler {
    public override IHttpHandler CreateHandler(DynamicDataRoute route, MetaTable table, string action) {
        // Always instantiate the same page.  The page itself has the logic to load the right user control
        return (IHttpHandler)BuildManager.CreateInstanceFromVirtualPath("~/RoutedTestPage.aspx", typeof(Page));
    }
}

It’s basically a trivial handler which always instantiates the same page!  That may look strange, but the idea is that all the relevant information is carried by the route data, which that page can then make use of.  Now let’s see what we’re doing in this one page.  It’s itself pretty trivial, with just a bit of logic in its Page_Init:

protected void Page_Init(object sender, EventArgs e) {
    // Get table and action from the route data
    MetaTable table = DynamicDataRouteHandler.GetRequestMetaTable(Context);
    var requestContext = DynamicDataRouteHandler.GetRequestContext(Context);
    string action = requestContext.RouteData.GetRequiredString("action");

    // Load the proper user control for the table/action
    string ucVirtualPath = table.GetScaffoldPageVirtualPath(action);
    ph.Controls.Add(LoadControl(ucVirtualPath));
}

The first 3 lines show you what it takes to retrieve the MetaTable and action from the route date.  Then the next couple lines load the right user control for the situation.  GetScaffoldPageVirtualPath is a simple helper method which has logic to locate the proper ascx:

public static string GetScaffoldPageVirtualPath(this MetaTable table, string viewName) {
    string pathPattern = "{0}PageTemplates/{1}.ascx";
    return String.Format(pathPattern, table.Model.DynamicDataFolderVirtualPath, viewName);
}

And that’s basically it for the routed case!

Making it work without routing

Now let’s look at the non-routed case.  Here, the routes are not involved, so the URL goes directly to a page.  That page is similar to the one above, but with some key differences.  Here is what it does in it Page_Init:

protected void Page_Init(object sender, EventArgs e) {
    // Get table and action from query string
    string tableName = Request.QueryString["table"];
    string action = Request.QueryString["action"];
    
    // Get the MetaTable and set it in the dynamic data route handler
    MetaTable table = MetaModel.Default.GetTable(tableName);
    DynamicDataRouteHandler.SetRequestMetaTable(Context, table);

    // Load the proper user control for the table/action
    string ucVirtualPath = table.GetScaffoldPageVirtualPath(action);
    ph.Controls.Add(LoadControl(ucVirtualPath));
}

The key difference  is that it can’t rely on route data being available, so it needs to get the table and action information from  somewhere else.  You can come up with arbitrary logic for this, but the most obvious way is to just use the query string (e.g. ?table=Products&action=List).

Then  it does sort of the reverse of the case above, and sets the MetaTable into the route handler.  Even though routing is not used, Dynamic Data tries to get the MetaTable from DynamicDataRouteHandler, so having it there allows many things to just work.

And finally, it loads the user control using the exact same steps as above.  And that’s that!

The code is attached below.

Creating a ControlBuilder for the page itself

One user on my previous post on ProcessGenerateCode asked how he could associate a ControlBuilder not with a control, but with the page itself.  There is in fact a way to do it, and it’s another one of those things that have never really been advertized.  The trick is that instead of associating the ControlBuilder using the standard ControlBuilderAttribute, you need to use a FileLevelControlBuilderAttribute.  Let’s walk through a little example.

First, we create a custom page type with that attribute:

[FileLevelControlBuilder(typeof(MyPageControlBuilder))]
public class MyPage : Page {
}

We could do all kind of non-default thing in the derived Page, but I’ll keep it simple.  Now let’s write the ControlBuilder for it:

// Note that it must extend FileLevelPageControlBuilder, not ControlBuilder!
public class MyPageControlBuilder : FileLevelPageControlBuilder {
    public override Type GetChildControlType(string tagName, IDictionary attribs) {
        // If it's a Label, change it into our derived Label
        Type t = base.GetChildControlType(tagName, attribs);
        if (t == typeof(Label)) {
            t = typeof(MyLabel);
        }

        return t;
    }
}

The most important thing to note here is that it extends FileLevelPageControlBuilder, not ControlBuilder.  If you extend ControlBuilder, it may seem like it works on simple pages but some things will break (e.g with master pages).

As for what we do in that class, it’s just some random thing to demonstrate that it is in fact getting called.  Here, we change the type of all Labels to a derived type which slightly modifies the output:

public class MyLabel : Label {
    public override string Text {
        set { base.Text = "[[" + value + "]]"; }
    }
}

And then of course, we need to tell our aspx page to use our base class:

<%@ Page Language="C#" Inherits="MyPage" %>

The full runnable sample is attached below.

ProcessGeneratedCode: A hidden gem for Control Builder writers

If you’ve ever written any non-trivial ASP.NET control, you’re probably familiar with the concept of a Control Builder.  Basically, it’s a class that you associate with your control and that affects the way your control gets processed at parse time.  While ControlBuilder has been around since the ASP.NET 1.0 days, a very powerful new feature was added to it in 3.5 (i.e. VS 2008).  Unfortunately, we never had a chance to tell people about it, and a web search reveals that essentially no one knows about it!  Pretty unfortunate, and obviously, the point of this post is to change that. :-)

So what is this super cool feature?  Simply put, it lets the ControlBuilder party on the CodeDom tree used for code generation of the page.  That means a ControlBuilder can inspect what’s being generated, and make arbitrary changes to it.

Warning: this post assumes some basic knowledge of CodeDom.  If you are not familiar with it, you may want to get a basic introduction to it on MSDN or elsewhere before continuing.

How do you use this?

To use this feature, all you have to do is override the new ProcessGeneratedCode() method on ControlBuilder.  here is what this method looks like:

//
// Summary:
//     Enables custom control builders to access the generated Code Document Object
//     Model (CodeDom) and insert and modify code during the process of parsing
//     and building controls.
//
// Parameters:
//   codeCompileUnit:
//     The root container of a CodeDOM graph of the control that is being built.
//
//   baseType:
//     The base type of the page or user control that contains the control that
//     is being built.
//
//   derivedType:
//     The derived type of the page or user control that contains the control that
//     is being built.
//
//   buildMethod:
//     The code that is used to build the control.
//
//   dataBindingMethod:
//     The code that is used to build the data-binding method of the control.
public virtual void ProcessGeneratedCode(CodeCompileUnit codeCompileUnit, CodeTypeDeclaration baseType, CodeTypeDeclaration derivedType, CodeMemberMethod buildMethod, CodeMemberMethod dataBindingMethod);

So basically you get passed a bunch of CodeDom objects and you get to party on them.  It may seem a bit confusing at first to get passed so many different things, but they all make sense in various scenarios.

• The CodeCompileUnit is the top level construct, which you would use for instance if you wanted to generate new classes.
• Then you have the two CodeTypeDeclarations, which represent the classes that are generated for this page.  The reason there are two has to do with how the page is generated.  First, there is a partial base class just has control declaration, and gets merged with the code behind class the user writes (in Web Sites, Web Applications are a bit different).  Then we derive another class from it, which has the bulk of the code that makes the page run.  I know this seems confusing, so let me give you a rule of thumb: when in doubt, use baseType rather than derivedType.
• Finally, we have the two CodeMemberMethods, which represent methods that relate to the particular control that’s being built.  One is for the method that builds the control (e.g. assigns its properties from the markup, …), and the other one deals with data binding.

Tip to make more sense of all that stuff: a great way to learn more about the code ASP.NET generate is simply to look at it!  To do this, add debug=”true” on your page, add a compilation error in there (e.g. <% BAD %>) and request the page.  In the browser error page, you’ll be able to look at all the generated code.

How about a little sample to demonstrate?

Let’s take a look at a trivial sample that uses this.  It doesn't do anything super useful but does demonstrate the feature.  First, let’s write a little control that uses a ControlBuilder:

[ControlBuilder(typeof(MyGeneratingControlBuilder))]
public class MyGeneratingControl : Control {
    // Control doesn't do anything other than generate code via its ControlBuilder
}

Now in the ControlBuilder, let’s implement ProcessGeneratedCode so that it spits out a little test property:

// Spit out a property that looks like:
//protected virtual string CtrlID_SomeCoolProp {
//    get {
//        return "Hello!";
//    }
//}
var prop = new CodeMemberProperty() {
    Attributes = MemberAttributes.Family,
    Name = ID + "_SomeCoolProp",
    Type = new CodeTypeReference(typeof(string))
};

prop.GetStatements.Add(new CodeMethodReturnStatement(new CodePrimitiveExpression("Hello!")));

baseType.Members.Add(prop);

So  it just generates a string property with a name derived from the control ID.  Now let’s look at the page:

    <test:MyGeneratingControl runat="server" ID="Foo" />

And finally, let’s use the generated property in code.  The simple presence of the this tag allows me to write:

Label1.Text = Foo_SomeCoolProp;

And the really cool things is that Visual Studio picks this up, giving you full intellisense on the code generated by your ControlBuilder.  How cool is that! :)

Full runnable sample is attached to this post.

What about security?

At first glance, it may seem like this feature gives too much power to ControlBuilders, letting them inject arbitrary code into the page that’s about to run.  The reality is that it really doesn’t let an evil control do anything that it could have done before.  Consider those two cases:

• Full trust: if the Control and ControlBuilder are running in full trust, then they can directly do anything that they want.  They gain nothing more from  generating code that does bad things
• Partial trust: if the Control and ControlBuilder are running in partial trust, then they can’t do much damage themselves directly, and can attempt to do additional damage indirectly via code they generate.  However, if the app is running in partial trust (as is typical in hosted environments), then the dynamically generated page code also runs in partial trust.  So effectively, it can’t do any more than the control could do directly.

Conclusion

ProcessGeneratedCode is a pretty powerful feature, giving your ControlBuilders full control over the code generation.  It’s also a pretty advanced feature, and you can certainly shoot yourself in the foot with it if you’re not careful.  So be careful!

A ‘Many To Many’ field template for Dynamic Data

Unlike Linq To Sql, Entity Framework directly supports Many to Many relationships.  I’ll first describe what this support means.

In the Northwind sample database, you have Employees, Territories and EmployeeTerritories tables.  EmployeeTerritories is a ‘junction’ table which has only two columns: an EmployeeID and a TerritoryID, which creates a Many to Many relationship between Employees and Territories.

When using Linq To Sql, all three tables get mapped in your model, and you need to manually deal with the EmployeeTerritories junction table.  But when using Entity Framework, the EmployeeTerritories junction table is not part of your model.  Instead, your Employee entity class has a ‘Territories’ navigation property, and conversely your Territory entity class has an ‘Employees’ navigation property.  What Entity Framework does under the cover to make all this work is pretty amazing!

Unfortunately, in ASP.NET Dynamic Data’s initial release (as part of Framework 3.5 SP1), we didn’t have time to add proper support for such Many to Many relationships, and in fact it behaves in a pretty broken way when it encounters them.

The good news is that it is possible to write a field template that adds great support for this, and that is exactly what this blog post is about.  I should note that a couple of our users have written such field templates before (in particular, see this post).  In fact, that’s what got me going to write one! :)

One difference is that I set mine out to be completely generic, in the sense that it doesn’t assume any specific database or table.  e.g. it works for Northwind’s Employees/Territories (which my sample includes), but works just as well for any other database that uses Many to Many relationships.

In read-only mode, it shows you a list of links to the related entities.  e.g. when looking at a Territory, you’ll see links to each of the Employees that work there.

In edit and insert mode, it gets more interesting: it displays a list of checkboxes, one for each Employee in the database.  Then, whether the Employee works in this territory is determined by whether the checkbox is checked.  Pretty much what you’d expect!

I won’t go into great details about how it works here, but if you are interested, I encourage you to download the sample and look at the code, which I commented pretty well.  The key things to look at are:

• The field templates ManyToMany.ascx (used for read-only) and ManyToMany_Edit.ascx (used for Edit and Insert).
• The AutoFieldGenerator, which automatically uses those field templates for Many to Many relationships.

Enjoy, and let me know if you have feedback on this.

Why posting questions to the forums beats private emails

While most of the Q&A about ASP.NET happens in the forums, I also receive a number of direct questions from users (and so do other people on the team).  In most cases, I try to convince the sender to post to the forums instead, and there are a couple of good reasons for it.  And no, the reason is not to just get rid of the user! :-)

The first reason is that posting to the forum gives you a much larger audience of people who may be able to answer the question.  The answer may come from one of the Microsoft folks who are active there, or from one of the many experienced users from the community.  As a result, you're much more likely to get a quick answer than by sending a private email to one person.

A second very important reason is that posting to the forums builds up the general knowledge base.  Your question and the answers that it receives will be searchable by others for years to come.  So asking your question there can effectively save others from asking the same thing.  Or maybe their question won't be exactly the same, but it will be similar enough that they will post a new question that 'builds' on the previous answers.

Of course, there is no guarantee that every question you post will get answered instantly in a way to fully solves it, and it may take a few iterations to get it there.  But it is your best bet!

Dynamic Data Filtering project up on CodePlex

Josh Heyse has just released a very cool extension to ASP.NET Dynamic Data which supports much fancier filtering than is available is the core Dynamic Data.  The core idea is that filters can contribute arbitrary LINQ expression trees to the Select query performed by the data source.  So for instance you can have a Range filter which checks that a column is between two values, or a Search filter which performs sub-string matching (e.g. StartsWith, EndsWith or Contains).  All that is made available with very little effort from the user.

It's all available on CodePlex, so check it out!  Start with the included AdventureWorks sample to get an idea what it's all about.  Great job Josh!

Using ASP.NET Dynamic Data with ObjectDataSource

Support for LINQ based O/R mappers

Out of the box, ASP.NET Dynamic Data has support for both Linq To Sql and Entity Framework.  In addition, it has a provider model which allows additional O/R mappers to be supported.  For instance, we have a sample provider that supports ADO.NET Data Services (aka Astoria).  There is also a provider for LLBLGen.

Note that all those scenarios are some things in common:

• They are based on some form of data context: it's called various things in the different O/R mappers (DataContext, ObjectContext, DataServiceContext), but essentially it's a class that has a property for each Entity Set.
• They support LINQ: the Entity Set properties on the data context class implement IQueryable<T>, which allows LINQ to be used on them.

Support for scenarios that use ObjectDataSource

While having this extensibility is great, it doesn't cover a whole range of scenarios where users have their own data layer, with no LINQ and no data context in sight.  For a simple example of such scenario, see the sample attached to this MSDN page.  The key aspects are:

• Entity class: you have a strongly typed some entity class (NewData in the MSDN sample).  It just defines the fields you care about, with no real  logic attached.
• Business layer class: you have a class that has various methods to perform your CRUD operations (the class is named AggregateData in that sample).
• ObjectDataSource: your page uses an ObjectDataSource to have ASP.NET interact with your entity and business layer classes.  You tell the data source exactly which method to call for each CRUD operation.

Quite a few people use this technique, as it is very flexible and can work with fairly arbitrary business layer.

So how does this all work when you want to use Dynamic Data in this type of scenarios?  It actually integrates very nicely, by using some simple functionality that's available as part of the Dynamic Data Futures on CodePlex.  If you download that solution, it includes a fully functional sample based on the MSDN sample above.

Here is more details on what it all looks like.

The business layer class

This remains pretty unchanged, except that you have the ability to throw an exception when you see invalid data come in, e.g.

public class AggregateData {

    // Unchanged pieces omitted

    public DataTable Select() {
        return (table == null) ? CreateData() : table;
    }
  
    public int Insert(NewData newRecord) {
        if (newRecord.Name.Length < 3) {
            throw new ValidationException("The name must have at least 3 characters");
        }

        table.Rows.Add(new object[] { newRecord.Name, newRecord.Number, newRecord.Date });
        return 1;
    }
}

Here, it's saying that the name must have at least 3 characters.  What's really nice is that the exception you throw here gets shown exactly where you expect it: in the ASP.NET validation summary.

The Entity class

Let's now look at the entity class.  Again, it's mostly unchanged from the original, except that we now have the ability to annotate it using Dynamic Data attributes:

public class NewData {
    [RegularExpression(@"[A-Z].*", ErrorMessage="The name must start with an upper case character")]
    [Required]
    [DisplayName("The name")]
    public string Name { get; set; }

    [Range(0, 1000)]
    [UIHint("IntegerSlider")]
    [DefaultValue(345)]
    public int Number { get; set; }

    [DataType(DataType.Date)]
    [UIHint("DateAjaxCalendar")]
    [Required]
    public DateTime Date { get; set; }
}

I won't go over the details of what each attribute means, as they are the standard Dynamic Data attributes and they're fairly self-explanatory.  The key point is that everything works exactly in the same way as they would when you use Dynamic Data with Linq To Sql: the rendering goes through the field templates, the UIHint is used to select the template, the validation attributes become ASP.NET validators, ...

The aspx page

Finally, let's look at the ASP.NET page.  Again, it's not all that different from the original sample:

    <asp:ValidationSummary ID="ValidationSummary1" runat="server" EnableClientScript="true"
        HeaderText="List of validation errors" />
    <asp:DynamicValidator runat="server" ID="DetailsViewValidator" ControlToValidate="DetailsView1" Display="None" />

    <asp:DetailsView ID="DetailsView1" runat="server" AllowPaging="True" AutoGenerateInsertButton="True"
        DataSourceID="ObjectDataSource1" EnableModelValidation="true">
    </asp:DetailsView>
    
    <asp:DynamicObjectDataSource ID="ObjectDataSource1" runat="server" DataObjectTypeName="DynamicDataFuturesSample.NewData"
        InsertMethod="Insert" SelectMethod="Select" TypeName="DynamicDataFuturesSample.AggregateData">
    </asp:DynamicObjectDataSource>

The most notable change is that ObjectDataSource became a DynamicObjectDataSource, and that we added a DynamicValidator.

So let's see it run!

Here is what it looks like with the above metadata:

Some notable things:

• The header for the name says 'the name' thanks to the DisplayName on the Name property
• It complains that the name doesn't start with an upper case, per the RegularExpression attribute
• The Number's UI shows up as an AJAX slider, thanks to the UIHint
• Likewise, the Date UI uses an AJAX calendar

Conclusion

Even though it wasn't a scenario that we originally intended to cover, it turns out that using Dynamic Data with ObjectDataSource scenarios is quite easy and works very well.  It really opens up a whole new range of scenarios that don't force users to switch to a different O/R mapper.

It does have a notable limitation: it is not meant to support the full site scaffolding that you get with Linq To Sql or Entity Framework.  This comes from the fact that without a data context, there is no easy way to know what the set of relevant tables is, and what relationships they have with each other.  Of course, you still get the scaffolding aspect at the page level, in so far as the UI for your DetailsView/GridView is completely model driven, which is the main appeal of Dynamic Data.

One of our users Dan Meineck just blogged about this as well, so check out what he had to say!

ASP.NET Dynamic Data on CodePlex

Today, ASP.NET Dynamic Data moved from its current home on Code Gallery to its new home on CodePlex.  Note that this is part of a general ASP.NET CodePlex project, so you'll see other features mentioned in there, like MVC and AJAX.  This doesn't mean that those features tie with Dynamic Data any more than they did before; they're just sharing a home on the same CodePlex project.

There are a couple different things related to Dynamic Data that are available there, and since there is room for confusion I'll try to help make sense of it all.

The 'core' Dynamic Data

The first piece is Dynamic Data itself.  This will ship later this summer as part of Framework 3.5 SP1, but until that happens you need to install preview bits.  In order to get the latest preview, you first need to install VS 2008 SP1 Beta.  This comes with an outdated build of Dynamic Data, so you then need to install the update from CodePlex.  This is the same update that we had on Code Gallery, so if you already installed that, you don't need to do it again.

Once Framework 3.5 SP1 officially ships, there will no longer be a need for this.  Note that for all practical purpose, this update is identical to what the final release will be in SP1.  We are now completely locked down, and have not made a change in 3 weeks.  It just takes a little more time for the whole VS SP1 stack to finish up and be out the door.

Note that even though this is on CodePlex, we are not at this time making the sources of the 'core' Dynamic Data available.  I certainly hope that we will be able to in the future, but that has to go through some lawyers and other fun things.

The Dynamic Data Futures

The second important piece that's available on CodePlex is the Dynamic Data Futures (previously calls 'Dynamic Data Extensions').  At this time, these Futures are basically a big VS solution that contain both a library of new features (Microsoft.Web.DynamicData.dll), and a couple web sites that show them in action.

Unlike the Dynamic Data 'core', this 'Futures' piece is fully available in source form.  The solution first builds Microsoft.Web.DynamicData.dll, and then builds the various samples that rely on it.

Note that those Futures do not replace the core Dynamic Data, but instead run on top of it.  So it shouldn't be looked at as being the 'next version' of Dynamic Data, but rather as being some extra functionality that will be available on top of the Framework 3.5 SP1 release.

This project serves a few different purposes:

• It gives us a place to experiment with new features that we didn't have time to put in SP1.  Some of those will eventually make their way into the next version of Dynamic Data, while others are likely to remain samples.  Obviously, we'd love to get feedback on those (the forum is the right place for that).
• It lets us address some issues that are in the SP1 release and couldn't be fixed in time.
• It has some great samples for Dynamic Data users to look at and learn how to achieve various results.  Note that a few of the samples that were previously separate are now included in there (e.g. Scott Hunter's DbImage sample).

What happens next

The 'Futures' project is very much work in progress, and we are actively adding things to it.  So you should expect to see fairly frequent updates on CodePlex.

Please give this a try and let us know how it goes on the forum!

Dynamic Data and the Associated Metadata Class

If you've used Dynamic Data or watched some demos, you may have been puzzled by the way metadata is associated with database fields.  Instead of being placed directly on the partial Entity class (e.g. Product), it needs to go on a different class which gets associated with the real class via a MetadataTypeAttribute.

Here is an example where we're adding a display name on the UnitsInStock column:

[MetadataType(typeof(Product_Metadata))]
public partial class Product {
}

public class Product_Metadata {
    [DisplayName("The Units In Stock")]
    public object UnitsInStock { get; set; }
}

Here are the key pieces at play:

• First you have the Product partial class, which is 'partial' with the Product class that was generated by the Linq To Sql or Entity Framework designer.  The generated class contains the 'real' UnitsInStock property, which is an integer (not shown above, but you'll find it in the generated Northwind.designer.cs file).
• This partial Product class has a MetadataType attribute pointing to a different class called Product_Metadata (the name is arbitrary but appending _Metadata is a good convention to indicate a MetaData buddy class).  This is the metadata class, sometimes referred to as the associated class or the 'buddy' class.  Note that this class only exists to hold metadata, and is never instantiated.
• On the metadata class, there is a UnitsInStock property.  This is sort of a 'dummy' property that is used as a place to put CLR attributes on.  We typically give it a return type of 'object', though in reality it is ignored and could be any type.
• And finally there is the metadata attribute itself, in this case DisplayName (and obviously you could have more than one attribute).

Why are we doing this?

So I have described what we do, but not really why we do it this way, instead of putting attributes directly on the real property.  Simply stated, the reason is that it's not possible, due to a C# language limitation (VB.NET has the same issue).  The limitation is that when one half of a partial class defines a property (in this case it's in the generated class), you cannot add attributes to that property from the other half of the partial class (in this case in the user code).

Of course, you could easily add the attributes directly on the generated class where the property is defined, but that would be a very bad idea, because you would lose it all as soon as you make a model change that requires the file to be regenerated.  In general, you should never make any changes to generated files, as that is asking for trouble!

What if you don't want to use CLR attributes for your metadata?

Some users don't like using CLR attributes on their entity class to define their metadata.  To see an alternative technique that lets you keep your metadata in arbitrary places, check out this really cool post by Marcin (one of the developers on Dynamic Data).

Understanding foreign keys and relationship columns in Dynamic Data

Suppose you're dealing with a table that has a foreign key relationship with another table.  To take the canonical example, let's say we're looking at the Northwind Products table, which has a foreign key into the Categories table.  What this means is that each Product is a member of a Category (and conversely, a Category can be seen as 'containing' a number of Products).

What can make this situation a little confusing is that there are two related concepts that come into play, and can often be mixed up:

1. The Foreign Key column: the Product class has a CategoryID property, which is an integer containing the ID of the Category that the Product is part of.  In some cases, there can be multiple Foreign Key columns used in a relationship (though this doesn't occur in Northwind).
2. The 'Entity Reference' column: Product has a Category property, which is typed as a Category.  This is sometimes referred to as a 'Relationship' property.

#1 is a lower level concept that typically matches what's in the database, while #2 is a higher level concept which lets you work directly at the object level.

What makes this particularly confusing in Dynamic Data is that we made some pretty poor decisions when it came to naming some of our classes.  Specifically, we are using the type MetaForeignKeyColumn to refer to the 'Entity Reference' column (#2), even though the name makes it sound like it's referring to #1.  Bad naming (sorry!), but we have to live with it now! :)

So if we're using the MetaForeignKeyColumn class for Entity References, what class are we using for real Foreign Key columns?  We don't have a special class from them, and we simply use the base MetaColumn class (which MetaForeignKeyColumn extends).  But you can recognize them  because their IsForeignKeyComponent property returns true.

Note that when you use Dynamic Data's scaffolding feature, by default it only shows the Entity Reference column and not the FK column.  This is because in most cases showing the CategoryID is completely uninteresting, and what people want to see is something more descriptive about the Category (e.g. its name), which is what the Entity Reference columns gives you.

Another thing worth noting is that there is a difference between Linq To SQL and Entity Framework when it comes to foreign  keys.  With Linq To SQL, your entity class has a property for the FK (e.g. CategoryID), while with Entity Framework you won't have that property at all.  However, in both cases you will get the Entity Reference (aka the MetaForeignKeyColumn in Dynamic Data), which is the one you usually care about.

Sample: Dynamic Data and attribute based permissions

Steve Naughton just posted a great series of articles on how he implemented an attribute based permission system that works with Dynamic Data.  This is something that we had been thinking about putting in the product early on, but decided against for a couple reasons:

• This was a short product cycle for us, and we had some pressure to limit the v1 feature set.
• We were not confident that we could come up with something generic enough to please everyone, so we opted instead to make Dynamic Data extensible enough that the permission system could be implemented on top of it.

Thanks a lot to Steve for coming up with this and taking the time to blog it so thoroughly!

Feel free to leave comments on Steve's blog, or in this forum thread where he announced it.

Dynamic Data, AJAX and JavaScript errors

The default Dynamic Data application uses AJAX partial rendering (using an UpdatePanel), which make browser post backs feel smoother.  One downside is that it can also make it harder to diagnose some error conditions.

Specifically, while using your application you may run into a JavaScript error that says something cryptic about PageRequestManagerServerErrorException, and generally doesn't relate to your application in a meaningful way.

The purpose of this post is to tell you how to see the real error behind it.  When reporting a Dynamic Data issue, please make sure you always do this as reporting a PageRequestManagerServerErrorException gives little information about what's going on.

Stepping back a little, there are two main categories of errors that your Dynamic Data app can run into:

1. Errors that get handled by validator controls: those get displayed inside the page, either inline (e.g. in a GridView cell) or at the top of the page (where the asp:ValidationSummary is), or often in both places.
2. Unhandled errors: those normally result in an ASP.NET runtime error page, which typically have a yellow background and include a clear error message and a stack trace.

The problem is that when using AJAX partial rendering, unhandled errors turn into those cryptic PageRequestManagerServerErrorException, which basically don't tell you what's wrong.

So what should you do about it?  Quite simply, when you run into this, I suggest you temporarily turn off partial rendering so you can see the real error.  To turn it off, open the file Site.master (at the root of the app) and set EnablePartialRendering="false".  Then rerun your scenario using the same step that previously resulted in the JavaScript error.  You should end up with a real ASP.NET error page.

Of course, there is no guarantee that this error will make complete sense, but at least you'll be looking at the real error, and if you report it (e.g. to the forum), there is a good chance that someone can help with it.

When you're done debugging the problem, feel free to set EnablePartialRendering back to true, unless you don't care for it.

As an aside, not that even though I'm writing this in the context of Dynamic Data, everything in this post applies to any ASP.NET application that uses AJAX rendering.  It's just that Dynamic Data has it on by default, and since you didn't have to explicitly turn it on, you may not know that it's happening.

Clearing some confusion about Dynamic Data and VS SP1 Beta

Today, we released Visual Studio 2008 and .NET FX 3.5 SP1 Beta, which include the new ASP.NET Dynamic Data.  The part that gets a little bit confusing is that the Dynamic Data bits that come with the brand new Beta are actually older than Preview bits that we've had out for a while on Code Gallery!  The reason for this is that getting a VS Beta out is a pretty big deal, and we need to start locking down the bits long before the beta goes out.  But while we stopped changing the beta bits a while back, we continued to work full speed on what will eventually become the RTM bits.  The great thing is that we have been able to put frequent updates up on Code Gallery, which has allowed us to get some super valuable user feedback.

I should also mentioned that we put up a refresh of the Dynamic Data bits on code gallery this morning, at the same time that the Beta was released.  And unlike the previews drops that were available earlier on Code Gallery, this new drop requires that you first install the VS SP1 Beta.  In fact, this new Preview drop is extremely similar to what the VS SP1 RTM bits will be, as this project cycle is coming to completion.

I know, this all sounds a bit confusing.  So what does this all mean to you?  Basically, it means that if you want to use Dynamic Data today, the best thing you can do is:

• Install VS 2008 SP1 Beta
• Go to our Code Gallery site and update Dynamic Data to the very latest.  You'll find it under the 'Releases' tab.
• Use our forum to report issues and give feedback.

Although the Dynamic Data bits that come with the beta are usable, they are essentially outdated and we would much prefer to have everyone use the very latest.

New Dynamic Data Preview available

I'm happy to announce that we have made a new public drop of ASP.NET Dynamic Data available.  The last public drop was made in December and was starting to get a little outdated.  More recently we had made some drops available via Microsoft Connect, but it was a non-public preview which required an approval process.

This new drop is openly available at all on MSDN Code Gallery.  One of the nice things about this is that we'll be able to give much more frequent updates, instead of the long delays that often happen between betas.

Another cool thing about this new Preview is that it includes the first public drop of the Dynamic Data Website Wizard, which gives a richer design time experience to Dynamic Data sites.  Though the wizard is still in a pretty early and incomplete state, we're hoping it will give you an idea of where we are going with this.

You can look at my previous post for an overview of the changes we made since the December drop.  Note that even though the drop has the ability to support Entity Framework (in addition to Linq To Sql), you will not quite be able to use it yet because it targets an Entity Framework build that is newer than the latest public build.  But don't worry, it won't be long before the missing pieces that will allow this are made available.

We are eagerly awaiting your feedback on this new drop.  As before, the best place to discuss Dynamic Data and report issues is our ASP.NET forum.

Thanks!