Angle Bracket Percent

T4MVC 2.2 update: Routing, Forms, DI container, fixes

To get the latest build of T4MVC:

Go to download page

 

This post is a continuation of various recent posts, most notably:

• A new and improved ASP.NET MVC T4 template
• The MVC T4 template is now up on CodePlex

First, I’d like to thank all those who are using the MVC T4 template and sent me suggestions and bug reports.  Most issues have been addressed, and most suggestions have been integrated.  I’m up to the 8th CodePlex drop, and it’s only been a week! You can see the history of changes at the top of the .tt file.

Frankly, when I started playing with this, I just thought it’d be a fun thing to spend the afternoon on.  Instead, I have probably spent close to half my time working on it in the last week.  And I do have other things to work on! :)  But it’s been a lot of fun, so no regrets!

I’m not going to detail every change since the last blog post, since many are bug fixes that are not particularly exciting (though they help make the thing work!).  Instead, I’ll just focus on a few new key areas.

 

Strongly typed support for MapRoute

This is similar to the RedirectToAction/ActionLink support, but applied to route creation.  The original Nerd Dinner routes look like this:

routes.MapRoute(
    "UpcomingDinners", 
    "Dinners/Page/{page}", 
    new { controller = "Dinners", action = "Index" }
);

routes.MapRoute(
    "Default",                                              // Route name
    "{controller}/{action}/{id}",                           // URL with parameters
    new { controller = "Home", action = "Index", id = "" }  // Parameter defaults
);

The ‘defaults’ line of each route is what we’re trying to fix.  It makes heavy use of anonymous objects and literal strings, which should be avoided whenever possible.  If your controller or action name change, you won’t easily catch it.

Previous version of T4MVC let you change that line to:

new { controller = MVC.Dinners.Name, action = MVC.Dinners.Actions.Index }

This fixes the issue for the most part, but it’s a bit wordy and doesn’t support refactoring.  Now, the latest version of T4MVC lets you change those routes to:

routes.MapRoute(
    "UpcomingDinners", 
    "Dinners/Page/{page}", 
    MVC.Dinners.Index(null)
);

routes.MapRoute(
    "Default",                                              // Route name
    "{controller}/{action}/{id}",                           // URL with parameters
    MVC.Home.Index(),                                       // Default action
    new { id = "" }                                         // Parameter defaults
);

Note how we are now pointing at the action by simply making a call to it (though it’s a pseudo-call, and your action method won’t actually be called).

Also note that in the second route, we’re still using an anonymous object for the ‘id’.  This is because even though the route expects an id, the default Action method (HomeController.Index()) doesn’t take one.  You could get around this by adding a ‘string id’ param to HomeController.Index() and ignoring it in the method.  You could then write MVC.Home.Index(null) in the route and avoid all anonymous objects.

 

BeginForm support

This came courtesy of Michael Hart.  Essentially, it’s the same support as ActionLink() but for the BeginForm() method.  But note that because form posts typically pass most of the data via the form and not the URL, BeginForm() is trickier to use correctly than the other methods.

Here is how you might use this:

using (Html.BeginForm(MVC.Account.LogOn(), FormMethod.Post)) { ... }

Or if the action method takes parameters, you would pass them in the call.  However, when you do this you need to make sure that you aren’t also trying to pass those same parameters via the form (e.g. using a text box).

Generally, the rule of thumb is that this support works well when you’re dealing with forms where the Action method signature exactly matches the form URL.

 

Support for Dependency Injection containers

Though this is a bug fix rather than a new feature, I’ll mention it because it affected several people and had me puzzled for a while.

The root of the problem is that T4MVC generate new constructors for the controllers, and that DI containers use a somewhat arbitrary algorithm to decide what constructor to use.  And in those users’ cases, they ended up calling the new constructor generated by T4MVC, messing things up badly.

The fix was simply to change the generated constructor to be protected, to prevent DI containers from using it.  Since T4MVC only uses it when it instantiates the controller derived class, protected works fine.

Well, I should note that I haven’t actually verified that the fix addresses the issue, but I’m sure those users will tell me if it doesn’t! :)

The MVC T4 template is now up on CodePlex, and it does change your code a bit

Short version: the MVC T4 template (now named T4MVC) is now available on CodePlex, as one of the downloads in the ASP.NET MVC v1.0 Source page.

Go to download page

 

Poll verdict: it’s ok for T4MVC to make small changes

Yesterday, I posted asking how people felt about having the template modify their code in small ways.  Thanks to all those who commented!  The fact that Scott Hanselman blogged it certainly helped get traffic there :)

The majority of people thought that it was fine as long as

• It’s just those small changes: make classes partial and action methods virtual. Don’t mess with ‘real’ code!
• It asks for permission, or at least tells you what it’s doing

I started looking for a way to pop up a Yes/No dialog, but ended up going with a slightly different approach: T4MVC adds a warning line for every item it modifies.  e.g. when you run it, you might see these in the warnings area:

Running transformation: T4MVC.tt changed the class DinnersController to be partial
Running transformation: T4MVC.tt changed the action method DinnersController.Index to be virtual

Some people were worried about version control.  I tried using TFS, and everything worked fine.  i.e. when the template modifies files, VS automatically checks them out.  We’ll need to see how that works for folks using different systems.

 

What’s new in this version?

The template on CodePlex (version 2.0.01 at the top of the file) supports what I described in my previous post, plus some new goodies.

Refactoring support for action methods

One of the big issues before was the lack of refactoring support.  e.g. when you wrote:

return RedirectToAction(MVC.Dinners.Details(dinner.DinnerID));

This looked like a call to you Details controller action, but it was actually an unrelated method by the same name.  Hence, if you renamed your action method and refactored, this call was not modified.  It would give a compile error, and had to be hand fixed.

Now the template takes a drastically different approach:

• It extends the controller class
• It overrides the action method (hence the need for it to be virtual!)
• The override never calls the base (that would be very wrong), but instead returns a special ActionResult which captures the call (controller name, action name, parameter value).
• The template emit a new RedirectToAction (or ActionLink, …) overload which understands this special ActionResults, and turns the call data into a ‘regular’ RedirectToAction call.

Pretty tricky stuff, but it works quite well.  Some credit to my manager Mike Montwill for coming up with this crazy idea!

Because the method you call is an override of the real action method, refactoring works perfectly.  Also, if you F12 (Go To Definition) on the call, it’ll go straight to your Action method and not some generated code.

Unfortunately, Visual Studio doesn’t support refactoring in Views, but 3^rd party tools like Resharper and CodeRush do, so if you use one of those, you’re fully covered.

 

The T4 file automatically runs whenever you build

This was the other big painful issue I was up against: every time you made a change to your code that affect the generated code (e.g. new Action, new View, …), you had to manually save the .tt file to cause it to regenerate the new helper code.

This was a really hard issue, and I must warn you that what I ended up with is more of a workaround than a fix.  However, it is pretty effective, so until we find a better solution, it’ll have to do.

Here is how it works.  Warning: reading this has been shown to cause headaches in lab rats:

• As part of its execution, the T4 file finds itself in the VS project system (it had to do that anyway)
• It then runs the magic instruction ‘projectItem.Document.Saved = false;’, which causes it to become dirty.
• It then proceeds to do its code generation, leaving its file in an unsaved state
• Next time you Build your project, VS first saves all the files
• This causes the ‘dirty’ T4 template to execute, mark itself as dirty again, and redo its code generation
• You get the idea!  If you feel like the lab rats, this may help.

One caveat is that you have to initiate the cycle by opening and saving T4MVC.tt once.  After you do that, you don’t need to worry about it.

 

Support for strongly typed links to static resources

Credit for this idea goes to Jaco Pretorius, who blogged something similar.

The template generates static helpers for your content files and script files.  So instead of writing:

<img src="/Content/nerd.jpg" />

You can now write:

<img src="<%= Links.Content.nerd_jpg %>" />

Likewise, instead of

<script src="/Scripts/Map.js" type="text/javascript"></script>

You can write:

<script src="<%= Links.Scripts.Map_js %>" type="text/javascript"></script>

The obvious benefit is that you’ll get a compile error if you ever move or rename your static resource, so you’ll catch it earlier.

Another benefit is that you get a more versatile reference.  When you write src="/Content/nerd.jpg", your app will only work when it’s deployed at the root of the site.  But when you use the helper, it executes some server side logic that makes sure your reference is correct wherever your site is rooted.  It does this by calling VirtualPathUtility.ToAbsolute("~/Content/nerd.jpg").

One unfortunate thing is that for some reason, VS doesn’t support intellisense in the view for parameter values.  As a workaround, you can type it outside of the tag to get intellisense and then copy it there.

 

More consistent short form to refer to a View from a Controller class

Previously, it supported an _ based short form inside the controller:

return View(View_InvalidOwner);

That was a bit ugly.  Now, the short form is:

return View(Views.InvalidOwner);

Here, Views.InvalidOwner is the same as MVC.Dinners.Views.InvalidOwner, but can be shortened because ‘Views’ is a property on the controller.

 

Many bug fixes

I also fixed a number of bugs that people reported and that I ran into myself, e.g.

• It supports controllers that are in sub-folders of the Controllers folder and not directly in there
• It works better with nested solution folder

I’m sure there are still quite a few little bugs, and we’ll work through them as we encounter them

Mind if my MVC T4 template changes your code a bit?

Update: Please see this post for what came out of this ‘poll’, and for a pointer to the newest T4 template on CodePlex.

 

When working on my MVC T4 template, I was not able to use reflection to discover the Controllers and Actions, because the code that the template generates is itself in the same assembly as the controllers.  So that causes a bit of a chicken and egg problem.

Instead, I had to get out of my element and learn something I was not familiar with: the Visual Studio File Code Model API.  It’s very different from using reflection, because instead of working at the assembly level, you work at the source file level.

You have to first locate the source file you want to look into.  You can then ask for the namespaces that it contains, and the classes that they contain, and finally the various members in those classes.  To be honest, I find this API quite ugly.  It’s a COM interop thing with a horrible object model that looks like it grew organically from version to version rather than having been designed with usability in mind.  So all in all, I used it because I had to, but the whole time I was wishing I could use reflection instead.

But then I made an important realization.  Ugly as it is, this object model supports something that would never be possible with reflection: it lets me modify the source code!

If you look at my previous post, I wrote “But to make things even more useful in the controller, you can let the T4 template generate new members directly into your controller class.  To allow this, you just need to make you controller partial”.  And I have logic in the template that tests this, and does extra generation if it is partial, e.g.

if (type.ClassKind == vsCMClassKind.vsCMClassKindPartialClass) { ... }

But instead, I have now realized that I can turn this check into an assignment, and change the class to partial if it isn’t already!

type.ClassKind = vsCMClassKind.vsCMClassKindPartialClass;

Likewise, I have scenarios where I can do cool things if the Controller actions are virtual, and I can just change them to be with a simple line:

method.CanOverride = true;

To be clear, those statements actually modify the original source file, the one that you wrote.  While this is certainly powerful and opens up some new doors, it also raises a big question which is the main purpose of this post:

Do you mind if the T4 template makes these small changes to your code?

We’re only talking about pretty harmless things (making classes partial and methods virtual), but I know developers can get nervous if even small changes magically happen in their source files.

So please tell me how you feel about this, e.g. is it more:

1. It’s harmless, go for it if it makes the template more useful
2. Undecided. I don’t really like it, but maybe I’ll put up with it.
3. No way I’ll use this template if it messes with my files in any way. I may even sue you!

Tell me where you stand, and please don't sue me.

After almost five years, my blog has a name!

So I have had this blog since October 2005, and the entire time it has been named “David Ebbo’s blog”.  There were a number of solid reasons that had led me to choose this catchy name:

• It’s a blog
• It’s mine
• My name is David Ebbo

Last December, I blogged with great excitement that I had picked not a new title, but a new subtitle for it: “The Ebb and Flow of ASP.NET”.  It was a week before xmas, there was almost no one at work and I might have had a drink or two, but for whatever reason, I came up with that brilliant name.  But that was just the subtitle, which is displayed in a small font that no one reads, so it went quietly unnoticed.

Today, I am taking a much bigger step in my blogging career as I’m actually changing the blog title.  Yes, the very string that makes or breaks a blog, the one that shows up in your browser’s title bar, the one that drives how much advertisement money I would get if only I had ads on there!

And the new name is… <% Angle Bracket Percent %>!  You know, like those things we have all over aspx pages and MVC views.  Hmmm, if I need to explain it, then it may not be as good a name as I first thought.  Now leave me alone, or I’ll stop following you on Twitter.

Update: ok, it appears that having “<%” as part of the title was not such a great idea, as it shows up as &lt;% is various places, which looks ugly.  So I dropped the <% %> and only kept “Angle Bracket Percent”.  I’m not sure if that makes it more or less lame, nor am I sure if I care.

A new and improved ASP.NET MVC T4 template

Update: Please read this post for the newest and greatest.

A couple weeks ago, I blogged about using a Build provider and CodeDom to generate strongly typed MVC helpers at runtime.  I followed up a few days later with another version that used T4 templates instead, making it easier to customize.

And now I’m back with yet another post on this topic, but this time with a much simpler and improved approach!  The big difference is that I’m now doing the generation at design time instead of runtime.  As you will see, this has a lot of advantages.

Drawbacks of the previous runtime approach

Before we go and re-invent the wheel, let’s discuss what the issues with the runtime T4 approach were, and how this is solved by this new approach.

Complex configuration: to enable the runtime template, you had to add a DLL to your bin, modify two web.config files, and drop two T4 files in different places.  Not super hard, but also not completely trivial.  By contrast, with this new approach you just drop one .tt file at the root of your app, and that’s basically it.

No partial trust support: because it was processing T4 files at runtime, it needed full trust to run.  Not to mention the fact that using T4 at runtime is not really supported!  But now, by doing it at design time, this becomes a non-issue.

Only works for Views: because only the Views are compiled at runtime, the helpers were only usable there, and the controllers were left out (since they’re built at design time).  With this new approach, Controllers get some love too, because the code generated by the template lives in the same assembly as the controllers!

Let’s try the new T4 template with the Nerd Dinner app

Let’s jump right in and see this new template in action!  We’ll be using the Nerd Dinner app as a test app to try it on.  So to get started, go to http://nerddinner.codeplex.com/, download the app and open it in Visual Studio 2008 SP1.

Then, simply drag the T4 template (the latest one is on CodePlex) into the root of the NerdDinner project in VS.  And that’s it, you’re ready to go and use the generated helpers!

Once you’ve dragged the template, you should see this in your solution explorer:

Note how a .cs file was instantly generated from it.  It contains all the cool helpers we’ll be using!  Now let’s take a look at what those helpers let us do.

Using View Name constants

Open the file Views\Dinners\Edit.aspx.  It contains:

<% Html.RenderPartial("DinnerForm"); %>

This ugly “DinnerForm” literal string needs to go!  Instead, you can now write:

<% Html.RenderPartial(MVC.Dinners.Views.DinnerForm); %>

Though it’s wordier, note that you get full intellisense when typing it.

ActionLink helpers

Now open Views\Dinners\EditAndDeleteLinks.ascx, where you’ll see:

<%= Html.ActionLink("Delete Dinner", "Delete", new { id = Model.DinnerID })%>

Here we not only have a hard coded Action Name (“Delete”), but we also have the parameter name ‘id’.  Even though it doesn’t look like a literal string, it very much is one in disguise.  Don’t let those anonymous objects fool you!

But with our cool T4 helpers, you can now change it to:

<%= Html.ActionLink("Delete Dinner", MVC.Dinners.Delete(Model.DinnerID))%>

Basically, we got rid of the two unwanted literal strings (“Delete” and “Id”), and replaced them by a very natural looking method call to the controller action.  Of course, this is not really calling the controller action, which would be very wrong here.  But it’s capturing the essence of method call, and turning it into the right route values.  And again, you get full intellisense:

 

By the way, feel free to press F12 on this Delete() method call, and you’ll see exactly how it is defined in the generated .cs file.  The T4 template doesn’t keep any secrets from you!

Likewise, the same thing works for Ajax.ActionLink.  In Views\Dinners\RSVPStatus.ascx, change:

<%= Ajax.ActionLink( "RSVP for this event",
                     "Register", "RSVP",
                     new { id=Model.DinnerID }, 
                     new AjaxOptions { UpdateTargetId="rsvpmsg", OnSuccess="AnimateRSVPMessage" }) %>

to just:

<%= Ajax.ActionLink( "RSVP for this event",
                     MVC.RSVP.Register(Model.DinnerID),
                     new AjaxOptions { UpdateTargetId="rsvpmsg", OnSuccess="AnimateRSVPMessage" }) %>

You can also do the same thing for Url.Action().

Even the controller gets a piece of the action

As mentioned earlier, Controllers are no longer left out with this approach.

e.g. in Controllers\DinnersController.cs, you can replace

return View("InvalidOwner");

by

return View(MVC.Dinners.Views.InvalidOwner);

But to make things even more useful in the controller, you can let the T4 template generate new members directly into your controller class.  To allow this, you just need to make you controller partial, e.g.

public partial class DinnersController : Controller {

Note: you now need to tell the T4 template to regenerate its code, by simply opening the .tt file and saving it.  I know, it would ideally be automatic, but I haven’t found a great way to do this yet.

After you do this, you can replace the above statement by the more concise:

return View(View_InvalidOwner);

You also get to do some cool things like we did in the Views.  e.g. you can replace:

return RedirectToAction("Details", new { id = dinner.DinnerID });

by

return RedirectToAction(MVC.Dinners.Details(dinner.DinnerID));

 

How does the T4 template work?

The previous runtime-based T4 template was using reflection to learn about your controllers and actions.  But now that it runs at design time, it can’t rely on the assembly already being built, because the code it generates is part of that very assembly (yes, a chicken and egg problem of sort).

So I had to find an alternative.  Unfortunately, I was totally out of my element, because my expertise is in the runtime ASP.NET compilation system, while I couldn’t make use of any of it here!

Luckily, I connected with a few knowledgeable folks who gave me some good pointers.  I ended up using the VS File Code Model API.  It’s an absolutely horrible API (it’s COM interop based), but I had to make the best of it.

The hard part is that it doesn’t let you do simple things that are easy using reflection.  e.g. you can’t easily find all the controllers in your project assembly.  Instead, you have to ask it to give you the code model for a given source file, and in there you can discover the namespaces, types and methods.

So in order to make this work without having to look at all the files in the projects (which would be quite slow, since it’s a slow API), I made an assumption that the Controller source files would be in the Controllers folder, which is where they normally are.

As for the view, I had to write logic that enumerates the files in the Views folder to discover the available views.

All in all, it’s fairly complex and messy code, which hopefully others won’t have to rewrite from scratch.  Just open the .tt file to look at it, it’s all in there!

In addition to looking at the .tt file, I encourage you to look at the generated .cs file, which will show you all the helpers for your particular project.

Known issues

T4 file must be saved to regenerate the code

This was briefly mentioned above.  The T4 generation is done by VS because there is a custom tool associated with it (the tool is called TextTemplatingFileGenerator – you can see it in the properties).  But VS only runs the file generator when the .tt file changes.  So when you make code changes that would affect the generated code (e.g. add a new Controller), you need to explicitly resave the .tt file to update the generated code.  As an alternative, you can right click on the .tt file and choose “Run Custom Tool”, though that’s not much easier.

Potentially, we could try doing something that reruns the generation as part of a build action or something like that.  I just haven’t had time to play around with this.  Let me know if you find a good solution to this.

No refactoring support

This was also the case with the previous template, but it is worth pointing out.  Because all the code is generated by the T4 template, that code is not directly connected to the code it relates to.

e.g. the MVC.Dinners.Delete() generated method results from the DinnersController.Delete() method, but they are not connected in a way that the refactoring engine can deal with.  So if you rename DinnersController.Delete() to DinnersController.Delete2(), MVC.Dinners.Delete() won’t be refactored to MVC.Dinners.Delete2().

Of course, if you resave the .tt file, it will generate a MVC.Dinners.Delete2() method instead of MVC.Dinners.Delete(), but places in your code that call MVC.Dinners.Delete() won’t be renamed to Delete2.

While certainly a limitation, it is still way superior to what it replaces (literal strings), because it gives you both intellisense and compile time check.  But it’s just not able to take that last step that allows refactoring to work.

It is worth noting that using Lamda expression based helpers instead of T4 generation does solve this refactoring issue, but it comes with a price: less natural syntax, and performance issues.

Final words

It has been pretty interesting for me to explore those various alternative to solve this MVC strongly typed helper issue.  Though I started out feeling good about the runtime approach, I’m now pretty sold on this new design time approach being the way to go.

I’d be interested in hearing what others think, and about possible future directions where we can take this.

A T4 based approach to creating ASP.NET MVC strongly typed helpers

Update: Please see this newer post for the latest and greatest MVC T4 template

 

Earlier this week, I wrote a post on using a BuildProvider to create ActionLink helpers.  That approach was using CodeDom to generate the code, and there was quite some user interest in it (and Phil blogged it, which helped!).

Then yesterday, I wrote a post on the Pros and Cons of using CodeDom vs T4 templates for source code generation.  They are drastically different approaches, and while both have their strengths, T4 has definitely been getting more buzz lately.

The logical follow-up to those two posts is a discussion on using T4 templates to generate MVC strongly typed helpers.  The general idea here is to use the existing ASP.NET extensibility points (BuildProvider and ControlBuilder), but rely on T4 templates to produce code instead of CodeDom.  Hence, I called the helper library AspNetT4Bridge (I’m really good at naming things!).

As far as I know, this is the first time that T4 templates are executed dynamically inside an ASP.NET application, so let’s view this as an experiment, which has really not been put to the test yet.  But it is certainly an exciting approach, so let’s see where it takes us!

 

What scenarios are we enabling

Before we get too far into the technical details, let’s look at the various MVC scenarios that this is enabling.  But the great thing to keep in mind is that since T4 templates are so flexible, it is very easy to tweak the generated code or add additional scenarios.  Hence it is best to view those scenarios as examples of the type of things that you can achieve with this model, rather than as a final end result.

ActionLink helpers

First, we have the same scenario that my previous post covered, except that we’re now using T4 instead of CodeDom.  Instead of writing:

<%= Html.ActionLink("Edit", "Edit", new { id = item.ID })%>

You can write the more strongly typed (here, ‘Test’ is the name of the controller):

<%= Html.ActionLinkToTestEdit("Edit", item.ID) %>

And note that you get full intellisense in VS for those helpers!

Action Url helpers

This is similar to the previous section, except it covers the case where you need to generate raw URL’s rather than HTML <a> tags.  Instead of writing:

<%= Url.Action("Edit", new { id = item.ID }) %>

You can write:

<%= Url.UrlToTestEdit(item.ID) %>

Constants for View Names

That’s another case where you often end up hard coding view names as literal string, e.g.

<% Html.RenderPartial("LogOnUserControl"); %>

With the generated helpers, you get to write:

<% Html.RenderPartial(Views.Shared.LogOnUserControl); %>

And again, you get full intellisense:

Rendering helpers for strongly typed views

Assume you are using a strongly typed view.  You might write something like this (which is what the View wizard generates by default):

<label for="Name">Name:</label>
<%= Html.TextBox("Name", Model.Name) %>
<%= Html.ValidationMessage("Name", "*") %>

With the helpers, you can instead write:

<%= ViewModel.Name.Label() %>
<%= ViewModel.Name.TextBox() %>
<%= ViewModel.Name.Validation() %>

Here ViewModel is some name I came up with (well, Fowler did!) as the class that holds the helpers.  There may be better names for it!

Note that the helpers can be smart about what they offer based on the data type.  e.g. for a boolean field, you typically want a check box instead of a text box, and the helpers account for this:

 

What do you need to do to enable this in your MVC app?

There are five relatively straightforward steps you need to take to enable this.

1. Copy AspNetT4Bridge.dll into the bin folder of your app (you’ll get it by building the attached sample).

2. In the web.config file at the root of your app, register the BuildProvider for .tt files

    <compilation>
      ...
      <buildProviders>
        <add extension=".tt" type="AspNetT4Bridge.AspNetT4BridgeBuildProvider"/>
      </buildProviders>
    </compilation>

3. In the web.config in the Views folder (which you didn’t know existed, did you?), change the pageParserFilterType as follows:

pageParserFilterType="AspNetT4Bridge.Mvc.ViewTypeParserFilter"

4. Copy the Templates and App_Code folders (each containing a .tt file) from the attached sample app to the root of your app. In VS, blank out the Custom Tool for the .tt files if you see it set to TextTemplatingFileGenerator (see this post for some details on this).

5. If you have any strongly typed views (i.e. views the inherit ViewPage<T> and not just ViewPage), you need to change the Inherits attribute from System.Web.Mvc.ViewPage<...> to AspNetT4Bridge.Mvc.ViewPage<...>.  I know that’s a bit ugly, and we can think of ways to clean that up later.

 

So what about those T4 templates?

This post is supposed to be about using T4 templates, and so far we haven’t said a whole lot about them.  They are certainly the magic piece that makes all this work.  We are actually using two different .tt files, which cover two distinct scenarios:

Top level code

This applies to generated code that every page is referencing, and is used for the first three scenarios above (Action Links, Urls and View name constants).  The T4 file that does this is App_Code\StrongTypedLinkExtensions.tt.

As an aside, many people have a misconception that App_Code is just for web sites and not for web apps.  However, while it is rarely used, is is fully available.  It’s built with a reference to the bin assemblies, and all pages are built to a reference to it.

The way we make App_Code work with T4 templates is that we have a BuildProvider registered for the .tt extension.  This BuildProvider uses its own ITextTemplatingEngineHost in order to process the T4 templates into source code.  It then uses a CodeDom CodeSnippetCompileUnit to wrap this code into something CodeDom understands.  That’s why I called this AspNetT4Bridge, as it really bridges the CodeDom world of ASP.NET with the T4 world.  Fine, you come up with a better name! :)

Look for this logic in AspNetT4BridgeBuildProvider.cs.

Page level code

This applies to code that is specific to a given page, and that we just want to inject in there.  This is what’s uses for the view helpers like ViewModel.Name.TextBox() described above.  The T4 file that does this is Templates\MvcHtmlRenderHelpers.tt.

This works by using a ControlBuilder and its ProcessGeneratedCode method, which I discussed in a previous post.  The situation is similar to the App_Code case: we need to come up with some source code, we use T4 to generate it, and we wrap it in a CodeDom construct (in this case it’s a CodeSnippetTypeMember, because the code goes inside the class).

Look for this logic in T4CodeGenerator.cs.

But what the heck is in those T4 templates?

So I’ve talked about how they get hooked up and executed, but still haven’t said a word about what’s in them.  Well, I told you where they are, so go ahead and take a look!  e.g. check out App_Code\StrongTypedLinkExtensions.tt.

There is nothing really hard about how it works, but the mix of generator code and generated code certainly makes it confusing at first.  Do yourself a favor and install the free Clarius Community T4 editor.  It doesn’t do a whole lot, but the fact that it shows the two types of code (generator and generated) in different color goes a long way to make things clearer.

The way the Action Links and Url helpers are generated is fairly straightforward:

• It gets the list of referenced assemblies from the Host
• It uses reflection to figure out what types are controllers
• For each controllers, it finds that Action methods
• For each Action method, it generates both an ActionLine and a Url helper, custom made for that action method (hence strongly typed).

The View Name Constant helpers are also fairly simple.  They just look at what directories and files are under the Views folder, and generate constants accordingly.  I’ll show this part here as an example:

namespace Views {
<#
var viewsDir = HostingEnvironment.VirtualPathProvider.GetDirectory("~/Views");
    foreach (VirtualDirectory dir in viewsDir.Directories) {
#>
    public static class <#= dir.Name #> {
<#      foreach (VirtualFile file in dir.Files) {
            string viewName = Path.GetFileNameWithoutExtension(file.Name);
#>
        public const string <#= viewName #> = "<#= viewName #>";
<#      } #>
    }
    
<#  } #>
}

 

Go ahead, try changing them!

One of the big reason to use T4 over CodeDom for this code generation is to not ‘lock’ the kind of code that get generated into a binary.  Instead, you can very easily tweak it, remove things you don’t care about, and generate entirely new things.  All this really takes is an understanding of <# #> and <#= #> blocks, which is pretty easy when you already know how <% %> and <%= %> blocks work, since they’re pretty much the same thing!

 

Final thoughts

So here we are, dynamically executing T4 templates at runtime in an ASP.NET app.  One big caveat that I mentioned in my previous post is that you’re not really supposed to do that!  Copying from there:

CodeDom is part of the framework, while T4 is not.  That means that if you need to dynamically do this at runtime (as opposed to within VS), then T4 is not really an option.  Technically, it is possible to use it at runtime outside VS, but you either have to run on a machine with VS installed, or you have to copy Microsoft.VisualStudio.TextTemplating.dll into your project (which works, but is not officially supported – hopefully it will be at some point!).

So I wrote that it wasn’t an option, and next thing I blog about doing it.  What’s the deal?  Well, there are some things that are changing in VS 2010 that will help.  In particular, they are adding the concept of preprocessing T4 templates, while today you can only fully process them all at once.  With preprocessing, we’ll be able to:

• Let VS preprocess the T4 templates into some intermediate source code (that itself doesn’t requite the T4 runtime.
• Run this code at ASP.NET runtime to generate the code we care about and add it to the compilation (e.g. in App_Code as above).  This takes no dependency on the T4 runtime.

Another caveat I need to mention is that T4 templates can’t be fully executed in medium trust (because they need to compile code).  However, with the VS 2010 changes, that will no longer be an issue.

So while in the short term, we are breaking the rules a bit by doing this and there are some tough edges, it demonstrates some very useful concepts that can be fully supported later.

And if all else fails, I had a lot of fun playing with it all!

CodeDom vs T4: two approaches to Code Generation

There are many scenarios where the need to generate source code arises.  The MVC helpers I introduced in my last post is one such example.  Note that I am focusing on generating source code here, and not on scenarios where you may want to generate IL directly (which certainly do exist as well, but it’s a difference discussion).

To perform the code generation, there are several different approaches that can be used.  The most simplistic one is to use a plain StringBuilder and write whatever code you want to it.  It’s rather primitive, but for simple scenarios, it just might be good enough.

For more complex scenarios there are two widely different approaches that I will be discussing here: CodeDom and T4 templates.  Let’s start by introducing our two competitors.

CodeDom: this has been around since the Framework 2.0 days, and is used heavily by ASP.NET.  Its main focus is on language independence.  That is, you can create a single CodeDom tree and have it generate source code in C#, VB, or any other language that has a C# provider.  The price to pay for this power is that creating a CodeDom tree is not for the faint of heart!

T4 Templates: it’s a feature that’s actually part of Visual Studio rather than the framework.  The basic idea here is that you directly write the source code you want to generate, using <#= #> blocks to generate dynamic chunks.  Writing a T4 template is very similar to writing an aspx file.  It’s much more approachable than CodeDom, but provides no language abstraction.  Funny thing about T4 is that it’s been around for a long time, but has only been ‘discovered’ in the last year or so.  Now everyone wants to use it!

 

Let’s look at some actual samples

Let’s say that you’re trying to generate a class that has a method that calls Console.WriteLine("Hello 1") 10 times (with the number incrementing).  It’s a bit artificial, since you could just as well generate a loop which makes the call 10 times, but bear with me for the sake of illustration, and assume that we want to generate 10 distinct statements.

First, let’s tackle this with CodeDom.  In CodeDom, you don’t actually write code, but you instead build a data structure which later gets translated into code.  We could say that you write metacode.  Here is what it would look like:

using System;
using System.CodeDom;
using Microsoft.CSharp;
using Microsoft.VisualBasic;

class Program {
    static void Main(string[] args) {
        var codeCompileUnit = new CodeCompileUnit();

        var codeNamespace = new CodeNamespace("Acme");
        codeCompileUnit.Namespaces.Add(codeNamespace);

        var someType = new CodeTypeDeclaration("SomeType");
        someType.Attributes = MemberAttributes.Public;

        codeNamespace.Types.Add(someType);

        // Create a public method
        var method = new CodeMemberMethod() {
            Name = "SayHello",
            Attributes = MemberAttributes.Public
        };

        someType.Members.Add(method);

        // Add this statement 10 times to the method
        for (int i = 1; i <= 10; i++) {
            // Create a statement that calls Console.WriteLine("Hello [i]")
            var invokeExpr = new CodeMethodInvokeExpression(
                new CodeTypeReferenceExpression(typeof(Console)),
                "WriteLine",
                new CodePrimitiveExpression("Hello " + i));

            method.Statements.Add(new CodeExpressionStatement(invokeExpr));
        }

        // Spit out the code in both C# and VB
        (new CSharpCodeProvider()).GenerateCodeFromCompileUnit(codeCompileUnit, Console.Out, null);
        (new VBCodeProvider()).GenerateCodeFromCompileUnit(codeCompileUnit, Console.Out, null);
    }
}

You will either find this beautiful or atrocious depending on your mind set :)  Basically, writing CodeDom code is analogous to describing the code you want.  Here, you are saying:

Build me a public class named SomeType in the namespace Acme.  In there, create a public method named SayHello.  In there, add 10 statements that call Console.Write(…).

It certainly takes a fair amount of work to do something so simple.  But note that you’re not doing anything that ties you to C# or VB or any other language.  To illustrate this language abstraction power, this test app outputs the code in both C# and VB, with no additional effort.  That is one of the strongest points of CodeDom, and should not be discounted.

Now, let’s look at the T4 way of doing the same things.  You’d write something like this (just create a test.tt file in VS and paste this in to see the magic happen):

<#@ template language="C#v3.5" #>

namespace Acme {
    public class SomeType {
        public virtual void SayHello() {
<# for (int i=1; i<=10; i++) { #>
            System.Console.WriteLine("Hello <#= i #>");
<# } #>
        }
    }
}

As you can see, for the most part you’re just writing out the code that you want to generate, in the specific language that you want.  So for ‘fixed’ parts of the code, it’s completely trivial.  And when you want parts of the generation to be dynamic, you use a mix of <# #> and <#= #> blocks, which work the same way as <% %> and <%= %> blocks in ASP.NET.  Even though it’s much simpler than CodeDom, it can get confusing at times because you’re both generating C# and writing C# to generate it.  But once you get used to that, it’s  not so hard.

And of course, if you want to output VB, you’ll need to write the VB equivalent.  To be clear, only the ‘gray’ code would become VB.  The code in the <# #> blocks can stay in C#.  If you want the <# #> blocks to be in VB, you’d change the language directive at the top.  The generator code and the generated code are two very distinct things!

 

What are the Pros and Cons of the two approaches

Now that we’ve looked at samples using both techniques, let’s take a look at their Pros and Cons, to help you make on informed decision on which one is best for your scenario.

 

CodeDom benefits over T4

• Language abstraction: if you need your generated code to compile in both C# and VB (or other languages), then CodeDom has a definite edge.  With T4, you need to create and maintain one template for each language, which can become painful.  That it the core reason that we use CodeDom in ASP.NET (in fact, it was created for ASP.NET).  When we were working on 2.0, we were actually supporting 4 languages: C#, VB, J# and JScript.  Being able to write the logic once and have it be translated in all the languages was a huge productivity boost (and caused less bugs).  Note that we actually started with a more T4 like approach, and then switched to CodeDom when it became unmanageable.
• Framework support: CodeDom is part of the framework, while T4 is not.  That means that if you need to dynamically do this at runtime (as opposed to within VS), then T4 is not really an option.  Technically, it is possible to use it at runtime outside VS, but you either have to run on a machine with VS installed, or you have to copy Microsoft.VisualStudio.TextTemplating.dll into your project (which works, but is not officially supported – hopefully it will be at some point!).  I will demonstrate using T4 at runtime in future blog posts.

T4 benefits over CodeDom

• More approachable: writing CodeDom is hard and unnatural!  You know what the code you want to generate looks like, but you’re forced to describe it using an abstract data structure.  I’ve been writing CodeDom on and off for years, and I still often need to look up the docs to find the right thing to do (and I’m the one who initially created it! ;) ).  By contrast, T4 is much more natural, as you more or less write the code you more or less write the code you want to generate.
• More maintainable: sort of a continuation of the previous point, but still pretty key: if you need to refactor the structure of the generated code, it’s a heck of a lot faster to do it with T4 than with CodeDom.  Of course, this benefit quickly fades if you are maintaining multiple language version of your T4 templates (see above).
• Distributed as source: because a T4 template is just a text file, it is trivial for others to go in there and tweak the generation.  By contrast, CodeDom logic is typically built into your assembly, and cannot easily be changed by others.  e.g. you can’t easily change the code ASP.NET generates for an aspx page.  I view this as a huge benefit, as the need to tweak and extend the generated code is quite common and opens great extensibility scenarios.
• No language limitations: with CodeDom, you are limited to the language constructs that CodeDom supports.  Since it hasn’t been well updated as the languages evolved, this can  be a real issue.  e.g for my last post, I had to generate extension methods.  Since CodeDom doesn’t support that (nor static classes), I had to use a huge hack to make it work.  By contrast, with T4 you generate whatever you want and there are effectively no limitations.  It’s ‘future proof’ when it comes to new language features.
• It’s hot: everyone is doing T4 these days, so if you haven’t played with it yet, you probably should!

 

Conclusion

Hopefully, this gave you good overview of the two technologies.  Clearly, T4 is the more popular one lately, and I certainly try to use it over CodeDom when I can.  With proper framework support, it would become an even easier choice.

A BuildProvider to simplify your ASP.NET MVC Action Links

Update: Please see this newer post for the latest and greatest MVC T4 template

 

One downside of using Html.ActionLink in your views is that it is late bound.  e.g. say you write something like this:

<%= Html.ActionLink("Home", "Index", "Home")%>

The second parameter is the Action name, and the third is the Controller name.  Note how they are both specified as plain strings.  This means that if you rename either your Controller or Action, you will not catch the issue until you actually run your code and try to click on the link.

Now let’s take the case where you Action takes parameters, e.g.:

public ActionResult Test(int id, string name) {
    return View();
}

Now your ActionLink calls looks something like this:

<%= Html.ActionLink("Test Link", "Test", "Home", new { id = 17, name = "David" }, null) %>

So in addition to the Controller and Action names changing, you are vulnerable to the parameter names changing, which again you won’t easily catch until runtime.

One approach to solving this is to rely on Lambda expressions to achieve strong typing (and hence compile time check).  The MVC Futures project demonstrates this approach.  It certainly has merits, but the syntax  of Lambda expressions in not super natural to most.

Here, I’m exploring an alternative approach that uses an ASP.NET BuildProvider to generate friendlier strongly typed helpers.  With those helpers,  the two calls below become simply:

<%= Html.ActionLinkToHomeIndex("Home")%>
<%= Html.ActionLinkToHomeTest("Test Link", 17, "David")%>

Not only is this more concise, but it doesn’t hard code any of the problematic strings discussed above: the Controller and Action names, and the parameter names.

Steps to enable the ActionLink helpers in your MVC app

You can easily integrate these helpers in any ASP.NET MVC app by following three steps:

1. First, add a reference to MvcActionLinkHelper.dll in your app (build the project  in the zip file attached to this post to get it)

2. Then, register the build provider in web.config.  Add the following lines in the <compilation> section:

      <buildProviders>
        <add extension=".actions" type="MvcActionLinkHelper.MvcActionLinkBuildProvider" />
      </buildProviders>

3. The third step is a little funky, but still easy.  You need to create an App_Code folder in your app, and add a file with the .actions extension in it.  It doesn’t matter what’s in the file, or what its full name is.  e.g. add an empty file named App_Code/generate.actions.  This file is used to trigger the BuildProvider.

How does it all work?

I included all the sources in the zip, so feel free to look and debug through it to see how it works.  In a nutshell:

• When the first request is made at runtime, ASP.NET needs to build the App_Code assembly
• It finds our .actions file, which triggers the registered BuildProvider
• The BuildProvider goes through all the reference assemblies and looks for Controller classes
• It then generates a static class with extension methods for each action
• Since every aspx page is built with a reference to the App_Code assembly, all  the views are able to use the generated helpers.

Where is this going?

At this point, this is just a quick proof of concept.  There are certainly other areas of MVC where the same idea can be applied.  e.g. currently it only covers Html.ActionLink, but could equally cover Url.Action(), or HTML form helpers (standard and AJAX).

Please send feedback whether you find this direction interesting as an alternative to the Lambda expression approach.

A new flag to optimize ASP.NET compilation behavior

Update (4/22/2009)

Actually downloading this fix for 3.5SP1 has been a confusing topic.  Here are links from which you can download it directly without having to call Product Support.  Note that even though the links appear to refer to an unrelated issue, the binaries do in fact contain this fix.  Sorry for the confusion!

For XP and Server 2003: go here

For Vista and Server 2008: go here

----------------

Quick summary: we are introducing a new optimizeCompilations switch in ASP.NET that can greatly improve the compilation speed in some scenarios.  There are some catches, so read on for more details.  This switch is currently available as a QFE for 3.5SP1, and will be part of VS 2010.

To turn on this switch, add it to your compilation section:

  <compilation optimizeCompilations="true">

 

What prompted us to add this switch

The ASP.NET compilation system takes a very conservative approach which causes it to wipe out any previous work that it has done any time a ‘top level’ file changes. ‘Top level’ files include anything in bin and App_Code, as well as global.asax. While this works fine for small apps, it becomes nearly unusable for very large apps. E.g. a customer was running into a case where it was taking 10 minutes to refresh a page after making any change to a ‘bin’ assembly.

To ease the pain, we added an ‘optimized’ compilation mode which takes a much less conservative approach to recompilation. The drawback is that there are some edge cases where it does not do the right thing, which is why it had to be an optional setting. But those cases are rare enough to still make the feature extremely useful for people running into these issues.

 

Why it is not necessary to always fully recompile

All the pages (aspx/ascx/master) in an ASP.NET site are compiled with a reference to top level files (bin assemblies, App_Code and global.asax). So it feels natural to think that if any of these change, we need to recompile everything that depends on them.

However, thinking about it a little deeper reveals that it is not the case, and that more often than not pages don’t need to be recompiled. Let’s look at the various types of changes that may occur in the top level assemblies.

Change to a method implementation

That’s the easiest case, and is clearly harmless. You’re not changing any public API signature, so any page compiled against the old version will continue to run fine without requiring recompilation.

Addition of new APIs

You add a class, or some new public members on an existing class. Since these things didn’t exist before, none of the already-compiled pages are using them, and there is no need to recompile.

Addition of a CLR attribute to an existing member

This is the typical Dynamic Data scenario where you add attributes like DisplayName to properties. Since all the CLR attributes are discovered at runtime via reflection, pages don’t need to be recompiled.

Rename or deletion of APIs

Now it starts getting interesting. You had a method Foo() that some pages were calling, and now it’s gone. Clearly, the old page assembly is no longer valid. But let’s compare what happens if you recompile the page vs. if you don’t:

1. You recompile the page: you get a compile error because it’s using a non-existent method
2. You don’t recompile the page: you get a MissingMethodException, and the exception message includes the name of the missing method

So in both cases you get an error, but the error in the second case is not quite as clear as in the first case. But still, it is pretty usable, so it is not a big degradation.

Change to a type used in an existing API’s signature

This is the nastiest case. E.g. suppose your Foo() method returns an ‘int’, and you change it to return a ‘short’. Now suppose you have a page that calls ‘Response.Write(o.Foo())’. Let’s compare what happens if you recompile the page vs. if you don’t

1. You recompile the page: everything compiles and runs fine, because the same C# (or VB) code can be used to call the method, no matter what its return type is.
2. You don’t recompile the page: you get a MissingMethodException, because the previously compiled code cannot work with the new method signature.

Undeniably, this case is broken, and is the primary reason that we can’t turn on this optimization by default. Luckily, in practice this situation in not extremely common, which is why the optimization is still very usable for users that are aware of the limitations.

 

What exactly is the optimization doing

ASP.NET uses a per application hash code which includes the state of a number of things, including the bin and App_Code folder, and global.asax. Whenever an ASP.NET app domain starts, it checks if this hash code has changed from what it previously computed. If it has, then the entire codegen folder (where compiled and shadow copied assemblies live) is wiped out.

When this optimization is turned on (via <compilation optimizeCompilations="true" />), the hash no longer takes into account bin, App_Code and global.asax. As a result, if those change we don’t wipe out the codegen folder.

Of course, if App_Code or global.asax have changed, they get rebuilt. But pages that have been previously compiled don’t (unless they have themselves changed).

Note that this optimization doesn’t affect the app domain shutdown behavior. i.e. we still shut down the domain as soon as any top level file changes. It’s only the amount of compilation in the new domain that is affected.

Using a DomainService in ASP.NET and Dynamic Data

One of the big things that I discussed in my MIX talk is the new DomainDataSource control.  It is currently available in Preview form as part of ASP.NET Dynamic Data 4.0 Preview 3.  This can be confusing, because even though Dynamic Data makes use of DomainDataSource, DomainDataSource is absolutely not tied to Dynamic Data, and is fully usable in ‘regular’ aspx pages.

In addition to this preview, you’ll want to also install Microsoft .NET RIA Services in order to get some useful tooling.  This too can be confusing, because it makes it sound like it’s tied to RIA and Silverlight in some way, when in fact it is not.

The deal is that there is this new thing called a DomainService, which is essentially a Business Layer class that exposes CRUD methods.  Once you have a DomainService, you can use it in various scenarios, including ASP.NET apps and Silverlight apps.  Here, we will focus on its use from ASP.NET.  For more information about the Silverlight side of things, check out Nikhil’s post (and his MIX talk).

Important: after getting the ASP.NET DD preview and the RIA Services mentioned above, you’ll need to do a little extra step to avoid a tricky setup issue. Find System.Web.DynamicData.dll under DefaultDomainServiceProject\bin in the ASP.NET Preview zip file, and copy it over the one in \Program Files\Microsoft SDKs\RIA Services\v1.0\Libraries\Server (which is where the RIA install puts them).

Now you’re actually ready to start playing with DomainService.

First, you’ll want to make a copy of the whole DefaultDomainServiceProject folder so you can work with it without touching the ‘original’.  Then, just open DefaultDomainServiceProject.sln in VS.  Normally, this would be a Project Template, but right now we don’t have one.

 

Creating a DomainService

DefaultDomainServiceProject comes with a default DomainService, but to make things more interesting you should just delete it and recreate one from scratch.  Just delete DomainServices\NorthwindEntitiesDomainService.cs.

Now right click on DomainServices, choose Add New Item, and pick Domain Service Class.  Name it for instance Catalog.

You’ll see a New Domain Service dialog.  In here, do the following:

• Uncheck Enable Client Access. That’s only useful for the Silverlight scenario.
• Check Categories and Products.  For Products, also click the right check box to enable editing.
• Check ‘Generate associated classes’ at the bottom. This will be useful for Dynamic Data.

You should end up with a class that looks like this:

public class Catalog : LinqToEntitiesDomainService<NorthwindEntities> {

    public IQueryable<Categories> GetCategories() {
        return this.Context.Categories;
    }

    public IQueryable<Products> GetProducts() {
        return this.Context.Products;
    }

    public void InsertProducts(Products products) {
        this.Context.AddToProducts(products);
    }

    public void UpdateProducts(Products currentProducts, Products originalProducts) {
        this.Context.AttachAsModified(currentProducts, originalProducts);
    }

    public void DeleteProducts(Products products) {
        if ((products.EntityState == EntityState.Detached)) {
            this.Context.Attach(products);
        }
        this.Context.DeleteObject(products);
    }

Basically, it’s just a class for a few CRUD methods that you want to expose.  In this case, it’s working over an Entity Framework model (hence the base class), but it would look similar with Linq to SQL or other ORM technologies.

The important part is that all data access goes through those methods, so you have full control over it.  This is quite different from using LinqDataSource/EntityDataSource, where they talk directly to the DAL without going through your code.  You have to write a little code, but the extra control you get makes it well worth it!

 

Registering your DomainService with Dynamic Data

Before we get into writing ‘standard’ ASP.NET pages by hand (in future posts), we’ll use Dynamic Data to get started quickly.  To do this, simply add this line to global.asax (replacing the similar line that’s already there):

DefaultModel.RegisterContext(
    new DomainModelProvider(typeof(Catalog)),
    new ContextConfiguration() { ScaffoldAllTables = true });

 

Running it

Now you can just Ctrl-F5 and you should get a working Dynamic Data app.  Note how it only lets you do things for which you have CRUD methods.  e.g.  you can edit Products but not Categories.

To make things more interesting, try various things:

• Debug the app and set break points in your CRUD methods to see them getting called.
• Change GetProduct() to only return a subset of the products, and watch it affect the UI
• Change UpdateProduct() to modify the product before saving it,
• Add some Dynamic Data style metadata in DomainServices\Catalog.metadata.cs.  e.g. add a [Range(0, 100)] attribute to some integer field, and try an edit that violates the range

 

Conclusion

This was really just a quick intro to using DomainService in ASP.NET and Dynamic Data.  I’ll try to follow up with some posts that go into more details on how to use DomainDataSource directly in a page without involving any Dynamic Data.  To reiterate, though Dynamic Data works great with DomainDataSource, DomainDataSource completely stands on its own without Dynamic Data (as was shown in my MIX talk).

My MIX 2009 ASP.NET Data talk is available online

Last Friday, I gave a talk at MIX on various things that we’re working on in ASP.NET data land.  This includes both some Dynamic Data features and some features usable outside Dynamic Data.

The great thing about MIX is that they make all talks freely available online shortly after, and you can watch mine here.  Enjoy!

I’ll try to blog in more detail about some of the features discussed in the talk in the next few days.

Tips on getting your ASP.NET Dynamic Data questions answered quickly

When you run into an issue or have a question about ASP.NET Dynamic Data, the best place for it is the Dynamic Data Forum.  When you write your question, there are a few simple things that you can do to make it easier for the ‘experts’ to answer (and hence to get an answer quicker!).

1. Mention what environment you’re running in

a. What ORM framework you are using

Out of the box, Dynamic Data supports LINQ To SQL and Entity Framework.  While they seem similar on the surface, they are actually fairly different once you get a little deeper into the API.  Just mention which one you’re using and you’ll save a roundtrip.

b. What web server are you using

Most likely, you are using either “Cassini” (the test web server built in Visual Studio), IIS6 (on Pre-Vista OSes), or IIS7 (on Vista, Windows 7 or Windows Server 2008).  In some cases it makes a relevant difference, so it’s good to mention it.

c. What Dynamic Data release are you using

The official release of Dynamic Data is the one that comes with .NET Framework 3.5 SP1 (or Visual Studio 2008 SP1).  There are also various preview bits and samples that you can download from Codeplex.  Just mention exactly what you are using.  If you’re using some preview bits, you can include a link to them to them to remove any ambiguity.

2. When possible, report issues using a standard database

Obviously, we don’t expect you to use Northwind or Adventure Works in your real project.  But when you run into an issue with your custom schema, you should always check whether you are able to see the same thing with the standard DBs.  Those DBs have schemas which contain many of the common patterns that you may be using.  e.g. For One to Many relationship, you Northwind’s Product/Category, and for Many To Many, use Employees/Territories.

There are a few reasons why doing this is preferred.  First, most readers are familiar with those schemas, so they can make sense of the situation quicker without having to analyze a custom schema.  It also makes it a lot easier for people to try to reproduce the issue you’re seeing, since everyone has those sample databases.  And finally, if the issue calls for a sample that demonstrates a workaround, using the standard DB, makes it much more useful to everyone else who runs into it.

Clearly, there are some situation where your custom schema is just different and using the standard DBs will not work.  In those cases, just try to precisely describe your schema.  Including a diagram or the SQL DDL can help too.  In a nutshell, make sure that a reader not familiar with your database will understand enough of it to make sense of your question.

3. Try to isolate the issue

If an issue you see in a complex scenario can also be seen in a simpler scenario, it is always better to report it on the simpler scenario.  It makes the question shorter and easier to read, as it keeps it focused on the essential.

4. Disable AJAX partial rendering

If you’re getting Javascript errors, the first thing you should try to to disable AJAX partial rendering.  Please see this post for more details on this.

5. Include complete stack traces

If you’re getting an error in the browser, make sure you include the full stack trace that you see in there, and not just the text of the error.  Looking through a stack trace can reveal some important clues about the issue.

A helper to easily set up change notifications in Entity Framework

When you use Entity Framework, you can perform Insert/Update/Delete operations on your entities, and you eventually call ObjectContext.SaveChanges() to actually make it all happen.  The call to SaveChanges() is either explicit, or can happen implicitly when you use the EntityDataSource (e.g. within a Dynamic Data application).

Before SaveChanges() actually performs the operations, it gives you a chance to look at the entities, letting you modify them, and possibly cancel certain operations.  Unfortunately, doing this requires using some pretty ugly code, because the API is a little too low level.  Instead of nicely handing you the changes one by one, it just gives you the raw change list and lets you deal with it.

For instance, suppose you want to do something special when a Product is being updated.  You have to:

1. Register for the ObjectContext.SavingChanges event
2. In your handler, call ObjectStateManager.GetObjectStateEntries(EntityState.Modified)
3. Go through the whole list looking for those where stateEntry.Entity is a Product
4. Finally they can do what they want with the Product

While certainly possible, it’s a lot more pain than it should be!  To make this easier, we’ll write a little helper which lets you trivially listen to the changes you care about.

Before we get to the implementation of the helper, let’s look at the end result with the helper.  It lets you write something like this:

public partial class NorthwindEntities {
    partial void OnContextCreated() {
        // This line hooks up the change notifications
        this.SetupChangeNotifications();
    }
}

public partial class Products : INotifyUpdating {
    public bool Updating() {
        // Make a small change to test that you can affect the entity
        ProductName += "!";
        return true;
    }
}

public partial class Categories : INotifyDeleting {
    public bool Deleting() {
        // Prevent the deletion
        return false;
    }
}

Notice how you just need to implement a simple interface on the partial class on the relevant entity type in order to get notified.  You don’t need to know anything about the change list, or other such low level things.

Now let’s take a look at how the help works.  First, we define the Insert/Update/Delete interfaces:

public interface INotifyInserting {
    /// <summary>
    /// Return false to cancel the Insert operation
    /// </summary>
    bool Inserting();
}

public interface INotifyUpdating {
    /// <summary>
    /// Return false to cancel the Update operation
    /// </summary>
    bool Updating();
}

public interface INotifyDeleting {
    /// <summary>
    /// Return false to cancel the Delete operation
    /// </summary>
    bool Deleting();
}

Pretty simple stuff.  And now, the core logic that makes it all happen:

public static class EntityFrameworkHelpers {
    public static void SetupChangeNotifications(this ObjectContext context) {
        new ObjectContextHelper(context);
    }

    private class ObjectContextHelper {
        ObjectContext _context;

        public ObjectContextHelper(ObjectContext context) {
            // Keep track of the context
            _context = context;

            // Register for the SavingChanges event
            _context.SavingChanges += new EventHandler(Context_SavingChanges);
        }

        void Context_SavingChanges(object sender, EventArgs e) {
            // Go through all the Insert/Update/Delete changes and notify the user code if needed
            ProcessObjectStateEntries(EntityState.Added);
            ProcessObjectStateEntries(EntityState.Modified);
            ProcessObjectStateEntries(EntityState.Deleted);
        }

        private void ProcessObjectStateEntries(EntityState entityState) {
            // Go through all the entries
            foreach (ObjectStateEntry entry in _context.ObjectStateManager.GetObjectStateEntries(entityState)) {

                if (entry.Entity == null)
                    continue;

                // If the entity implements the interface (Insert, Update or Delete), call the
                // method.  If it returns false, cancel the operation
                bool proceedWithChange = true;
                switch (entityState) {
                    case EntityState.Added:
                        var notifyInserting = entry.Entity as INotifyInserting;
                        if (notifyInserting != null)
                            proceedWithChange = notifyInserting.Inserting();
                        break;
                    case EntityState.Modified:
                        var notifyUpdating = entry.Entity as INotifyUpdating;
                        if (notifyUpdating != null)
                            proceedWithChange = notifyUpdating.Updating();
                        break;
                    case EntityState.Deleted:
                        var notifyDeleting = entry.Entity as INotifyDeleting;
                        if (notifyDeleting != null)
                            proceedWithChange = notifyDeleting.Deleting();
                        break;
                }

                // If the method returned false, cancel the change
                if (!proceedWithChange)
                    entry.AcceptChanges();
            }
        }
    }
}

I’ll let the comments speak for themselves here.  But in any case, you don’t really need to look at the details of this code if you just want to use the helpers.

I attached a complete Dynamic Data solution that includes and uses the helpers, so just download it and start from there!

Two worlds of Dynamic Data customization: generic vs schema specific

There are many ways to customize a ASP.NET Dynamic Data site, which can sometimes be a bit overwhelming to newcomers.  Before deciding what customization makes sense for you, it is important to understand the two major buckets that they fall into:

1. Generic customization: things that apply generically to any table/column.
2. Schema specific customization: things that apply to specific tables and/or columns.

They can both very useful depending on your scenario, but it is important to understand how they are different in order to make the right choices.  The rule of thumb is that you want to stay in the world of generic customization whenever possible, and only use the schema specific customization when you have to.  Doing this will increase reusability and decrease redundancy.

We’ll now look at each in more details.

Generic customization

Generic customization includes everything that you can do without any knowledge of the database schema that it will be applied to.  That is, it’s the type of things that you can write once and potentially use without changes for any number of projects.  It lets you achieve a consistent and uniform behavior across an arbitrarily large schema, without needing to do any additional work every time you add or modify a table.

Here are some key types of generic customization:

Page Template customization

Under the ~/DynamicData/PageTemplates folder, you’ll find some default Page Templates like List.aspx and Edit.aspx.  If you look at them, you won’t find any references to a specific table/column.  Instead, they define the general layout and look-and-feel of your pages in an agnostic way.

e.g. if you look at List.aspx, you’ll see a GridView and a LinqDataSource (or EntityDataSource with Entity Framework), but the data source doesn’t have any ContextTypeName/TableName attributes, and the GridView doesn’t define a column set.  Instead, all of that gets set dynamically at runtime based on what table is being accessed.

You can easily make changes to these Page Templates, such as modifying the layout, adding some new controls or adding logic to the code behind.  It’s a ‘normal’ aspx page, so you can treat is as such, but you should never add anything to it that is specific to your schema.  When you feel the need to do this, you need to instead create a Custom Page (see below).

Field Template customization

Under the ~/DynamicData/FieldTemplates folder, you’ll find a whole bunch of field templates, which are used to handle one piece of data of a given type.  e.g. DateTime_Edit.ascx handles DateTime columns when they’re being edited.

As is the case for Page Templates, Field Templates should always be schema agnostic.  That is, you don’t write a field template that’s only meant to handle a specific column of a specific table.  Instead, you write a field template that can handle all columns of a certain type.

For example, in this post I describe a field template that can handle Many To Many relationships.  It will work for any such relationship in any schema (though it is Entity Framework specific).

Field Generator customization

Under Page Template above, I mentioned that the GridView didn’t specify a column set.  Instead, the way it works is that there is something called a Field Generator which comes up with the set of columns to use for the current table.

The idea is that the Field Generator can look at the full column set for a table, and use an arbitrary set of rules to decide which one to include, and in what order to include them.  e.g. it can choose to look at custom model attributes to decide what to show.

Steve Naughton has a great post on writing a custom field generator, so I encourage you to read that for more info on that topic.

Schema specific customization

Any time you make a customization that is specific to a certain table or column, you are doing schema specific customization.  Here are the main type of things you can do:

Custom Pages

When you want to have a very specific look for a certain page (e.g. for the Edit page for your Product table), a generic Page Template will no longer do the job.  At that point, you want to create a Custom Page.  Those live under ~/DynamicData/CustomPages/[TableName].  e.g. to have a custom page to edit products, you would create a ~/DynamicData/CustomPages/Products/Edit.aspx.

You can start out with the custom page being an identical copy of the page template, but then you can start making all kind of schema specific changes.  For instance, you could define a custom <Fields> collection in the DetailsView, at which point you no longer rely on the Field Generator (discussed below).  Taking this one step further, you can switch from a DetailsView into a FormView (or ListView), giving you full control over the layout of the fields.

Model annotations

Another very important type of schema specific customization is model annotation.  This is what most introductory Dynamic Data demos show, where you add CLR attributes to the partial class of your entity classes.

For instance, you can add a [Range(0, 100)] attribute to an integer field to specify that it should only take values between 0 and 100.  Dynamic Data comes with a number of built-in annotation attributes, and you can easily build your own to add news ways to annotate your model.

The general idea here is that it is cleaner to add knowledge at the model level than to do it in your UI layer.  Anything that you can add to describe your data in more details is a good fit for model annotations.

Conclusion

All of the different types of customization that I describe above have useful scenarios that call for them.  My suggestion is to get a good understanding of what makes some of them Generic while others are Schema Specific, in order to make an informed decision on the best one to use for your scenarios.

Time for a new useless blog subtitle

For the longest time, I had set my blog subtitle to “Dynamic Data and other ASP.NET topics”, which some might argue would not have won any originality contests.  On the bright side, it was a fine match for my equally original blog title: “David Ebbo's blog”.

So I went on a quest for a new subtitle as part of my 2009 New Year resolutions (ok, it’s the only one so far).  As for changing the Title itself, I’ll save that for New Year 2010.

My first stop in this memorable journey resulted in the use of the extremely witty word ‘Ebblog’.  While undeniably cool, I somehow decided to pass on it.  The future will probably not tell whether it was a good move.

Still looking to capitalize on my uniquely catchy last name, I came up with the soon-to-be memorable phrase: “The Ebb and Flow of ASP.NET”.

Some online dictionary defines it as “the continually changing character of something”.  I guess that’s not so bad.

And then there is Wikipedia, which defines it as “a form of hydroponics that is known for its simplicity, reliability of operation”.  Of course, I don’t have a clue what hydroponics means (and neither do you), but the rest sounds pretty good.

So that’s that.  Thanks for letting me waste your time.  By now, the momentum is clearly building for renaming the blog Title itself, but you’ll just have to wait another year for this thriller to unfold.