Marcin On ASP.NET

Keeping my eye on the dot

Posts
  • Marcin On ASP.NET

    Optimizing ASP.NET MVC view lookup performance

    • 6 Comments

    Earlier today Sam Saffron from the StackExchange team blogged about the performance of view lookups in MVC. Specifically, he compared referencing a view by name (i.e. calling something like @Html.Partial("_ProductInfo")) and by path (@Html.Partial("~/Views/Shared/_ProductInfo.cshtml")). His results indicate that in scenarios where a page is composed from many views and partial views (for example, when you are rendering a partial view for each item in a list) it’s more efficient to reference your views using the full path instead of just the view name.

    I know that bit of code quite well and it seemed strange to me that such a performance difference would exist because view lookups are cached regardless of how you reference them. Specifically, once an application is warmed up both the Razor and WebForms view engines use the lookup parameters to retrieve the cached results. If anything the cache key produced from a view path is usually longer than the one produced from a view name but even that should not have a measurable impact.

    While I cannot explain the differences that Sam is seeing (and since I know he is seeing them in production too I’m quite sure he has the application configured correctly for performance testing) I thought this would be a good opportunity to present the basics of MVC view lookup optimizations as well as a few additional techniques that people might not be familiar with.

    Hopefully you won’t go replacing all of your view references with full paths. Using view names is still easier and more maintainable. And before you do any perf-related changes you should always measure your application. It might turn out that it is fast enough for your needs already.

    Run in Release mode

    You should always make sure that your application is compiled in Release mode and that your web.config file is configured with <compilation debug="false" />. That second part is super-important since MVC will not do any view lookup caching if you are running your application in debug mode. This helps when you are developing your application and frequently adding/deleting view files, but it will kill your performance in production.

    This might seem like obvious advice, but I have seen even experienced devs get bitten by this.

    Use only the View Engines that you need

    I’ve mentioned this before but it’s worth repeating. The MVC framework supports having multiple view engines configured simultaneously in your application and will query each in turn when it is trying to find a view. The more you have the longer the lookups will take, especially if the view engine you are using is registered last. In MVC 3 we register two view engines by default (WebForms and Razor) and in all versions you might have installed 3rd party view engine such as Spark on nHaml. There is no reason to pay the performance price for something you are not using so make sure you specify only the view engines you need in your Global.asax file:

    protected void Application_Start() {
        ViewEngines.Engines.Clear();
        ViewEngines.Engines.Add(new RazorViewEngine());
        ...
    }

    Customize the view lookup caching

    By default (when running in Release mode, of course) MVC will cache the results of the lookups in the application cache available via HttpContext.Cache. While this cache works great and helps us avoid having to check for view files on disk there is also a cost associated with using it (this includes the cost of a thread-safe lookup as well as all the additional cache management such as updating entry expiration policies and performance counters).

    To speed things up you could introduce a faster cache in front of the application cache. Fortunately all view engines deriving from VirtualPathProviderViewEngine (that includes WebForms and Razor) have an extensibility point via the settable ViewLocationCache property.

    So we can create a new class that implements the IViewLocationCache interface:

    public class TwoLevelViewCache : IViewLocationCache {
        private readonly static object s_key = new object();
        private readonly IViewLocationCache _cache;
    
        public TwoLevelViewCache(IViewLocationCache cache) {
            _cache = cache;
        }
    
        private static IDictionary<string, string> GetRequestCache(HttpContextBase httpContext) {
            var d = httpContext.Items[s_key] as IDictionary<string, string>;
            if (d == null) {
                d = new Dictionary<string, string>();
                httpContext.Items[s_key] = d;
            }
            return d;
        }
    
        public string GetViewLocation(HttpContextBase httpContext, string key) {
            var d = GetRequestCache(httpContext);
            string location;
            if (!d.TryGetValue(key, out location)) {
                location = _cache.GetViewLocation(httpContext, key);
                d[key] = location;
            }
            return location;
        }
    
        public void InsertViewLocation(HttpContextBase httpContext, string key, string virtualPath) {
            _cache.InsertViewLocation(httpContext, key, virtualPath);
        }
    }

    and augment our view engine registration in the following way:

    protected void Application_Start() {
        ViewEngines.Engines.Clear();
        var ve = new RazorViewEngine();
        ve.ViewLocationCache = new TwoLevelViewCache(ve.ViewLocationCache);
        ViewEngines.Engines.Add(ve);
        ... 
    }

    This TwoLevelViewCache will work best in views that call the same partial multiple times in a single request (and should hopefully have minimum impact on simpler pages).

    You could go even further and instead of the simple dictionary stored in httpContext.Items you could use a static instance of ConcurrentDictionary<string, string>. Just remember that if you do that your application might behave incorrectly if you remove or add view files without restarting the app domain.

  • Marcin On ASP.NET

    Project Templates and preinstalled NuGet packages

    • 7 Comments

    10/22 Update: This post was written before this feature became a part of NuGet.
    This feature is now a part of NuGet and the latest version of this information can be found on the NuGet docs site.
    There is also a sample project available.

    Over the last few years the ASP.NET team has been including more and more libraries into the various project templates that we ship with Visual Studio and other products. The latest MVC 3 templates come with jQuery, jQuery UI, jQuery Validation, Modernizr, and Entity Framework. All this built-in functionality is great but one remaining snag is that upgrading all the packages by hand can be a pain. Of course the NuGet package manager is an excellent tool that solves the package upgrade problem but so far having project templates with preinstalled NuGet packages was not possible.

    This post describes a new feature that will be shipping in NuGet 1.5: preinstalled NuGet packages. Since it is not documented anywhere at this time the purpose of this post is to describe how to create a VS project template with NuGet packages. If you play around with it please also provide feedback so we can make changes/improvements.

    Authoring VS project templates is out of scope of this article but you can read more on how to create a project template directly using Visual Studio or using the Visual Studio SDK.

    Adding packages to a project templates

    NuGet 1.5 is not out yet but the feature has already been checked in and you can try it out by grabbing a recent NuGet preview build and installing the Visual Studio add-in (note that you might have to remove the previous version via the Extension Manager first).

    Preinstalled packages work using project template wizards. A special wizard gets invoked when the project gets instantiated. The wizard loads the list of packages that need to be installed and passes that information to the appropriate NuGet APIs. The project template needs to specify where to find the package nupkg files. Currently two package repositories are supported:

    1. Packages embedded inside of a VSIX package.
    2. Packages embedded inside of the project template itself.

    A frequent question is why not support downloading the nupkg files directly from http://nuget.org. We decided not to support such an option because users expect project templates to instantiate quickly and downloading files from the internet would slow things down. Also, it would not work on a plane or in other situations where a connection is not available.

    To add preinstalled packages to your project template you need to:

    1. Edit your vstemplate file and add a reference to the NuGet template wizard by adding a WizardExtension element:
      <WizardExtension>
          <Assembly>NuGet.VisualStudio.Interop, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null</Assembly>
          <FullClassName>NuGet.VisualStudio.TemplateWizard</FullClassName>
      </WizardExtension>
      NuGet.VisualStudio.Interop.dll is a new assembly that only contains the TemplateWizard class. This class is a simple wrapper that calls into the actual implementation that lives in NuGet.VisualStudio.dll. The assembly version will never change so that project templates continue to work with new versions of NuGet.
    2. Add the list of packages to install in the project:
      <WizardData>
          <packages>
              <package id="jQuery" version="1.4.4" />
          </packages>
      </WizardData>
      You can of course add multiple <package> elements. Both the id and version attributes are required. You will notice that this means that a specific version of a package will be installed, even if a newer version is available in the online package feed. We cannot just update to the latest version because a future version of a package might introduce incompatibilities that would break an older version of the project template. It is up to the developer to upgrade using the package manager.

    The remaining step is to specify the repository where NuGet can find the package files. As mentioned earlier, two package repository modes are supported:

    VSIX package repository

    The approach I recommend for deploying Visual Studio project templates is through a VSIX package (read more about VSIX deployment here). The VSIX method is preferable because it allows you to package multiple project templates together and allows developers to easily discover your templates using the VS Extension Manager or the Visual Studio Gallery. On top of that you can easily push updates to your users using the Extension Manager automatic update mechanism.

    1. To specify a VSIX as a package repository you modify the <package>element:
      <packages repository="extension"
                repositoryId="MyTemplateExtension.46431780-a0c7-44e0-83c7-5dbd985f8e49">
          ...
      </packages>

    The repository attribute specifies the type of repository (“extension”) while repositoryId is the unique identifier of your VSIX (i.e. the value of the ID attribute in the extension’s vsixmanifest file).

    You need to add your nupkg files as custom extension content and ensure that they are located under a folder called Packages within the VSIX package. You can place the nupkg files in the same VSIX as your project templates or you can have the packages be located in a separate VSIX if that makes more sense for your scenario (just note that you should not reference VSIXs you do not have control over since they could change in the future and your project templates would break).

    Template package repository

    If packaging multiple projects is not important to you (e.g. you’re only distributing a single project template), a simpler but also more limited approach is to include the nupgk files in the project template zip file itself.

    However, if you are bundling a set of project templates that relate to each other and share NuGet packages (e.g. you are shipping a custom MVC project template with versions for Razor, Web Forms, C#, and VB.NET), we do not recommend adding the NuGet packages directly to each project template zip file. It needlessly increases the size of the project template bundle.

    1. To specify the project template as a package repository you modify the <package>element:
      <packages repository="template">
          ...
      </packages>

    The repository attribute now has the value “template” and the repositoryId attribute is not longer required. The nupkg files need to be placed into the root directory of the project template.

  • Marcin On ASP.NET

    Granular Request Validation in ASP.NET MVC 3

    • 7 Comments

    12/10 Update: In MVC 3 RC 2 SkipRequestValidationAttribute got renamed to AllowHtmlAttribute. I have updated the examples below.

    A little while ago I wrote a blog post describing granular request validation that shipped in MVC 3 Beta. However, since then we have changed the API for this feature and that post is no longer valid. In this post I will present the new API which is usable in the recently-shipped MVC 3 Release Candidate.

    But first: a quick refresh on request validation and why it’s great to make it granular. Request validation is a feature of ASP.NET that analyzes the data that a browser sends to the server when a user interacts with your site (such as form or query string data) and rejects requests that contain suspicious input that looks like html code (basically anything with a ‘<’). This protects you from HTML injection attacks such as cross-site scripting (XSS). It is enabled by default, however in previous versions it was an all-on-or-off feature, meaning that if you want to be able to accept HTML-formatted input from your users in just one field you had to completely turn this protection off. This in turn meant that you now had to validate every bit of data that came from the client.

    AllowHtmlAttribute SkipRequestValidationAttribute

    In MVC 3 we are introducing a new attribute called AllowHtmlAttribute. You can use this attribute to annotate your model properties to indicate that values corresponding to them should not be validated. Let’s take this User model and UserController as an example:

    public class User {
        public int Id { get; set; }
        public string Name { get; set; }
        public string Email { get; set; }
        [AllowHtml]
        public string Description { get; set; }
        [AllowHtml]
        public string Bio { get; set; }
    }
    
    public class UserController {
        [HttpPost]
        public ActionResult Update(User user) {
            // update user database
        }
    }

    I have annotated the Description and Bio properties to indicate they should not be request-validated. Now when the Update action method gets invoked these two properties on the User object will not be validated and any HTML they might contain will be passed straight through to the action method. However, everything else will still go through request validation and requests that contain suspicious content in the Name or Email fields will get rejected.

    ValidateInputAttribute

    AllowHtmlAttribute can only be applied to properties of a model class. For other request validation scenarios the existing ValidateInputAttribute is still helpful. For example, you can use it to disable request validation for action methods that bind to a loose collection of parameters:

    [ValidateInput(false)]
    public ActionResult Update(int userId, string description) {
    }

    Now when the parameters of the Update method get bound request validation will not be performed. You can apply ValidateInput to action methods as shown above or to the entire controller to affect all of its action methods.

    ValidateInput is also more usable in MVC 3. In MVC 2 running on .NET 4 you had to set requestValidationMode="2.0" in order to turn request validation off. In MVC 3 this is no longer necessary.

  • Marcin On ASP.NET

    ASP.NET MVC 3 Performance – on par with MVC 2

    • 9 Comments

    Now that the final version of MVC 3 has been released I think information about the performance characteristics of this latest installment would be very useful for those weighing the pros and cons of updating their MVC 2 applications. In this post I will talk about what you might expect when moving to MVC 3. The short answer is things should be about the same.

    Of course that short statement is not accurate because the only correct answer to such performance questions is: “it depends”. It depends on what your application does and which features it is using: database connections, business logic, output caching, html helpers, etc. I say this because in a moment I will present some numbers and they might not correspond to what you would see in your application. But they should provide you with a good starting point.

    The more detailed answer is that some applications will perform better while others will perform worse. That is because as we added new features (dependency injection, improved filters, etc.) some components of MVC 3 got slower than in MVC 2 simply because they are now doing more work. At the same time we have introduced targeted performance optimizations which improved other components. The only definite way you can know the impact on your application is if you carefully measure performance yourself. At the end of this post I will provide a rough list of some of the things that have gotten faster or slower.

    The Setup

    The application I will use for the tests is the default “Internet Application” project template that ships with every installation of MVC. While quite small, it is still a pretty good choice because it uses a wide variety of MVC features. This means that the test exercises a large part of the framework without focusing on any individual features.

    For my tests I have the following applications:

    1. MVC 2 Aspx Internet Application project running against MVC 2 runtime
    2. MVC 2 Aspx Internet Application project running against MVC 3 runtime
    3. MVC 3 Aspx Internet Application project running against MVC 3 runtime
    4. MVC 3 Razor Internet Application project running against MVC 3 runtime

    I modified each of these applications in the following ways to ensure consistency and remove unnecessary noise from the results:

    • Set debug=”false” in the root Web.config file.
    • Changed application to build in Release mode.
    • Changed the FormsAuthenticationService and MembershipService classes to return hardcoded values (I do not want to test database connection latency).

    For each of the applications I executed the following sequence of requests and recorded the throughput (number of requests per second). The sequence ran for about 1 hour in our performance testing lab (meaning that the results should be amortized with respect to any other OS tasks that might have occurred on the server).

    1. GET /Home/Index (arrive at site)
    2. GET /Account/LogOn (click on “log on” link)
    3. POST /Account/LogOn (attempt to log on with invalid credentials)
    4. GET /Account/Register (click on “register” link)
    5. POST /Account/Register (submit the registration form)

    The Results

    ApplicationRequests/second (higher is better)% change (wrt baseline)
    1. MVC 2 Aspx on v2 8135 (baseline) n/a
    2. MVC 2 Aspx on v3 7987 -1.82%
    3. MVC 3 Aspx on v3 7457 -8.33%
    4. MVC 3 Razor on v3 7136 -12.28%

    Note: The above results were captured on a 12-core server and should only be used for comparative purposes. Absolute RPS values on your own machine might de vastly different but their relative distribution should remain the same. 

    Applications 1 vs. 2

    As you can see from comparing the results for Application 1 and Application 2 the difference is quite small. In fact –1.82% falls within the margin of error. Application 1 and Application 2 are the same identical application, except 1. references System.Web.Mvc.dll version 2.0 and 2. references System.Web.Mvc.dll version 3.0. These results are why we say that MVC 3 is on par with the performance characteristics of MVC 2.

    Of course followers of Scott Guthrie will point out that with the RC2 release he claimed that MVC 3 was faster than MVC 2. While that was true at the time it is not correct any longer for RTM. We had to undo a pretty significant performance fix related to model metadata caching because it caused some functional bugs (after all it does not matter how fast your application is if it does not actually work correctly). However, we will have that improvement available as an add-on in MVC Futures and we will roll it into MVC 4.

    Applications 2 vs. 3

    Comparing Application 2 with Application 3 shows that 3. is –6.64% slower. However, in this case the runtimes are the same. The difference is in the project templates used. The project template that ships with MVC 3 has more functionality enabled by default. Most notably it has unobtrusive client-side validation enabled; MVC 2 project templates did not have client-side validation enabled. The extra time is taken up inspecting the model metadata and emitting additional markup and javascript code. Of course while page rendering will be slower, the overall result of enabling client-side validation should be reduced load on the server as the majority of invalid inputs will be caught on the client thus avoiding the need for unnecessary roundtrips.

    Applications 3 vs. 4

    The comparison of Application 3 and Application 4 will be of interest to those looking at adopting the new Razor view engine. Application 4 is –4.3% slower than Application 3. This shows that using Razor does have a performance cost. However, that difference is not that big and is outweighed by the elegance and increased readability of the new view engine. Razor is also a new technology and I am sure that as it matures we will be able to optimize it even further.

    The Changes

    At the beginning of this post I promised that I would list how the various features or components of MVC changed in their performance characteristics. This list is compiled from our internal performance tests as well as change logs. It is meant to be taken as informational only. Your results may vary and depends on how you actually use each feature and what else you have happening in your application

    Things that got faster (in no particular order):

    • Expression caching (internal subsystem that helps in the performance of methods that accept lambdas)
    • URL generation
    • Action lookup
    • Filter invocation
    • TagBuilder

    Things that got slower (also in no particular order):

    • Things for which Dependency Injection entry points have been exposed
    • Auto-generated editor and display helpers
    • Model binding

    Of course I am not going to tell you by how much things have changed because it depends... you will have to measure yourself.

  • Marcin On ASP.NET

    Dynamic Data samples: Extending the FilterRepeater

    • 2 Comments

    6/25 Update: This sample has become part of the Dynamic Data Futures project on codeplex. You can still use this code but you should check out the Futures project as it contains a lot more. The key conecpts are the same, though some names or APIs might have changed.

    Note: This post is part of a series, see the list of other Dynamic Data samples.

    The FilterRepeater is a control in Dynamic Data that is responsible for automatically emitting a list of filter controls that can be used to filter data rows displayed for a table. The filters work by providing a list of where parameters to the data source for each supported column type. By default Dynamic Data supports foreign key and boolean columns and renders them using a DropDownList. This and upcoming posts will talk about ways in which you can add filtering for other column types and how to modify the filter UI.

    All of the code referenced here is part of the AdvancedFilterRepeaterSample solution.To run the sample open the solution in Visual Studio 2008 SP1 Beta or later and run AdvancedFilterRepeaterSite (note: it might not be the default project for the solution, so right click on Default.aspx inside the project and choose View in Browser). Navigate to the list page for the Products table where you can experiment with some new filtering options. For example, type "con" into the Categories filter. You should see an AJAX autocomplete drop-down instead of the default drop-down list.

    Introduction to FilterRepeater

    FilterRepeater is a specialized Repeater control that will automatically bind to a collection of filterable columns for the given request's table: if a request comes in for /Products/List.aspx, it will bind to the columns in the Products table. The table is chosen based on the route that the request matched, or it can be overridden by setting the TableName and ContextTypeName properties.

    To work correctly FilterRepeater expects its ItemTemplate to have correct content. Specifically, it requires a control with a known ID ("DynamicFilter" by default) that derives from FilterUserControlBase. When databinding occurs FilterRepeater initializes an instance of the filter control with information about the column to filter.

    Below is a snippet of the code that ships in the default Dynamic Data templates:

    <asp:FilterRepeater ID="FilterRepeater" runat="server">
    <
    ItemTemplate>
    <
    asp:Label runat="server" Text='<%# Eval("DisplayName") %>' />
    <asp:DynamicFilter runat="server" ID="DynamicFilter"
    OnSelectedIndexChanged="OnFilterSelectedIndexChanged" />
    </
    ItemTemplate>
    <
    FooterTemplate><br /><br /></FooterTemplate>
    </
    asp:FilterRepeater>

    FilterRepeater version 2.0 (well, more like 1.1 really)

    The DynamicDataExtensions project contains a simple extension of FilterRepeater called AdvancedFilterRepeater. It is not really that advanced (I started with that name and never bothered to change it) as it overrides only one method: GetFilteredColumns. This method is what is used as the databinding source. Below is a slightly simplified source of how it is implemented:

    public class AdvancedFilterRepeater : FilterRepeater {
    protected override IEnumerable<MetaColumn> GetFilteredColumns() {
    return Table.Columns.Where(c => IsFilterableColumn(c)).OrderBy(column => column, new FilterOrderComparer());
    }

    protected bool IsFilterableColumn(MetaColumn column) {
    if (column.IsCustomProperty) return false;

    var filterAttribute = column.Attributes.OfType<FilterAttribute>().FirstOrDefault();
    if (filterAttribute != null) return filterAttribute.Enabled;

    if (column is MetaForeignKeyColumn) return true;

    if (column.ColumnType == typeof(bool)) return true;

    return false;
    }

    private class FilterOrderComparer : IComparer<MetaColumn> {
    // implementation omitted for brevity
    }
    }

    The Table property is automatically populated with the right MetaTable instance based on the table resolution rules mentioned in the previous section. The rest is a simple LINQ extension method query and some boolean logic to choose the filterable columns. The only way this deviates from the default process is the addition and handling of FilterAttribute, which is a new attribute written for the purpose of this sample. It combines the roles of UIHintAttribute and ScaffoldColumnAttribute from Dynamic Data field templates and should be applied to columns in an analogous manner. Here's the full signature:

    public sealed class FilterAttribute : Attribute {
    public string FilterControl { get; set; }
    public int Order { get; set; }
    public bool Enabled { get; set; }
    }

    The FilterControl property lets you specify which user control to use as the filter while the Enabled property lets you omit a given column's filter in the AdvancedFilterRepeater. The Order property let's you specify an ordering weight.

    Specifying custom filter controls

    The reason why the implementation of AdvancedFilterRepeater is so simple is because a lot of work is done in two other components:

    • FilterFactory is the equivalent of FieldTemplateFactory for filters. It uses the information in a column's FilterAttribute to determine which filter user control to instantiate for the column. It takes the value of the attribute's FilterControl property and looks for a user control located at ~\DynamicData\Filters\{FilterControl}.ascx. If a filter attribute is not specified ~\DynamicData\Filters\Default.ascx is used, which is identical to the FilterUserControl.ascx that is part of the default Dynamic Data template.
    • DelegatingFilter is a control that derives from FilterUserControlBase (which means that it is itself a filter) and delegates the filtering behavior to another filter control. It uses the FilterFactory class to determine which control to use.

    In order to take advantage of AdvancedFilterRepeater and DelegatingFilter you need to replace the existing FilterRepeater in your List.aspx page template with the following code:

    <asp:AdvancedFilterRepeater id="AdvancedFilterRepeater" runat="server">
    <
    HeaderTemplate>
    <
    table>
    </
    HeaderTemplate>
    <
    ItemTemplate>
    <
    tr>
    <
    td valign="top"><%# Eval("DisplayName") %>:</td>
    <
    td><asp:DelegatingFilter runat="server" ID="DynamicFilter"
    OnSelectionChanged="OnFilterSelectionChanged" />
    </
    tr>
    </
    ItemTemplate>
    <
    FooterTemplate>
    </
    table>
    </
    FooterTemplate>
    </
    asp:AdvancedFilterRepeater>

    This template is pretty similar to the default one seen earlier in this post except it wraps each filtered column in a table row for cleaner formatting (yes, using tables for layout is wrong but this post is not about layouts). And of course DynamicFilter has been replaced with DelegatingFilter.

    To take advantage of the custom filter controls included in the sample such as autocomplete filter or cascading filter you need to annotate your data model with FilterAttribute. Here's a snippet of how it is done in the sample:

    [MetadataType(typeof(Product_MD))]
    public partial class Product { }

    public class Product_MD {
    // Display the Category and Supplier filters using the Autocomplete.ascx filter control
    [Filter(FilterControl = "Autocomplete")]
    public object Category { get; set; }
    [Filter(FilterControl = "Autocomplete")]
    public object Supplier { get; set; }

    // Display the Discontinued filter using the BooleanRadio.ascx filter control
    // Make sure the Discontinued filter is displayed first
    [Filter(FilterControl = "BooleanRadio", Order = 1)]
    public object Discontinued { get; set; }

    // Display the UnitsInStock filter using Integer.ascx filter control
    [Filter(FilterControl = "Integer")]
    public object UnitsInStock { get; set; }
    }

    When you run the application and view the list page for the Products table the appropriate filters get inserted automatically. Future posts will discuss how the more advanced filters are implemented.

Page 2 of 5 (25 items) 12345