Henrik's Blog

ASP.NET Web API and HTTP Range Requests (Link)

Henrik F Nielsen — Fri, 23 Nov 2012 20:40:44 GMT

Posted the blog ASP.NET Web API and HTTP Byte Range Support on the .NET Web Development and Tools Blog explaining how to support HTTP Range requests with ASP.NET Web API.

Henrik

An ASP.NET Open Source How-To Decoder Ring (Link)

Henrik F Nielsen — Wed, 19 Sep 2012 17:44:53 GMT

Posted the blog An ASP.NET Open Source How-To Decoder Ring on the .NET Web Development and Tools Blog explaining how to get started with ASP.NET and Open Source development by using the latest nightly drops, compiling it yourself, or creating pull requests.

Have fun!

Henrik

Link to Getting Symbols and Source with ASP.NET Nightly NuGet Packages

Henrik F Nielsen — Thu, 30 Aug 2012 22:31:50 GMT

Posted the blog Getting Symbols and Source with ASP.NET Nightly NuGet Packages on the .NET Web Development and Tools Blog.

Henrik

Link to List of ASP.NET Web API and HttpClient Samples

Henrik F Nielsen — Sun, 26 Aug 2012 18:20:33 GMT

Posted the blog List of ASP.NET Web API and HttpClient Samples on the .NET Web Development and Tools Blog

Have fun!

Henrik

Getting Started With ASP.NET Web Stack Source on CodePlex

Henrik F Nielsen — Fri, 17 Aug 2012 15:46:00 GMT

Note: Updated instructions to target Visual Studio 2012. While you can still use Visual Studio 2010 there are big advantages to using VS 2012.

Note: Please also check out the documentation on Codeplex for how to build.

As you probably know by now, ASP.NET MVC, Web Pages, Web API is now all open source (if you haven’t heard then please read ScottGu’s blog). What this means is that the source for all our runtime components is now available on CodePlex for you to review and use as well as suggest changes to in the form of issues or pull requests. You can also track recent code submits and check that the code looks good.

This blog will give you some hints and tricks for getting started working with the source code. Note you still have the official installers which contain the usual MSI with NuGet packages. In fact, if you don’t intend to work directly with the source, I would go with the installer because it provides tooling support for Visual Studio (project templates, etc.) which are not available in the CodePlex repository.

If you want to live on the edge (as in without support) then you can also use the nightly NuGet packages.

Before getting started, you should take a quick look at the ways in which you can contribute. It contains a lot of good information about how you can contribute, what the process is for contributing, and how contributions are evaluated. The more you familiarize yourself with the process, the less chance of getting disappointed later.

Prerequisites

Before getting the source you should setup your dev machine with the right tools to compile the source (obviously you shouldn’t be running anything critical on the dev machine). The prerequisites listed are not the only way of setting up your developer environment but the purpose here is to reduce the steps and improve the chance of getting off to a good start.

Visual Studio

You can use either a Visual Studio 2012 full edition or Visual Studio Express 2012 for Web edition to open and compile the project solution.

Git

Git is the version control system used to manage the source. If you are not already a Git expert then you should check out the Git site for how to get started. Git is great to work with but if you are used to more traditional versioning control systems then it will take a little time to get “re-programmed”. Here are the steps for the Git tools that I use and recommend you install:

Git for Windows (use one of the binary downloads). Using the default setup options should be fine – for reasons that will become clear later I don’t recommend using a Windows Command Prompt for issuing Git commands.
TortoiseGit (use one of the binary downloads) which integrates Git with Windows Explorer allowing you to see the state of your local repository clone in an graphical manner.
PowerShell Git extensions. Even if you are not a PowerShell user I recommend this for executing Git commands from the command line as it gives you critical clues as to what branch you are on and what state you are in. You can always use a regular shell for everything other than Git commands if you can’t find your way around PowerShell. The installation here is slightly more complicated but if you follow the readme then you should be fine.

CodePlex

Make sure you have a CodePlex account – you need it to participate in discussion forums, enter work items, and create projects. Once you have an account, I would encourage you to browse around the CodePlex project – there’s a lot of good information. Also you can sign up for notifications or subscribe to RSS feeds for discussions, source code changes, and updates to issues.

Cloning the Repository

Ok, now that you have all the prerequisites it is time to get the source. We use PowerShell and then clone the repository like this:

git clone https://git01.codeplex.com/aspnetwebstack.git

The result of the command should look like this:

To sync to the latest changes use the commands

git checkout master
git pull --rebase origin

That will pull down all changes to the CodePlex repository and get your master branch up to date.

Skipping Strong Name Verification

By default the projects are delay signed which means that they are not fully signed. Unless we explicitly turn off strong name verification for the compiled assemblies, building will cause a set of strong name verification failures. To only disable strong name verification for the WebStackRuntime assemblies, download the SkipStrongNames zip file and unzip it. Then run the command

.\SkipStrongNames.exe /e

The result should look like this:

You can later run .\SkipStrongNames.exe /d to re-enable strong name verification if/when you want to do so.

Compiling from Command Line

To compile and run unit tests as well as StyleCop and FxCop directly from the command line, simply run this from the aspnetwebstack folder:

.\build.cmd

This is the authoritative way to build ASP.NET Web Stack. If the build is successful you should see BUILD SUCCESSFUL when it has completed.

Compiling from Visual Studio

In Visual Studio Express you can build the source but not run the unit tests. In full Visual Studio you can both compile and run unit tests – just install xUnit.net runner for Visual Studio 2012.

In either case, just open the solution Runtime.sln and compile as normal. If you haven’t compiled from command line before then you need to enable NuGet Package Restore by opening the Tools/Options/Package Manager menu and make sure Package Restore is enabled:

That’s it – you are now free to move around the cabin!

You may also want to check out the blog Contributing to ASP.NET Web Stack Source on CodePlex which explains how to setup and work with a fork.

Henrik

ASP.NET Web API Released and a Preview of What’s Next

Henrik F Nielsen — Wed, 15 Aug 2012 20:54:38 GMT

Today we announced the release of Visual Studio 2012 and .NET 4.5 which of course includes ASP.NET Web API. There is a ton of new features in VS 2012 and I highly recommend checking out the videos by Scott Hanselman demonstrating a broad set of features of VS 2012 and .NET 4.5.

In addition to shipping as part of Visual Studio 2012 you can also download Web API for Visual Studio 2010 as a standalone installer and as always, you can find us in our Open Source home on codeplex where you can see the source, contribute pull requests, and log issues. The official release (RTM) corresponds to this branch in the source repository.

Further, you can find a bunch of ASP.NET Web API and HttpClient samples demonstrating many of the features. The samples also live on codeplex so that you can download them and work with them. Here you will find samples for both VS 2010 and VS 2012 illustrating a variety of features of both ASP.NET Web API and HttpClient.

In addition to all this we also today release four preview NuGet packages which build on the ASP.NET Web API RTM bits but provide new features. The previews are explicitly marked as alpha releases as they are early builds. The previews are continuously updated and you can get the latest builds from our nightly NuGet feed, please see Using Nightly ASP.NET Web Stack NuGet Packages for details.

OData

The Microsoft ASP.NET Web API OData package contains everything you need to create OData endpoints using ASP.NET Web API and to support OData query syntax for your Web APIs. It supersedes the early query support in ASP.NET Web API Beta with much better support for both querying as well as sending and receiving OData formatted payloads. For a detailed description of the OData support, please see the blog OData support in ASP.NET Web API.

There are two samples demonstrating how to use the OData package that both available in source as part of ASP.NET Web API and HttpClient samples:

ODataQueryableSample

This sample shows how to introduce OData queries in ASP.NET Web API using either the [Queryable] attribute or by using the ODataQueryOptions action parameter which allows the action to manually inspect the query before it is being executed. The sample illustrates queries using $orderby, $skip, $top, and $filter as well as any() and all().

ODataServiceSample

This sample illustrates how to create an OData service consisting of four entities and three ApiControllers. The controllers provide various levels of functionality in terms of the OData functionality they expose. In addition the OData service exposes a $metadata document which allows the data to the consumed by WCF Data Service clients and other clients that accept the $metadata format.

The data model contains three entities (Supplier, ProductFamily, and Product) connected in various ways and uses three sample controllers demonstrating various levels of exposing OData ranging from Query to Update and Delete.

Furthermore, the ODataService exposes a Service Document (aka. $metadata document) that lists all the top-level entities so clients can discover them. This enables OData clients to discover and consume OData Services exposed through ASP.NET Web API.

Help Page

The Microsoft ASP.NET Web API Help Page package automatically generates help page content for Web APIs on your site. Visitors to your help page can use this content to learn how to call your web APIs. Everything generated by the help page is fully customizable using ASP.NET MVC and Razor. ASP.NET Web API Help Page is a great addition to any ASP.NET Web API project.

For more information and a nice video walkthrough, please see the blog Introducing the ASP.NET Web API Help Page (Preview)

Tracing

The Microsoft ASP.NET Web API Tracing package allows the ASP.NET Web API framework to trace to System.Diagnostics.Trace so that you can now get trace messages from the entire ASP.NET Web API message pipeline from when a request starts being processed to a response is generated.

When installing this package in your project, it adds the following source file was added to your project:

App_Start\TraceConfig.cs

To enable tracing in your application, simply add the following line of code to your startup code (for example in WebApiConfig.cs or Global.asax.cs in an MVC 4 project):

TraceConfig.Register(config);

where 'config' is the HttpConfiguration instance for your application. When running your application you can now see trace output written toSystem.Diagnostics.Trace.

For more information about tracing, please see the blog ASP.NET Web API Tracing (Preview).

System.Net.Http.Formatting for Windows 8

The Microsoft ASP.NET Web API Client Libraries package adds support for formatting and content negotiation to System.Net.Http when writing Windows Store Apps. It includes support for JSON, XML, form URL encoded data, as well as MIME multipart. This package requires Visual Studio 2012 and Windows 8.

We are looking forward to your feedback – both on RTM as well as these previews.

Have fun!

Henrik

ASP.NET Web API Samples on Codeplex

Henrik F Nielsen — Wed, 15 Aug 2012 09:02:00 GMT

Here’s how to get up and running with the ASP.NET Web API and HttpClient samples that you can find in our samples repository on aspnet.codeplex.com. For a list of the samples please see List of ASP.NET Web API and HttpClient Samples as well as our list of samples on www.asp.net.

Prerequisites

The samples are split into two parts: those that target .Net 4 and those that target .Net 4.5. The latter use the async/await keywords introduced in Visual Studio 2012 which makes programming with asynchronous tasks much simpler.

Many of the samples use self host for starting ASP.NET Web API (that is, they run outside ASP.NET). These samples all use the local TCP port 50231 to listen for HTTP traffic. In order to start listening on that port you have to register an HTTP namespace with http.sys (or alternatively run Visual Studio as Administrator). You register the namespace from an elevated command prompt like this:

netsh http add urlacl http://+:50231/ user=%USERDOMAIN%\%USERNAME%

For details, see how to register an http namespace with http.sys.

Getting the Samples

You can either clone the samples repository using the command (see Using Git with CodePlex for details):

git clone https://git01.codeplex.com/aspnet

or download the samples by selecting the Download link:

The former (i.e. using git) is best if you want to stay up to date with respect to the samples.

Running the Samples

The samples can all be run by hitting F5 in Visual Studio but some may need additional information such as an application key. The first time you run a sample, it will download the various NuGet packages and ask you to confirm their installation for that project.

Note: When downloading the NuGet packages you may see a compilation warning saying something like this:

Package restore is disabled by default. To give consent, open the Visual Studio Options dialog, click on Package Manager node and check 'Allow NuGet to download missing packages during build.' You can also give consent by setting the environment variable 'EnableNuGetPackageRestore' to 'true'.

In that case do as the message says and check Allow NuGet to download missing packages during build under Package Manager in Options.

Comments and Suggestions

Please file issues on the samples as well as ideas for other samples. Let us know what you think; what you would like to see, and how this can be a useful collection of samples.

Have fun!

Henrik

HttpClient, HttpClientHandler, and WebRequestHandler Explained

Henrik F Nielsen — Tue, 07 Aug 2012 14:32:36 GMT

In two previous blogs I describe how to use HttpClient as well as how to use the HttpMessageHandler pipeline. What we haven’t done explicitly is showing when and how to use HttpClient, HttpClientHandler, and WebRequestHandler. This is the purpose of this blog.

The first thing to remember is that HttpClient uses the HttpMessageHandler pipeline for sending and receiving requests. The default handler implementations (HttpClientHandler and WebRequestHandler) both sit on top of HttpWebRequest which has a lot of properties for controlling how to deal with requests and responses. However, in order to stay simple, HttpClient doesn’t expose all these properties up front – it would make it unwieldy and show a lot of things that are rarely used.

If you need to access these properties then there are two ways of doing it: use either HttpClientHandler or WebRequestHandler. Both are HttpMessageHandlers hooked in as part of the HttpMessageHandler pipeline. Below we will describe when and how to use these handlers, how they differ, as well as how to combine them with your own handlers.

You can find this code for this sample in the aspnet.codeplex.com code repository and there are instructions here for how to use the samples.

HttpClient

The default HttpClient is the simplest way in which you can start sending requests. A single HttpClient can be used to send as many HTTP requests as you want concurrently so in many scenarios you can just create one HttpClient and then use that for all your requests.

HttpClient has a few properties that are commonly used and which you can set up until the point where you start sending requests, namely:

The properties exposed by HttpClient directly
Name	Description
BaseAddress	Gets or sets the base address of Uniform Resource Identifier (URI) of the Internet resource used when sending requests.
DefaultRequestHeaders	Gets the headers which should be sent with each request.
MaxResponseContentBufferSize	Gets or sets the maximum number of bytes to buffer when reading the response content.
Timeout	Gets or sets the number of milliseconds to wait before the request times out.

Here’s an example of how to create a simple HttpClient and issue a request while setting the timeout for all requests:

        1: // Create an HttpClient and set the timeout for requests

       2: HttpClient client = new HttpClient();

       3: client.Timeout = TimeSpan.FromSeconds(10);

       4:  

       5: // Issue a request

       6: client.GetAsync(_address).ContinueWith(

       7:     getTask =>

       8:     {

       9:         if (getTask.IsCanceled)

      10:         {

      11:             Console.WriteLine("Request was canceled");

      12:         }

      13:         else if (getTask.IsFaulted)

      14:         {

      15:             Console.WriteLine("Request failed: {0}", getTask.Exception);

      16:         }

      17:         else

      18:         {

      19:             HttpResponseMessage response = getTask.Result;

      20:             Console.WriteLine("Request completed with status code {0}", response.StatusCode);

      21:         }

      22:     });

If you have additional handlers which you want to wire up as well then the simplest way is to use the HttpClientFactory class. Here’s an example wiring up 3 custom message handlers:

       1: // Create an HttpClient and add message handlers for the client

       2: HttpClient client = HttpClientFactory.Create(

       3:     new SampleHandler("Client A", 2),

       4:     new SampleHandler("Client B", 4),

       5:     new SampleHandler("Client C", 6));

HttpClientHandler

HttpClientHandler is an HttpMessageHandler with a common set of properties that works across most versions of the HttpWebRequest API. This is the default handler and so is what you get if you use the default constructor. However, in order to actually get access to the various properties (see below) you have to explicitly create it and then pass it to HttpClient.

List of Properties for HttpClientHandler
Name	Description
AllowAutoRedirect	Gets or sets a value that indicates whether the handler should follow redirection responses.
AutomaticDecompression	Gets or sets the type of decompression method used by the handler for automatic decompression of the HTTP content response.
ClientCertificateOptions	Gets or sets the collection of security certificates that are associated with this handler.
CookieContainer	Gets or sets the cookie container used to store server cookies by the handler.
Credentials	Gets or sets authentication information used by this handler.
MaxAutomaticRedirections	Gets or sets the maximum number of redirects that the handler follows.
MaxRequestContentBufferSize	Gets or sets the maximum request content buffer size used by the handler.
PreAuthenticate	Gets or sets a value that indicates whether the handler sends an Authorization header with the request.
Proxy	Gets or sets proxy information used by the handler.
SupportsAutomaticDecompression	Gets a value that indicates whether the handler supports automatic response content decompression.
SupportsProxy	Gets a value that indicates whether the handler supports proxy settings.
SupportsRedirectConfiguration	Gets a value that indicates whether the handler supports configuration settings for the AllowAutoRedirect and MaxAutomaticRedirections properties.
UseCookies	Gets or sets a value that indicates whether the handler uses the CookieContainer property to store server cookies and uses these cookies when sending requests.
UseDefaultCredentials	Gets or sets a value that controls whether default credentials are sent with requests by the handler.
UseProxy	Gets or sets a value that indicates whether the handler uses a proxy for requests.

Here’s an example of how to create an HttpClientHandler, set a property, and pass it to HttpClient:

       1: // Create HttpClientHandler and set UseDefaultCredentials property

       2: HttpClientHandler clientHandler = new HttpClientHandler();

       3: clientHandler.UseDefaultCredentials = true;

       4:  

       5: // Create an HttpClient using the HttpClientHandler

       6: HttpClient client = new HttpClient(clientHandler);

Like above, you can combine your HttpClientHandler instance with other custom handlers, here’s what it looks like:

       1: // Create HttpClientHandler and set UseDefaultCredentials property

       2: HttpClientHandler clientHandler = new HttpClientHandler();

       3: clientHandler.UseDefaultCredentials = true;

       4:  

       5: // Create an HttpClient and add message handlers for the client

       6: HttpClient client = HttpClientFactory.Create(

       7:     clientHandler,

       8:     new SampleHandler("Client A", 2),

       9:     new SampleHandler("Client B", 4),

      10:     new SampleHandler("Client C", 6));

WebRequestHandler

WebRequestHandler derives from HttpClientHandler but adds properties that generally only are available on full .NET. The WebRequestHandler is not included in the System.Net.Http DLL but rather in System.Net.Http.WebRequest DLL so you have to explicitly include that as a reference in order to see it. Otherwise it won’t show up.

List of Properties for WebRequestHandler in addition to those of HttpClientHandler
Name	Description
AllowPipelining	Gets or sets a value that indicates whether to pipeline the request to the Internet resource.
AuthenticationLevel	Gets or sets a value indicating the level of authentication and impersonation used for this request.
CachePolicy	Gets or sets the cache policy for this request.
ClientCertificates	Gets or sets the collection of security certificates that are associated with this request.
ContinueTimeout	Gets or sets the amount of time, in milliseconds, the application will wait for 100-continue from the server before uploading data.
ImpersonationLevel	Gets or sets the impersonation level for the current request.
MaxResponseHeadersLength	Gets or sets the maximum allowed length of the response headers.
ReadWriteTimeout	Gets or sets a time-out in milliseconds when writing a request to or reading a response from a server.
ServerCertificateValidationCallback	Gets or sets a callback for verifying the remote Secure Sockets Layer (SSL) certificate used for authentication.
UnsafeAuthenticatedConnectionSharing	Gets or sets a value that indicates whether to allow high-speed NTLM-authenticated connection sharing.

Here’s an example of how to create an WebRequestHandler, set a property, and pass it to HttpClient:

       1: // Create WebRequestHandler and set UseDefaultCredentials and AllowPipelining (for HTTP) properties

       2: WebRequestHandler webRequestHandler = new WebRequestHandler();

       3: webRequestHandler.UseDefaultCredentials = true;

       4: webRequestHandler.AllowPipelining = true;

       5:  

       6: // Create an HttpClient using the WebRequestHandler

       7: HttpClient client = new HttpClient(webRequestHandler);

Again, you can combine the WebRequestHandler with your own handlers as well – here’s what it looks like:

       1: // Create WebRequestHandler and set UseDefaultCredentials and AllowPipelining (for HTTP) properties

       2: WebRequestHandler webRequestHandler = new WebRequestHandler();

       3: webRequestHandler.UseDefaultCredentials = true;

       4: webRequestHandler.AllowPipelining = true;

       5:  

       6: // Create an HttpClient and add message handlers for the client

       7: HttpClient client = HttpClientFactory.Create(

       8:     webRequestHandler,

       9:     new SampleHandler("Client A", 2),

      10:     new SampleHandler("Client B", 4),

      11:     new SampleHandler("Client C", 6));

Have fun!

Henrik

Performance Implications of Bundling and Minification on Web Browsing

Henrik F Nielsen — Sat, 16 Jun 2012 23:18:34 GMT

In this blog we analyze the impact of bundling, minification, compression, and pipelining on HTTP performance and in particular the impact on traditional browser scenarios. First we look at the impact of bundling and minification on the content in terms of size. Then we look at the impact of bundling and minification on download and page layout times in a browser-style application.

The purpose is to get a handle on what kind of performance gains can be achieved with a modern HTTP/1.1 implementation when combined with efficient organization of content. The results presented show that only when content and protocol work together is it possible to get significantly faster performance. Just looking at the protocol alone is not sufficient.

The data analyzed shows that bundling and minification can result in significant savings in terms of bytes exchanged and requests executed and hence lead to faster page render times. Further, by adding compression and pipelining it is possible to get significant speedup in the time it takes to download the HTML, CSS and JS for laying out a page, especially by putting the links to the bundled and minified CSS and JS at the very top of the HTML.

In the test evaluated, the speedup was from 638 ms (uncompressed, unbundled, unminified, and not pipelined) down to 146 ms for the equivalent compressed, bundled, minified, and pipelined content. However, by just looking at the data necessary to lay out the page (HTML, CSS, and JS but not images), the time went from 631 ms to 126 ms with the images being finalized within the remaining timespan from 126 to 146 ms.

Thanks to Howard Dierking for help getting the test data and for computing the gains of bundling and minification!

Bundling and Minification Impact on Content Size

Bundling and minification are two popular (and relatively new) mechanisms for optimizing content used in web pages (see for example these guidelines on saving round trip times (RTT)). Traditionally they focus on optimizing CSS and JS resources but there is no inherent limitation in the general model that prevents it from being used on other content. In this experiment we only looked at saving with respect to CSS and JS but future work could include looking at other content as well.

The general idea behind bundling is to concatenate multiple files of the same type into a single file hence getting a single large resource rather than a number of small resources. Minification takes bundling one step further by adding model-based compression to the content. In addition to removing white space, minification typically shortens variable names and other identifiers and removes pieces that are not used.

Bundled and minified content is a resource like any other resource with a URI so no modification of the client side is needed to accommodate bundling or minification. Further, these resources are fully cachable and can be compressed like any other content.

The bundling and minification we use in this experiment is shipping with ASP.NET 4.5. However, there is nothing particular about that implementation compared to other alternatives and so we think it is possible to get similar benefits from other implementations.

How about GZip-style Compression?

Compression algorithms like GZip and Deflate are commonly used in HTTP and so we have compared compression with bundling and minification. Note that the two are not mutually exclusive; in fact our data shows that compression works better on a bundled document due to the larger size and hence the opportunity for better compression ratio. For example, for one of the web pages tested with 7 JS and 6 CSS resources, the results of compressing the individual resources vs. compressing the minified bundles were as follows:

Compressing 7 JS individually	161178 bytes
Compressing the minified JS bundle	137817 bytes

Similarly for CSS:

Compressing 6 CSS individually	77495 bytes
Compressing the minified CSS bundle	69803 bytes

While not something we have investigated, one can imagine that minification also reduces the time it takes to parse the content as there is less data that has to be parsed.

In addition to the bytes saved in the content, it also saves HTTP overhead in going from 7 to 1 request for JS and 6 to 1 for CSS. This, as we shall see, makes other significant savings possible due to network implications of being able to retrieve the entire bundle in a single request.

Bundling and Minification Content Size Results

We first looked at data from the following web site home pages to get the difference in size by bundling and minifying the CSS and JS files referenced from those pages. We only looked at CSS and JS coming from the same DNS domain (i.e. for digg.com we looked at anything under *.digg.com. The result is listed below where the last three columns illustrate the percentage saved in terms of bytes and the requests saved by bundling all the CSS into one file and all the JS into another.

The data shows significant savings in terms of size of CSS and JS as well as number of requests executed. This bundling algorithm may not work in all cases but it should be possible to limit the number of bundles to less than a handful and typically it would be one or two.

**Analysis of content from various web sites around June 11 2012.**
url	# js	# css	# imgs	js size (kb)	css size (kb)	image size (kb)	css bundle size (kb)	css minified size (kb)	js bundle size (kb)	js minified size (kb)	%css savings	%js savings	Requests saved
digg.com	7	2	17	512	161	170	162	133	514	235	21.80%	118.72%	7
bbc.co.uk	8	5	31	165	168	492	169	134	166	146	26.12%	13.70%	11
bloomberg.com	7	4	29	408	517	273	519	419	410	383	23.87%	7.05%	9
huffingtonpost.com	7	6	24	568	636	138	638	510	570	456	25.10%	25.00%	11
decor8.com	11	6	69	269	32.	12.9	33	26	270	204	26.92%	32.35%	15
cricinfo.com	10	1	14	568	464	73	464	370	571	508	25.41%	12.40%	9
economictimes.com	11	1	53	369	94	274	94	76	372	322	23.68%	15.53%	10

Bundling and Minification Impact on Download and Page Layout Time

In addition to comparing the bundled and minified content with the original content we ran a set of download experiments to evaluate the impact on the HTTP performance and rendering time within a browser. For this experiment we only looked at huffingtonpost.com which we re-created as a local copy so that we could work with the data without it changing under us as the web site gets updated.

Page Layout Performance

A critical part of being able to measure performance of a web page is the time it takes to layout the page which may be shorter than the time it takes to download the entire content. With modern uses of HTML, there are typically three parts that needs to be downloaded in order for the content to be rendered:

The HTML itself
CSS resources determining the layout
JS resources containing scripts to be executed on the page

Other content such as images are typically not critical for the layout to be completed and can be filled out as data is being downloaded. To get a measure for how quickly a page may be rendered we measured the download time for each of four groups of content: HTML, CSS, JS, and images. We could then determine when the entire content was downloaded vs. when the HTML, CSS, and JS (but not images) were downloaded.

How About Pipelining?

In the download data presented below we look at both pipelined GET requests and non-pipelined requests (but with persistent TCP connections). In the setup we used (100MBit wireless Ethernet) and even though the RTTs in this environment are very small we did notice a difference between pipelined and non-pipelined requests. However, in any networking environment with larger RTTs being able to submit requests without waiting for a response becomes a necessity. As has been most recently documented in Mark Nottingham’s IETF draft "Making HTTP Pipelining Usable on the Open Web”, there are known problems with practical deployment if pipelining in open network environments. However, it is clear that in order to achieve the best possible performance where RTTs is a factor, it is required to eliminate RTTs aggressively and pipelining can help doing that.

Bundling and Minification Download Results

The first table shows the total download time for each of the four content categories starting from the point where the first resource (i.e. the HTML page) is requested. That is, the values indicate the times it took to complete the download of all the HTML, CSS, JS, and images respectively counting from the beginning of the very first request asking for the HTML. This means that not all the time is actually used downloading as the CSS, JS, and images only can be initiated once the HTML is being downloaded and links are identified. The times do not include DNS lookups as we used literal IP addresses to connect to the server.

**Total download completing times in ms of various types of content in different configurations.**
Content category	Plain, no pipeline (4 TCP)	GZip’ed, no pipeline (4 TCP)	GZip’ed and pipelined (2 TCP)	GZip’ed, bundled and minified but no pipeline (4 TCP)	GZip’ed, pipelined, bundled, and minified (2 TCP)
HTML	264	49	44	45	48
CSS	622	162	134	97	104
JS	631	175	146	135	126
Images	638	179	150	162	146

The first thing that is apparent is that the download time shrinks from left to right in the table with plain being the slowest with a total download time of 638 ms and GZip compressed, pipelined, bundled, and minified being the fastest with a total download time of 146 ms for the same web page. However, there is more to the data than just the total download time. In the section on page layout we described how getting the HTML, CSS, and JS was critical to laying out the page.

To better being able to interpret the data we represent it in a set of graphs where the Y axis is total download time in ms for the content of a particular category. Looking at the plain data column, it is apparent that the HTML is done after 264 ms but neither the CSS nor the JS is anywhere near done. In fact, the last parts of the CSS, JS, and images all end roughly at the same time (622, 631, and 638 ms respectively).

Below are similar graphs for the other four columns above but we have adjusted the scale of the Y axis to more easily see the differences in detail. Looking at the compressed data, it is clear that even though the download is much faster than plain data, downloading the CSS and JS still takes almost as long as downloading the images (162, 175, and 179 ms respectively). That is, compression does not reorganize the order in which links are referenced within the HTML and hence it is still possible to see a significant delay before the page can be rendered.

Turning on HTTP/1.1 pipelining further lowers the total download time, but just like compression, does not alter the location of the links in the HTML, leading to similar data as for compression. That is, the JS is not completely downloaded until the images also have been downloaded (146 vs. 150 ms). The important thing to note here is that without deliberate organization of the content, there is little the protocol can do to optimize the performance. Only when content and protocol work together is it possible to get significantly faster performance.

With bundling and minification, the picture starts to look different for two reasons: First, the links to the CSS and JS have been put all the way at the top of the HTML so that it the client parser gets to the links very fast and can submit them to the server. In the original content, the first CSS link was found at byte offset 2213 within the HTML and the first JS link was found at byte offset 3730. By moving these links to the very beginning they could be initiated earlier.

Second, with the fewer requests there is also less protocol overhead and less interruption of parsing the data. The result is that the CSS and JS are completely downloaded after 135 ms, allowing the page to be rendered while the remaining images are still being loaded.

Lastly, we tried the full suite of compressed, pipelined, bundled, and minified content. Even with the low RTT we did observe a difference in lower times to download the content and it produced the fastest time of 146 for downloading the images and only 126 to download the JS and hence being able to layout the page,

Summary

Bundling and minification can result in significant savings in terms of bytes to exchange and requests necessary to execute and hence lead to faster page render times. In addition, by applying compression and pipelining to bundling and minification it is possible to get significant speedup in the time it takes to download the HTML, CSS and JS for laying out a page, especially by putting the links to the bundled and minified CSS and JS at the very top of the HTML. In short, only when content and protocol work together is it possible to get significantly faster performance. Just looking at the protocol alone is not sufficient.

In the test evaluated, the speedup was from 638 ms in plain mode down to 146 ms for the equivalent compressed, pipelined, bundled and minified content. However, by just looking at the data necessary to lay out the page (HTML, CSS, and JS but not images), the time went from 631 ms to 126 ms with the images being finalized within the remaining timespan from 126 to 146 ms.

Practical Advice for Page Designers

Here’s a short list of the possible optimizations that we observed:

Bundle and minify CSS and JS resources. This saves both rendering time and reduces bandwidth consumption.
Add references to the CSS and JS bundles as close to the top of the HTML page as possible. Many HTML pages have a huge header containing several kByte of metadata before links referencing CSS and JS which takes time to parse through and costs valuable time before subsequent HTTP requests can be initiated.
Ensure that both the HTML and the CSS and JavaScript bundles use compression if supported by the client. It will further reduce the bandwidth and speed up getting the data to the client.

These parts will in combination with an efficient browser implementation leveraging compression and pipelining provide a much faster rendering time.

ASP.NET MVC, Web API, and Web Pages RC NuGet Packages

Henrik F Nielsen — Fri, 01 Jun 2012 18:52:08 GMT

The official ASP.NET Web Stack RC NuGet packages are now available on NuGet. If you are new to NuGet then it is a Visual Studio extension that makes it easy to add, remove, and update libraries and tools in Visual Studio projects (check out the NuGet overview).

NuGet Packages

Microsoft ASP.NET Web API (RC) -- this package contains everything you need to host ASP.NET Web API on IIS.
Microsoft ASP.NET Web API Self Host (RC) -- this package contains everything you need to host ASP.NET Web API within your own process (outside of IIS).
Microsoft ASP.NET Web API Client Libraries (RC) -- this package adds support for formatting and content negotiation to System.Net.Http. It includes support for JSON, XML, and form URL encoded data.
Microsoft ASP.NET MVC 4 (RC) -- this package contains the runtime assemblies for ASP.NET MVC.
Microsoft ASP.NET Razor 2 (RC) -- this package contains the runtime assemblies for ASP.NET Web Pages.

If you are interested in using the latest nightly builds instead then please check out this description for how to get set up.

NuGet Package Tags

When searching for a package on NuGet it is convenient to use tags for your query. Here are some convenient tags for the ASP.NET Web Stack packages:

Additional Information

Release Notes for RC
Useful links with blogs, additional information, etc.

Have fun!

Henrik

del.icio.us Tags: asp.net,webapi,mvc,aspnetwebapi

Using Nightly ASP.NET Web Stack NuGet Packages with Visual Studio 2012

Henrik F Nielsen — Fri, 01 Jun 2012 06:28:31 GMT

Note: this blog uses Visual Studio 2012 but the instructions for VS 2010 are the same.

This blog describes using the nightly NuGet packages with Visual Studio 2012. Note well that the packages come with no guarantees and they do not substitute for the official installers which contain the official MSI and NuGet packages. And of course, don’t use your dev machine for anything important.

Prerequisites: Visual Studio 2012 and GacUtil

In contrast to compiling the code yourself, it is possible to use the nightly NuGets with Visual Studio 2012 installed so that you can use the project templates.

As Web API is not in the GAC it is very simple to update – just get the nightly NuGets and you are done. For MVC and Web Pages you still need to work with the Global Assembly Cache (GAC) so you do need the GacUtil tool for updating assemblies in the GAC.

Configuring NuGet

In order to see the nightly packages you first need to configure Visual Studio to point to the MyGet feed at

http://www.myget.org/F/aspnetwebstacknightly/

Under the Visual Studio Tools menu select Options and then Package Manager and add the feed – it should look like this:

Updating an ASP.NET MVC4 Web Application Project

Assume we have an ASP.NET MVC4 Web Application Visual Studio project (below we created a Web API but it should work for any of the project variants). Under the Visual Studio Project menu, select Manage NuGet Packages, then select Updates from the MyGet feed you just added, and then select Include Prerelease:

Install the updates and close the window.

Updating the GAC

For Web API you are all set – nothing else is needed! However, if you are using MVC or Web Pages then before you can run the project you have to update the GAC with the new assemblies which NuGet downloaded and stuck in the packages folder which you can find in your solutions folder. You can get a view of the assembles by typing the following in the packages folder

dir /s /b *.dll

It should look something like this:

The list below is the set of DLLs that are GAC’ed in VS 2012. Look for the assemblies on the list below and match them against the directory list above. Depending on which project variant you picked you may not have all of them – that’s ok.

System.Web.Mvc
System.Web.Razor
System.Web.Helpers
System.Web.WebPages.Deployment
System.Web.WebPages
System.Web.WebPages.Razor

For each of the assemblies that match the GAC’ed list install them in the GAC using GacUtil (running as Administrator) like this:

GacUtil.exe -i <path to assembly in your packages folder>

For example,

Trying It Out

You are now ready to run your application. When running in the Visual Studio debugger you can verify that you use the new assemblies by looking at the Modules window available under the Visual Studio Debug menu. It should look something like this:

The version numbers should correspond to the numbers used to name the folders under the packages folder we looked at above.

Have fun!

Henrik

del.icio.us Tags: Asp.net,webapi,mvc,codeplex

ASP.NET Web API Updates – May 14

Henrik F Nielsen — Mon, 14 May 2012 00:30:35 GMT

We have been on a roll with respect to changes over the last few weeks – here’s some of the highlights. If you want to see the full list then you can either track the commit page or follow the project which will then show you the updates on your codeplex home page. As in the previous update this is a sneak-peak of code checked into our CodePlex source code repository but not yet released as part of our official installer. See Using Nightly ASP.NET Web Stack NuGet Packages for how to try out these features.

Per Controller Type Configuration

ASP.NET Web API has a HttpConfiguration object that provides configuration for features including routing, dependency resolution, media type formatters, and message handlers. However, while the HttpConfiguration allows you to configure most aspects of ASP.NET Web API, it does not allow different configurations per controller type. This feature changes that in that we have added per-controller-type configuration. In essence, a controller type can have its own “shadow copy” of the global HttpConfiguration object but override specific settings that only applies to that controller type. This is automatically applied to all controller instances of the given controller-type.

Mike Stall has a great blog about the feature in detail including examples of how to use it with an ApiController so check it out – this is an exciting feature!

Content Negotiated Error Messages

We have unified the generation of error messages regardless of whether they are generated as part of a normal message flow, as part of an exception path, or as part of model state validation. This means that regardless of how you generate a 4xx or 5xx status code, the result will be a uniform, content negotiated message, making it easier to handle by clients.

For example, you can generate a 409 Conflict response like this:

        1: public Contact Get()

       2: {

       3:     HttpResponseMessage response = Request.CreateErrorResponse(

       4:         HttpStatusCode.Conflict, "Check out this message!");

       5:     throw new HttpResponseException(response);

       6: }

or like this:

       1: public HttpResponseMessage Get(HttpRequestMessage request)

       2: {

       3:     HttpResponseMessage response = request.CreateErrorResponse(

       4:         HttpStatusCode.Conflict, "Check out this message!");

       5:     return response;

       6: }

In either case, the response will look like this in Fiddler if you ask for “text/json”:

Note: By default the HttpConfiguration.IncludeErrorDetailPolicy setting only exposes a generic message to external users so that we inadvertently leak internal information.

Content Negotiation Improvements

Lastly, we re-organized the content negotiation code to all be gathered in our DefaultContentNegotiator class. The content negotiation is fully pluggable so that if you want to roll your own then you can implement the IContentNegotiator interface and hook it in through the configuration. However, now you can also just derive from DefaultContentNegotiator and pretty much override any aspect that you like instead of starting from scratch. For a full description, please see this commit.

In addition we added support for not picking accept header values with a quality factor of 0.0. Before we treated this as a small but possible match while now we completely filter them out of the picture. That is, if you send something like “Accept: text/xml; q=0.0” then you won’t get it.

We also added a hook so that it is possible to create an instance of the DefaultContentNegotiator that returns null if none of the accept header values were matched. This makes it possible to inject your own DefaultContentNegotiator instance (in HttpConfiguration.Services) that is suited for generating 406 responses if no match on accept headers were found.

This sample shows a simple self host which hooks in a special DefaultContentNegotiator instance set to return null if not matching on any Accept header field in the request:

       1: string baseAddress = "http://localhost:8080/";

       2: HttpSelfHostConfiguration config = new HttpSelfHostConfiguration(baseAddress);

       3:  

       4: DefaultContentNegotiator negotiator = new DefaultContentNegotiator(excludeMatchOnTypeOnly: true);

       5: config.Services.Replace(typeof(IContentNegotiator), negotiator);

       6:  

       7: // Add a route

       8: config.Routes.MapHttpRoute(

       9:   name: "defaultAction",

      10:   routeTemplate: "api/{controller}/{action}/{id}",

      11:   defaults: new { controller = "Home", id = RouteParameter.Optional });

      12:  

      13: config.Routes.MapHttpRoute(

      14:   name: "default",

      15:   routeTemplate: "api/{controller}/{id}",

      16:   defaults: new { controller = "Home", id = RouteParameter.Optional });

      17:  

      18: HttpSelfHostServer server = new HttpSelfHostServer(config);

      19: server.OpenAsync().Wait();

      20:  

      21: Console.WriteLine("Listening on " + baseAddress);

We should now see a 406 Not Acceptable response if there is no matching Accept header field so let’s try it out in Fiddler with no Accept header at all:

And indeed there is a 406 response. If we add an Accept header then we get a 200 response like this:

Have fun!

Henrik

del.icio.us Tags: asp.net,webapi,httpclient

ASP.NET Web API Updates – May 3

Henrik F Nielsen — Thu, 03 May 2012 22:05:32 GMT

Some interesting updates were checked in over the last week. As in the previous update this is a sneak-peak of code checked into our CodePlex source code repository but hasn’t yet been released as part of our official installer. See Using Nightly ASP.NET Web Stack NuGet Packages for how to try out these features.

Per-Route Message Handlers

Support for per-route message handlers has been a popular request on our issues list. The issue is that HttpMessageHandlers is a flexible extensibility point for dealing with HTTP requests and responses but they are configured once per application within the HttpConfiguration. In some scenarios you have different needs for different services so a more flexible registration is needed.

Brad Wilson describes the per-route message handler feature thusly: The implementation of this feature allows users to specify the "final" message handler in the request chain for a given route. This will enable two scenarios:

Support for adding message handlers on a per-route basis.
Allow “ride-along” frameworks to use routing to dispatch to their own (non-IHttpController) endpoints.

To illustrate the change in the pipeline, this is what the pipeline looked like before this change:

The old HttpControllerDispatcher was in charge of

Routing lookup (if not already done)
Generating a 404 when no routes match
Generate a 404 when no controllers match
Dispatch to IHttpController

In the new pipeline we have split this into two handlers like this:

The new responsibilities are as follows:

HttpRoutingDispatcher

Routing lookup (if not already done)
Generating a 404 when no routes match
Dispatch to IHttpRoute.Handler (HttpControllerDispatcher by default when null)

HttpControllerDispatcher

Generating a 404 when no controllers match
Dispatch to IHttpController

What this means in practical terms is that, if you want to provide one or more message handlers on a per-route basis, those message handlers will have to run after routing has run (so, basically last), and if you want to preserve the dispatching to controllers, your final handler must delegate to HttpControllerDispatcher. To generate a message handler pipeline you can use the HttpClientFactory as described in the previous update.

As an example of how to use this, we introduce a simple “hello world” message handler that simply responds with a static HTTP response like this:

        1: public class HelloWorldMessageHandler : HttpMessageHandler

       2: {

       3:     protected override Task<HttpResponseMessage> SendAsync(

       4:         HttpRequestMessage request,

       5:         CancellationToken cancellationToken)

       6:     {

       7:         var response = request.CreateResponse(HttpStatusCode.OK, "Hello, world!");

       8:  

       9:         var tcs = new TaskCompletionSource<HttpResponseMessage>();

      10:         tcs.SetResult(response);

      11:         return tcs.Task;

      12:     }

      13: }

We can now hook it in by adding a route with the new handler – in this case we use a self-host snippet:

       1: class Program

       2: {    

       3:     static void Main(string[] args)

       4:     {

       5:         var config = new HttpSelfHostConfiguration("http://localhost:8080/");

       6:  

       7:         IHttpRoute route = config.Routes.CreateRoute(

       8:             routeTemplate: "hello",

       9:             defaults: new HttpRouteValueDictionary("route"),

      10:             constraints: null,

      11:             dataTokens: null,

      12:             parameters: null,

      13:             handler: new HelloWorldMessageHandler());

      14:         config.Routes.Add("HelloRoute", route);

      15:         

      16:         config.Routes.MapHttpRoute(

      17:           name: "default",

      18:           routeTemplate: "api/{controller}/{id}",

      19:           defaults: new { controller = "Home", id = RouteParameter.Optional });

      20:         

      21:         var server = new HttpSelfHostServer(config);

      22:         server.OpenAsync().Wait();

      23:         Console.WriteLine("Listening on {0}...", config.BaseAddress);

      24:         Console.ReadLine();

      25:         server.CloseAsync().Wait();

      26:     }

      27: }

If you now access http://localhost:8080/hello you will get the response from HelloWorldMessageHandler and for requests under http://localhost:8080/api/* you will get the regular Web API controllers. Because message handlers can be chained you can also insert the default HttpControllerDispatcher at the end – the opportunities are endless!

Reading Form Data

In addition to doing model binding in your ApiController actions where you get the form data directly as part of your action signature, you can parse form data throughout Web API (both client side and server side). Two common ways to get form data is either as a NameValueCollection or as a JToken which is the Json.NET LINQ to JSON data model. This commit rounds out the functionality by providing equal support for reading form data from a URI or from the body of an HTTP message.

Reading as NameValueCollection

You can read the query component of a URI as a NameValueCollection doing something like this (first line creates some sample content and the second line reads it):

    Uri address = new Uri("http://example.com/path?a[]=1&a[]=5&a[]=333");
    NameValueCollection nvcUri = address.ParseQueryString();

Similarly you can read HTML form URL encoded data from an HttpContent which can be part of an HttpRequestMessage or HttpResponseMessage like this (first line creates some sample content and the second line reads it):

    StringContent content = new StringContent("a[]=1&a[]=5&a[]=333",
        Encoding.UTF8, "application/x-www-form-urlencoded");
    NameValueCollection nvcContent = content.ReadAsFormDataAsync().Result;

In both cases will you get a NameValueCollection with one key (“a”) and three values (“1”, “5”, and “333”).

Reading as JObject

You can also read the query component of a URI as a JToken which is the Json.NET LINQ to JSON data model (we use the same sample URI as above):

    JObject jObjectUri;
    address.TryReadQueryAsJson(out jObjectUri);

Similar you can read the HTML form URL encoded data from an HttpContent as JObject like this (we use the same sample content as above):

    JObject jObjectContent = content.ReadAsAsync<JObject>().Result;

In both cases will you get a JObject which you can use to traverse the data as you wish.

Accepted Pull Requests

Finally we have accepted a set of pull requests – thanks for the contributions!

Andre Azevedo: Correct unit tests by replacing culture
Patrick McDonald: Modified System.Web.Mvc.RouteCollectionExtensions.MapRoute to work with dictionary defaults and/or constraints
Patrick McDonald: Fixed RouteCollectionExtensions.MapRoute to work when defaults and constraints are dictionaries, currently method only works with anonymous types or similar

Have fun!

Henrik

del.icio.us Tags: asp.net,webapi,httpclient

Using Nightly ASP.NET Web Stack NuGet Packages

Henrik F Nielsen — Sun, 29 Apr 2012 18:25:06 GMT

Note: This blog has been superseded by this one – the instructions for Visual Studio 2012 and 2010 are the same.

We are very excited about the recent announcement of nightly NuGet packages being available on MyGet. This blog will show using these packages with Visual Studio 2010 but first you should be aware of the disclaimers. In short, these packages come with no guarantees and they do not substitute for the official installers which contain the official MSI and NuGet packages. And of course, don’t use your dev machine for anything important.

Prerequisites: Visual Studio 2010 and GacUtil

In contrast to compiling the code yourself, it is possible to use the nightly NuGets with ASP.NET MVC 4 Beta for Visual Studio 2010 installed so that you can use the project templates. The instructions here are for .NET 4.0 and Visual Studio 2010 only – it won’t work with Dev 11 Beta due to a conflict in System.Net.Http. Also, you do need to work with the Global Assembly Cache (GAC) so you do need the GacUtil tool for updating assemblies in the GAC.

Note: In ASP.NET MVC 4 Beta for Visual Studio 2010 we GAC’ed more than necessary and so when updating the NuGet packages you have to update the GAC as well as the GAC always wins when resolving assembly references. In the next official drop we GAC a lot less so much of this will go away.

Configuring NuGet

In order to see the nightly packages you first need to configure Visual Studio to point to the MyGet feed at

http://www.myget.org/F/aspnetwebstacknightly/

Under the Visual Studio Tools menu select Options and then Package Manager and add the feed – it should look like this:

Updating an ASP.NET MVC4 Web Application Project

Assume we have an ASP.NET MVC4 Web Application Visual Studio project (below we created a Web API but it should work for any of the project variants). Now, under the Visual Studio Project menu, select Manage NuGet Packages and select Updates from the MyGet feed you just added:

Install the updates and close the window.

Updating the GAC

Before you can run the project you have to update the GAC with the new assemblies which NuGet downloaded and stuck in the packages folder which you can find in your solutions folder. You can get a view of the assembles by typing the following in the packages folder

dir /s /b *.dll

It should look something like this:

The list below is the set of DLLs we GAC’ed in ASP.NET MVC 4 Beta for Visual Studio 2010. Moving forward we are GAC’ing a lot less so this will get much easier. Look for the assemblies on the list below and match them against the directory list above. Depending on which project variant you picked you may not have all of them – that’s ok.

System.Web.Mvc
System.Web.Razor
System.Net.Http.Formatting
System.Web.Http
System.Web.Http.WebHost
System.Web.Helpers
System.Web.WebPages.Deployment
System.Web.WebPages
System.Web.WebPages.Razor
System.Net.Http
System.Net.Http.WebRequest
System.Json

For each of the assemblies that match the GAC’ed list install them in the GAC using GacUtil (running as Administrator) like this:

GacUtil.exe -i <path to assembly in your packages folder>

For example,

Trying It Out

The version numbers should correspond to the numbers used to name the folders under the packages folder we looked at above.

Have fun!

Henrik

del.icio.us Tags: Asp.net,webapi,mvc,codeplex

ASP.NET Web API Updates – April 27

Henrik F Nielsen — Fri, 27 Apr 2012 04:11:22 GMT

Here’s a few more updates coming your way in ASP.NET Web API. As in the previous update the code has been checked into our CodePlex source code repository but hasn’t yet been released as part of our official installer. Think of this as a sneak-peak for you to try out and comment on as part of our source code repository (see Getting Started With ASP.NET Web Stack Source on CodePlex for details).

Helper for Wiring up Message Handlers

HttpMessageHandlers are pluggable “filters” that sit between HttpClient and the network and allow you to modify the request and/or response as they travel from a HttpClient to the network and back (see Mike Wasson’s blog on message handlers for a detailed walkthrough). HttpMessageHandlers are plugged together in a “Russian doll” model where each handler delegates to the inner handler until the last one hits the network (or a previous one short-circuited the path). The model can be illustrated like this:

This commit provides a simple HttpClientFactory that can wire up DelegatingHandlers and create an HttpClient with the desired pipeline ready to go. It also provides functionality for wiring up with alternative inner handlers (the default is HttpClientHandler) as well as do the wiring up when using HttpMessageInvoker or another DelegatingHandler instead of HttpClient as the top-invoker.

To demonstrate this we first create a handler where we override the SendAsync method to intercept the request and response path:

        1: public class MyHandler : DelegatingHandler

       2: {

       3:     private string _name;

       4:  

       5:     public MyHandler(string name)

       6:     {

       7:         _name = name;

       8:     }

       9:  

      10:     // We override the SendAsync method to intercept both the request and response path

      11:     protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)

      12:     {

      13:         Console.WriteLine("Request path in handler {0}", _name);

      14:         return base.SendAsync(request, cancellationToken).ContinueWith(

      15:             requestTask =>

      16:             {

      17:                 Console.WriteLine("Response path in handler {0}", _name);

      18:                 return requestTask.Result;

      19:             }, TaskContinuationOptions.OnlyOnRanToCompletion);

      20:     }

      21: }

Then we use this handler in a sample where we instantiate three instances and wire them together:

       1: // Create three instances of handlers

       2: MyHandler handlerA = new MyHandler("A");

       3: MyHandler handlerB = new MyHandler("B");

       4: MyHandler handlerC = new MyHandler("C");

       5:  

       6: // Create HttpClient with handlers A, B, and C wired up

       7: HttpClient client = HttpClientFactory.Create(handlerC, handlerB, handlerA);

       8:  

       9: // Issue asynchronous GET request

      10: client.GetAsync("http://msdn.microsoft.com").ContinueWith(

      11:     requestTask =>

      12:     {

      13:         if (requestTask.IsCanceled)

      14:         {

      15:             Console.WriteLine("Request was cancelled");

      16:         }

      17:  

      18:         if (requestTask.IsFaulted)

      19:         {

      20:             Console.WriteLine("Request failed with an exception: {0}", requestTask.Exception.GetBaseException());

      21:         }

      22:  

      23:         HttpResponseMessage response = requestTask.Result;

      24:         {

      25:             Console.WriteLine("Request succeeded");

      26:         }

      27:     });

When running this sample we get the following output on the console (note the order with A being first on request and last on response, etc.):

Progress Notifications

Being able to get progress notifications on data being uploaded and downloaded was a popular request on the beta bits which we didn’t support directly. This commit defines a ProgressMessageHandler which generates progress notification for both request entities being uploaded and response entities being downloaded. Using this handler it is possible to keep track of how far you are uploading a request body or downloading a response body.

The ProgressMessageHandler exposes two events, one for tracking upload and download progress respectively. Each progress notification is associated with a specific request so it is possible to track progress notifications from multiple ongoing requests at the same time. A sample receive progress event handler looks like this:

       1: private static void HttpReceiveProgress(object sender, HttpProgressEventArgs e)

       2: {

       3:     // The sender is the originating HTTP request

       4:     HttpRequestMessage request = sender as HttpRequestMessage;

       5:  

       6:     // Write different message depending on whether we have a total length or not

       7:     string message;

       8:     if (e.TotalBytes != null)

       9:     {

      10:         message = String.Format("Request {0} downloaded {1} of {2} bytes ({3}%)", request.RequestUri, e.BytesTransferred, e.TotalBytes, e.ProgressPercentage);

      11:     }

      12:     else

      13:     {

      14:         message = String.Format("Request {0} downloaded {1} bytes", request.RequestUri, e.BytesTransferred, e.TotalBytes, e.ProgressPercentage);

      15:     }

      16:  

      17:     // Write progress message to console

      18:     Console.WriteLine(message);

      19: }

The send progress event handler looks exactly the same – it also receives a HttpProgressEventArgs instance with the exact same shape. Below is a sample which uses the HttpClientFactory to create an HttpClient with progress notifications hooked in:

       1: // Create a progress notification handler and wire up events for upload and download progress

       2: ProgressMessageHandler progress = new ProgressMessageHandler();

       3: progress.HttpReceiveProgress += new EventHandler<HttpProgressEventArgs>(HttpReceiveProgress);

       4: progress.HttpSendProgress += new EventHandler<HttpProgressEventArgs>(HttpSendProgress);

       5:  

       6: // Create a client and wire up the progress handler

       7: HttpClient client = HttpClientFactory.Create(progress);

       8:  

       9: // We can now submit an HTTP request and see the progress notifications in action

      10: client.GetAsync("http://msdn.microsoft.com").ContinueWith(

      11:     (requestTask) =>

      12:     {

      13:         if (requestTask.IsCanceled)

      14:         {

      15:             Console.WriteLine("Request was cancelled");

      16:         }

      17:  

      18:         if (requestTask.IsFaulted)

      19:         {

      20:             Console.WriteLine("Request failed with an exception: {0}", requestTask.Exception.GetBaseException());

      21:         }

      22:  

      23:         HttpResponseMessage response = requestTask.Result;

      24:         {

      25:             Console.WriteLine("Request succeeded");

      26:         }

      27:     });

A sample output written to the console looks like this:

Multipart Improvements

Support for reading MIME multipart messages as handled by the HttpContent.ReadAsMultipartAsync extension methods had two usability problems in the original design:

The result was IEnumerable<HttpContent> which is too low-level to facilitate easy consumption. For example, it doesn't allow for converting multipart html form data into NameValueCollection data or for any other higher-level abstraction on top of the raw IEnumerable<HttpContent>.
The stream provider doesn't enable any post processing of the data such as converting the read data into another format before the read task completed.

This made it hard to provide a higher-level abstraction on top of IEnumerable<HttpContent> as the MultipartStreamProvider does not know when the message has been completely read.

This commit fixes these short comings by first changing the signature of the ReadAsMultipartAsync to

public static Task<T> ReadAsMultipartAsync<T>(this HttpContent content, T streamProvider) where T : MultipartStreamProvider

This means that it is now possible to write a MultipartStreamProvider that is completely tailored to the type of MIME multipart that it can read and present the result in the optimal way to the user.

Second, we introduce a “post processing” step as part of the MultipartStreamProvider that allows the implementation to do whatever post processing it wants on the MIME multipart body parts. For example, the MultipartFormDataStreamProvider implementation reads the HTML form data parts and adds them to a NameValueCollection so they are easy to get at from the caller.

The built-in set of MultipartStreamProvider consists of:

MultipartMemoryStreamProvider saves all MIME body parts in memory allowing them to be read, deserialized, etc.
MultipartFileStreamProvider saves all MIME body parts as individual files.
MultipartFormDataStreamProvider saves all form data parts in a NameValueCollection and all file parts to file. This allows the form data to be easily manipulated and files to be persisted.
MultipartRelatedStreamProvider parses multipart related entities as defined by RFC 2387 “The MIME Multipart/Related Content-type”.

An example of how to implement an Action that handles file uploads with parameters using MultipartFormDataStreamProvider looks like this:

       1: [HttpPost]

       2: public Task<List<FileResult>> UploadFile()

       3: {

       4:     // Verify that this is an HTML Form file upload request

       5:     if (!Request.Content.IsMimeMultipartContent("form-data"))

       6:     {

       7:         throw new HttpResponseException(Request.CreateResponse(HttpStatusCode.UnsupportedMediaType));

       8:     }

       9:  

    // Create a stream provider for setting up output streams

    MultipartFormDataStreamProvider streamProvider = new MultipartFormDataStreamProvider("d:\\src\\tmp\\uploads");

  

    // Read the MIME multipart asynchronously content using the stream provider we just created.

    return Request.Content.ReadAsMultipartAsync(streamProvider).ContinueWith(

        readTask =>

        {

            MultipartFormDataStreamProvider provider = readTask.Result;

  

            // Create response containing information about the stored files.

            return provider.FileData.Select(fileData =>

            {

                FileInfo fileInfo = new FileInfo(fileData.LocalFileName);

                ContentDispositionHeaderValue disposition = fileData.Headers.ContentDisposition;

                string filename = (disposition != null && disposition.FileName != null) ? disposition.FileName : String.Empty;

  

                return new FileResult

                {

                    FileName = filename,

                    LocalPath = fileInfo.FullName,

                    LastModifiedTime = fileInfo.LastWriteTimeUtc,

                    Length = fileInfo.Length,

                    Submitter = provider.FormData["submitter"]

                };

            }).ToList();

        });

}

As always please use the CodePlex discussion forum and go vote on your favorite issue in Issue Tracker – we would love to hear you feedback.

Have fun!

Henrik

del.icio.us Tags: asp.net,webapi,httpclient,http

Recent ASP.NET Web API Updates – April 24

Henrik F Nielsen — Mon, 23 Apr 2012 19:58:10 GMT

Here’s a few updates coming your way in ASP.NET Web API. The code has been checked into our CodePlex source code repository but hasn’t yet been released as part of our official installer. Think of this as a sneak-peak for you to try out and comment on as part of our source code repository (see Getting Started With ASP.NET Web Stack Source on CodePlex for details).

Cookie Support

Until now the HTTP Cookie and Set-Cookie headers were only exposed as raw strings and not structured classes in the HttpRequestMessage and HttpResponseMessage classes. This made it cumbersome and error prone to work with cookies in ASP.NET Web API. To fix this we introduced two new classes called CookieHeaderValue and CookieState that follow the RFC 6265 HTTP State Management Mechanism. Each Cookie or Set-Cookie header is represented as one CookieHeaderValue. A CookieHeaderValue contains information about the domain, path, expiration, and other common cookie information as well as one or more CookieStates. Each CookieState contains a name and whatever state is associate with that name. This representation allows for multiple related CookieStates to be carried within the same Cookie header while still providing separation between each cookie state.

To see how this works, look at the sample cookie exchange where a server sends a cookie to the client using the Set-Cookie header including the cookie state as well as domain and expiration information:

Set-Cookie: StateA=n1=v1&n2=v2; StateZ=n3=v3&n4=v4; Domain=domain1; Path=path1; Expires=Mon, 23 Apr 2012 08:49:37 GMT

The client then returns the cookie on a subsequent request passing just the cookie state:

Cookie: StateA=n1=v1&n2=v2; StateZ=n3=v3&n4=v4

In this example, there is one CookieHeaderValue with two CookieStates in the Set-Cookie response and also in the subsequent Cookie request. The CookieStates have names StateA and StateZ respectively. The CookieState names are followed by a value consisting of name-value pairs encoded as URL-encoded form data (also known as application/x-www-form-urlencoded) so that you can pass relatively structured data back and forth. In the above example, StateA has two name/value pairs (n1=v1 and n2=v2).

Getting all the CookieHeaderValues from a request:

var cookies = Request.Headers.GetCookies();

Getting a CookieHeaderValue with a particular CookieState name:

var cookie = Request.Headers.GetCookies(“StateA”);

Adding one or more Set-Cookie headers to a response

Response.Headers.AddCookies(cookies);

You can access the CookieState using C# indexers. For example, the following would return “v2”:

var value = cookie[“StateA”][“n2”];

Push Content

The HttpContent model supports writing data by either pushing data to the output stream directly or pulling data from an input stream and then copying it to the output stream (see Push and Pull Streams using HttpClient). Pull is supported out of the box in the form of StreamContent but until there was no equivalent built-in support for pushing data.

The PushStreamContent class enables scenarios where a data producer wants to write directly (either synchronously or asynchronously) using a stream. When the PushStreamContent is ready to accept data it calls out to an action with the output stream. The developer can then write to the stream for as long as necessary and close the stream when writing has completed. The PushStreamContent detects the closing of the stream and completes the underlying asynchronous Task for writing out the content.

PushStreamContent is entirely asynchronous and does not block any threads. It is up to the user of the PushStreamContent to write data either synchronously or asynchronously which of course then will determine whether a thread is blocked or not. For more details, see this commit. Here’s a sample push controller which writes out the plaintext string “Here’s some more data” every second. On the wire this will be sent using HTTP Chunked Encoding where each string will be an individual chunk.

        1: public class PushController : ApiController

       2: {

       3:     private static readonly Lazy<Timer> _timer = new Lazy<Timer>(() => new Timer(TimerCallback, null, 0, 1000));

       4:     private static readonly ConcurrentDictionary<StreamWriter, StreamWriter> _outputs = new ConcurrentDictionary<StreamWriter, StreamWriter>();

       5:  

       6:     public HttpResponseMessage GetUpdates(HttpRequestMessage request)

       7:     {

       8:         Timer t = _timer.Value;

       9:         request.Headers.AcceptEncoding.Clear();

        HttpResponseMessage response = request.CreateResponse();

        response.Content = new PushStreamContent(OnStreamAvailable, "text/plain");

        return response;

    }

  

    private static void OnStreamAvailable(Stream stream, HttpContentHeaders headers, TransportContext context)

    {

        StreamWriter sWriter = new StreamWriter(stream);

        _outputs.TryAdd(sWriter, sWriter);

    }

  

    private static void TimerCallback(object state)

    {

        foreach (var kvp in _outputs.ToArray())

        {

            try

            {

                kvp.Value.Write("Here's some more data...");

                kvp.Value.Flush();

            }

            catch

            {

                StreamWriter sWriter;

                _outputs.TryRemove(kvp.Value, out sWriter);

            }

        }

    }

}

Buffered Media Type Formatter

The BufferedMediaTypeFormatter provides a convenient way for writing a MediaTypeFormatter that primarily is writing or reading small, synchronous pieces of data. This is for example the case by many serializers which don’t quite support asynchronous reading and writing. Instead of having to implement the Task-oriented MediaTypeFormatter for reading and writing the BufferedMediaTypeFormatter exposes a simplified model backed by a buffer that is well suited to handle many small reads or writes. For more details, see this commit.

Below is a sample plaintext formatter written using the BufferedMediaTypeFormatter which supports writing or reading plain text using either UTF8 or UTF-16 as determined automatically by the built-in content negotiation algorithm (see ASP.NET Web API Content Negotiation and Accept-Charset):

       1: public class SampleBufferedMediaTypeFormatter : BufferedMediaTypeFormatter

       2: {

       3:     public SampleBufferedMediaTypeFormatter()

       4:     {

       5:         // Set supported media type

       6:         SupportedMediaTypes.Add(new MediaTypeHeaderValue("text/plain"));

       7:  

       8:         // Set default supported character encodings

       9:         SupportedEncodings.Add(new UTF8Encoding(encoderShouldEmitUTF8Identifier: false, throwOnInvalidBytes: true));

        SupportedEncodings.Add(new UnicodeEncoding(bigEndian: false, byteOrderMark: true, throwOnInvalidBytes: true));

    }

  

    public override bool CanReadType(Type type)

    {

        return type == typeof(string);

    }

  

    public override bool CanWriteType(Type type)

    {

        return type == typeof(string);

    }

  

    public override object ReadFromStream(Type type, Stream stream, HttpContentHeaders contentHeaders, IFormatterLogger formatterLogger)

    {

        Encoding effectiveEncoding = SelectCharacterEncoding(contentHeaders);

        using (StreamReader sReader = new StreamReader(stream, effectiveEncoding))

        {

            return sReader.ReadToEnd();

        }

    }

  

    public override void WriteToStream(Type type, object value, Stream stream, HttpContentHeaders contentHeaders)

    {

        Encoding effectiveEncoding = SelectCharacterEncoding(contentHeaders);

        using (StreamWriter sWriter = new StreamWriter(stream, effectiveEncoding))

        {

            sWriter.Write(value);

        }

    }

}

What do you think? Please use the CodePlex discussion forum and go vote on your favorite issue in Issue Tracker – we would love to hear you feedback.

Have fun!

Henrik

del.icio.us Tags: asp.net,webapi,rest,http

ASP.NET Web API Content Negotiation and Accept-Charset

Henrik F Nielsen — Sun, 22 Apr 2012 19:46:34 GMT

There are several good blogs on content negotiation in ASP.NET Web API (see for example ASP.NET Web API: How content negotiation works?) describing how to get JSON, XML, and other response formats depending on what you ask for in a request using the Accept HTTP request header field.

However, recently we added support for the header Accept-Charset which can be used by the client to indicate which character encoding it prefers the response to be encoded with. Typical encodings are UTF-8 and UTF-16 (and our built-in formatters support those) but the set is extensible to other encodings as well.

Note: This is the first in a series of blogs on upcoming features that have been checked into our CodePlex source code repository but haven’t yet been released as part of our official installer. Think of this as a sneak-peak for you to try out and comment on as part of our source code repository (see Getting Started With ASP.NET Web Stack Source on CodePlex for details).

Seeing Accept-Charset in Action

To see how the Accept-Charset header affects the content negotiation it is simplest to use Fiddler. In the following we use the completely vanilla out-of-the-box ValuesController without any changes – we just run it. In Fiddler we use the Composer tab to send a request to the ValuesController like this:

The response coming from the server looks like this. Note the media type parameter charset=utf-8:

Now, go back to the composer tab and add Accept-Charset: utf-16 and you will see this response:

Note also that the Content-Length is twice as big as UTF-16 takes more space.

Now, you can also use q-values with the Accept-Charset header to indicate multiple values with varying quality factors associated. For example if you in the composer tab insert the header field Accept-Charset: utf-16, utf-8; q=0.5 (which means I prefer UTF-16 but can also take UTF-8) then you get the same response as above with a UTF-16 encoded response.

Changing the Default Encodings

The default encoding used by the built-in formatters is UTF-8. This means that if the request doesn’t indicate any preference then we will use UTF-8. However, you can easily change this on a per-formatter basis by changing the SupportEncodings property on the MediaTypeFormatter class. For example, if you want to swap the default encoding from UTF-8 with UTF-16 then you can just reverse the default list so that UTF-16 is fist and UTF-8 is second (see line 3-5):

        1: protected void Application_Start()

       2: {

       3:     Encoding oldDefault = GlobalConfiguration.Configuration.Formatters.JsonFormatter.SupportedEncodings[0];

       4:     GlobalConfiguration.Configuration.Formatters.JsonFormatter.SupportedEncodings.Add(oldDefault);

       5:     GlobalConfiguration.Configuration.Formatters.JsonFormatter.SupportedEncodings.RemoveAt(0);

       6:  

       7:     AreaRegistration.RegisterAllAreas();

       8:  

       9:     RegisterGlobalFilters(GlobalFilters.Filters);

      10:     RegisterRoutes(RouteTable.Routes);

      11:  

      12:     BundleConfig.RegisterBundles(BundleTable.Bundles);

      13: }

When running the same sample above you will see a UTF-16 response when no accept-charset header is present. However, if you set the accept-charset header to UTF-8 then you will still get that.

Have fun!

Henrik

del.icio.us Tags: asp.net,webapi,rest,http

Contributing to ASP.NET Web Stack Source on CodePlex

Henrik F Nielsen — Tue, 10 Apr 2012 20:16:14 GMT

In the blog Getting Started With ASP.NET Web Stack Source on CodePlex we describe how to set up a read-only clone of the ASP.NET Web Stack source repository. This allows you to sync your tree to the latest changes as soon as they appear but you can’t update the code, fix bugs, or suggest changes. In short, it is read-only.

This blog describes how to set up and then use a fork to be able to contribute to the project. It first describes setting up a fork as a one-time process and then it describes three common workflows for working with the fork on a regular basis.

If you haven’t already, then now is a good time to check out the process for contributing to the ASP.NET Web Stack project. It describes in detail what you need to do in order to contribute with as little pain as possible.

Why a Fork?

A common way for open source projects to allow contributions is by using forks. A fork is not actually a Git concept but a way for you to host your own copy of the ASP.NET Web Stack repository in CodePlex where you have write access and can submit changes without interfering with anybody else. The fork also allows you to work on your changes simultaneously across multiple machines or even with other people with whom you collaborate.

Creating a Fork

The first step is to create a fork which you do by going to the CodePlex repository and selecting Create Fork:

You will now see another page where you can fill in the fork name and a description. A good name to use for your fork is aspnetwebstack.

Now hit Save to create the fork. Select the Forks menu and you will see the fork you just created along with a clone URI.

Cloning the Fork

As the fork lives on CodePlex you need to clone it to your local dev machine in order to work on it. As in the previous blog we use PowerShell with Git extensions so if you haven’t yet set that up then please check that blog for details.

We use the fork URI that you got above to create a clone on your local dev machine. This looks similar to when you cloned the main repository in the previous blog – the only difference is the repository URI which has your CodePlex user name in two places:

https://<CodePlexUserName>@git01.codeplex.com/forks/<CodePlexUserName>/aspnetwebstack

You use it with the clone command like this:

git clone https://<UserName>@git01.codeplex.com/forks/<UserName>/aspnetwebstack

Make sure the folder aspnetwebstack doesn’t conflict with anything you already have in the folder where you do the clone.

The next step of getting the fork up and running is configuring Git with your name and email alias so that it is clear that the commits come from you. From the same prompt as above run the two commands from the aspnetwebstack folder:

git config user.name YourCodePlexUserName
git config user.email name@example.com

Also, if you run a commit from the command line and you don’t include the –m option, it will pop up a text editor for you to compose the commit message. That text editor is VIM by default. If you’re not a VI person, then you might want to set an alternative editor (I use notepad2 but you can also use GitPad). You can do this with this command (note the forward slashes instead of backslashes):

git config core.editor C:/Path/To/MyFavoriteEditor.exe

In order not to get confused about the setup, the thing to keep in mind is that there are three repositories in play:

The upstream repository which is the main repository on CodePlex
The origin repository which is your fork on CodePlex
The local repository on your dev machine

To make it easy to refer to the upstream repository we create an alias in addition to origin. You add an alias called upstream referring to the main repository URI and not the fork URI by running the command from the aspnetwebapi folder:

git remote add upstream https://git01.codeplex.com/aspnetwebstack.git

This command merely creates a local alias – nothing else happens so you won’t see more than this:

Ok, that’s it for setting up the repositories. While it took some effort to get here the good part is that up to this point the process is something you only have to do once. With some care (as will be described below) a fork can last you a long time serving as your environment for fixing any number of bugs, suggesting new features, and more. In short, it is your environment for actively working on the source.

The actual day-to-day process of working in a fork typically involves the following three steps:

Synchronizing master branches across the three repositories.
Creating a feature branch for each bug fix or feature work you work on.
Submitting pull requests with your feature branch changes.

In the following we will describe all three of these workflows in detail.

Workflow 1: Synchronizing Master Branches

The term origin is now referring to your fork repository and not to the upstream repository. That is, when you push to origin, you push to your fork, not to the upstream repository. In other words, the fork acts in all respects like the upstream repository except that you can submit your changes whenever you want.

After you created your origin and your local repository, changes will continue to happen in the upstream repository. In order to see these changes in your origin and local repositories you need to pull them – it is not done automatically. The goal of this workflow is to ensure that your local master is in sync with your origin master which is in sync with the upstream master.

The synchronization actually happens in your local repository on your dev box – the origin won’t get updated until you push the edits to it. That is, you sync your local master with the upstream master and push that to the origin master. Here is what you do:

git checkout master
git pull --rebase upstream master
git push origin

It should look something like this:

Workflow 2: Creating a Feature Branch

In order to keep the fork clean and use it for a long time you should do all your work in feature branches and not in the master branch. Keeping the master branch clean will make it easier to stay in sync with the upstream repository and generally avoid heartaches.

Whenever you create a new feature branch, make sure you are in the master branch (you don’t want to create a branch of a branch just yet). That is, you should see [master] in your prompt. You create a branch using the command

git checkout -b Feature1

which will look like this:

Note that the prompt now says [Feature1] and not [master]. Within the new branch we start Visual Studio and load the runtime.sln solution as described in the previous blog. Here it doesn’t matter what the change is but for illustration purposes we fix a typo in the class MediaTypeHeaderValueExtensions where we change the documentation to say “subset” instead of “superset”.

We then compile and run unit tests to see that everything is as we expect. Once the fix has been verified you can either commit it using TortoiseGit or the Git Source Control Provider directly from within Visual Studio. For simplicity we here just commit it using the PowerShell and the command

git checkout Feature1
git commit -am "Fixed documentation typo"

which will show this (note the red numbers which tells you that you have 1 modified file):

You now make sure that your feature branch is up to date with the local master branch by typing this command:

git checkout Feature1
git rebase master

Depending on your changes and the changes in your local master you may be asked to merge but in this scenario there are no merges.

Once you are done with your change and want to submit it back to the origin (the fork) you push to origin

git checkout Feature1
git push origin

Workflow 3: Submitting a Pull Request

When you are done with your change in the feature branch and committed it to origin you can now suggest that it be incorporated into the main repository. You do this by submitting a pull request with your change. Before submitting a pull request please review carefully the process for submitting contributions. The pull request will then go through a process of review looking at things like how it fits with the overall design, whether it has unit tests, whether the coding guidelines are used, etc.

To submit a pull request go to your fork and select the repository link as shown below:

Now set Branch to your feature branch and you should see the submits that you have done:

Now select Send Pull Request, fill in the details including which issue you are submitting a pull request for.

Once the change passes the process and gone through a full review it will get checked into the main repository and voila – you have contributed to the project!

Have fun!

Henrik

del.icio.us Tags: Asp.net,webapi,mvc,codeplex

Async Mashups using ASP.NET Web API

Henrik F Nielsen — Sat, 03 Mar 2012 23:41:27 GMT

This blog shows an example of how to build an ASP.NET Web API ApiController that asynchronously talks to multiple other Web APIs in parallel without blocking a thread on the server. Btw, if you have detected a certain theme in these blogs around using Tasks with ASP.NET Web API then you are indeed on to something :)

Asynchronous programming is an important part of building scalable, robust, and responsive Web applications regardless of whether on client side or server side. Asynchronous programming has traditionally been very complicated leaving it to only the most dedicated to implement it but with the new Task model even complex patterns such as dealing with multiple asynchronous requests in parallel are manageable without braking too much of a sweat. The sample is written using Visual Studio 11 Beta but can be modified to run on Visual Studio 2010 as well.

Scenario

In the scenario we have an ApiController that exposes a query operation with a single token (in the sequence diagram below the sample query token is “microsoft”). The ApiController then turns around and issues a query to digg and delicious respectively for stories on the query token. When both results are complete, the results are accumulated into a single response that is then sent back to the client. The goal is to do the entire ApiController request asynchronously and to issue the queries to digg and delicious in parallel so that we don’t every block a thread and optimize network utilization.

The response we send back to the client is a collection of “stories” that simply look like this:

        1: public class Story

       2: {

       3:     // The source (either digg or delicious)

       4:     public string Source { get; set; }

       5:  

       6:     // The description of the story

       7:     public string Description { get; set; }

       8: }

Creating the ApiController

Writing the controller involves three things: two helpers for processing the requests for digg and delicious respectively and then the controller to put it all together.

Executing digg Query

First we write a helper for processing the digg query. Here we submit a request, wait for the response, and then process it as JsonValue to build a set of Story instances as defined in the scenario above. As there is no instance state involved the query processing can be done as a static method.

       1: private static async Task<List<Story>> ExecuteDiggQuery(string queryToken)

       2: {

       3:     List<Story> result = new List<Story>();

       4:  

       5:     // URI query for a basic digg query -- see http://developers.digg.com/documentation

       6:     string query = string.Format("http://services.digg.com/2.0/search.search?query={0}", queryToken);

       7:  

       8:     // Submit async request 

       9:     HttpResponseMessage diggResponse = await _client.GetAsync(query);

      10:  

      11:     // Read result using JsonValue and process the stories 

      12:     if (diggResponse.IsSuccessStatusCode)

      13:     {

      14:         JsonValue diggResult = await diggResponse.Content.ReadAsAsync<JsonValue>();

      15:         foreach (var story in diggResult["stories"] as JsonArray)

      16:         {

      17:             result.Add(new Story

      18:             {

      19:                 Source = "digg",

      20:                 Description = story["title"].ReadAs<string>()

      21:             });

      22:         }

      23:     }

      24:  

      25:     return result;

      26: }

Executing delicious Query

Then we write a similar helper for processing the delicious query. It follows the exact same pattern and also uses JsonValue to read the response and create a set of Story instances. Again, as there is no instance state involved the query processing can be done as a static method.

       1: private static async Task<List<Story>> ExecuteDeliciousQuery(string queryToken)

       2: {

       3:     List<Story> result = new List<Story>();

       4:  

       5:     // URI query for a basic delicious query -- see http://delicious.com/developers

       6:     string query = string.Format("http://feeds.delicious.com/v2/json/tag/{0}", queryToken);

       7:  

       8:     // Submit async request 

       9:     HttpResponseMessage deliciousResponse = await _client.GetAsync(query);

      10:  

      11:     // Read result using JsonValue and process the stories 

      12:     if (deliciousResponse.IsSuccessStatusCode)

      13:     {

      14:         JsonArray deliciousResult = await deliciousResponse.Content.ReadAsAsync<JsonArray>();

      15:         foreach (var story in deliciousResult)

      16:         {

      17:             result.Add(new Story

      18:             {

      19:                 Source = "delicious",

      20:                 Description = story["d"].ReadAs<string>()

      21:             });

      22:         }

      23:     }

      24:  

      25:     return result;

      26: }

Writing the Controller

Now we can write the actual ApiController. First we look for a valid query token (we don’t want it to contain any ‘&’ characters). Then we kick off the two helpers in parallel and wait for them to complete (but without blocking a thread using the Task.WhenAll construct). Finally we aggregate the two results and return them to the client.

We of course use HttpClient to submit requests to dig and delicious but as HttpClient can handler requests submitted from multiple threads simultaneously we only need one instance for all requests. This means that the same HttpClient instance is reused for all requests across all ApiController instances.

       1: public async Task<List<Story>> GetContent(string topic)

       2: {

       3:     List<Story> result = new List<Story>();

       4:  

       5:     // Check that we have a topic or return empty list

       6:     if (topic == null)

       7:     {

       8:         return result;

       9:     }

      10:  

      11:     // Isolate topic to ensure we have a single term

      12:     string queryToken = topic.Split(new char[] { '&' }).FirstOrDefault();

      13:  

      14:     // Submit async query requests and process responses in parallel

      15:     List<Story>[] queryResults = await Task.WhenAll(

      16:         ExecuteDiggQuery(queryToken),

      17:         ExecuteDeliciousQuery(queryToken));

      18:  

      19:     // Aggregate results from digg and delicious

      20:     foreach (List<Story> queryResult in queryResults)

      21:     {

      22:         result.AddRange(queryResult);

      23:     }

      24:  

      25:     return result;

      26: }

That’s all we need for the controller – next is just to host it and then run it.

Hosting the Controller

As usual we use a simple command line program for hosting the ApiController and it follows the usual pattern seen in the other blogs:

       1: static void Main(string[] args)

       2: {

       3:     var baseAddress = "http://localhost:8080/";

       4:     HttpSelfHostServer server = null;

       5:  

       6:     try

       7:     {

       8:         // Create configuration

       9:         var config = new HttpSelfHostConfiguration(baseAddress);

  

        // Add a route

        config.Routes.MapHttpRoute(

          name: "default",

          routeTemplate: "api/{controller}/{id}",

          defaults: new { controller = "Home", id = RouteParameter.Optional });

  

        // Create server

        server = new HttpSelfHostServer(config);

  

        // Start server

        server.OpenAsync().Wait();

  

        Console.WriteLine("Hit ENTER to exit");

        Console.ReadLine();

    }

    finally

    {

        if (server != null)

        {

            server.CloseAsync().Wait();

        }

    }

}

Trying it Out

To try it out it is useful to use Fiddler as it allows you to see both the incoming and outgoing requests. In the screen capture below we use fiddler to directly compose a request to our localhost ApiController. To the left we can see the two requests to digg and delicious respectively and in the right lower corner you can see the result with parts coming from digg and parts coming from delicious.

Have fun!

Henrik

del.icio.us Tags: asp.net,webapi,mvc,rest,http,httpclient

Asynchronous File Upload using ASP.NET Web API

Henrik F Nielsen — Thu, 01 Mar 2012 09:43:00 GMT

HTML Form File upload (defined in RFC 1867) is a classic mechanism for uploading content to a Web server and is supported by all browsers that I am aware of. This blog shows how to handle Form File Upload in ASP.NET Web API asynchronously using the Task-based pattern introduced in .NET 4 and enhanced in .NET 4.5.

Using ASP.NET Web API you can upload files of any size when self hosted (well, up to System.Int64.MaxValue which in practice is “any size”). ASP.NET has a maximum limit of 2G in terms of file size that you can upload.

What is HTML File Upload?

Let’s first remind ourselves what HTML File Upload is. If you don’t need to brush up on HTML file upload then you can just skip to the next section…

You enable support for HTML File Upload in an HTML form by using the attribute enctype=”multipart/form-data” and then have a input field of type “file” like this:

        1: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">

       2: <html>

       3: <head>

       4:     <title>File Upload Sample</title>

       5: </head>

       6: <body>

       7:     <form action="http://localhost:8080/api/upload" enctype="multipart/form-data" method="POST">

       8:     What is your name?

       9:     <input name="submitter" size="40" type="text"><br>

      10:     What file are you uploading?

      11:     <input name="data" size="40" type="file">

      12:     <br>

      13:     <input type="submit">

      14:     </form>

      15: </body>

      16: </html>

This will cause all the data to be encoded using MIME multipart as follows when submitted in an HTTP POST request:

       1: Content-type: multipart/form-data, boundary=AaB03x

       2:  

       3: --AaB03x

       4: content-disposition: form-data; name="submitter"

       5:  

       6: Henrik Nielsen

       7: --AaB03x

       8: content-disposition: form-data ; name="data"; filename="file1.txt"

       9: Content-Type: text/plain

      10:  

      11:  ... contents of file1.txt ...

      12: --AaB03x--

Note how the input field names in the form are mapped to a Content-Disposition header in the MIME multipart message. Each form field such as the submitter field above is encoded as its own MIME body part (that is the term for each segment between the boundaries -- above it is the string –AaB03x).

In HTML5 many browsers support upload of multiple files within a single form submission using the multiple keyword:

       1: <!DOCTYPE HTML>

       2: <html>

       3: <head>

       4:     <title>HTML5 Multiple File Upload Sample</title>

       5: </head>

       6: <body>

       7:     <form action="http://localhost:8080/api/upload" enctype="multipart/form-data" method="POST">

       8:     What is your name?

       9:     <input name="submitter" size="40" type="text"><br>

      10:     What files are you uploading?

      11:     <input name="data" type=file multiple>

      12:     <br>

      13:     <input type="submit" />

      14:     </form>

      15: </body>

      16: </html>

      17:

The principle in submitting the data is the same as for HTML 4 but you can now select multiple files that each end up in their own MIME body part.

Creating an ApiController

First we create an ApiController that implements an HTTP POST action handling the file upload. Note that the action returns Task<T> as we read the file asynchronously.

Note: We use the new async/await keywords introduced in Visual Studio 11 Beta but you can equally well use Tasks and the ContinueWith pattern already present in Visual Studio 2010.

The first thing we do is check that the content is indeed “multipart/form-data”. The second thing we do is creating a MultipartFormDataStreamProvider which gives you control over where the content ends up. In this case we save the file in the folder “c:\tmp\uploads”. It also contains information about the files stored.

If you want complete control over how the file is written and what file name is used then you can derive from MultipartFormDataStreamProvider, override the functionality you want and use that StreamProvider instead.

Once the read operation has completed we check the at is done we read the content asynchronously and when that task has completed we generate a response containing the submitter and the file names we use on the server, Obviously this is not a typical response but this is just so that you can see the information.

       1: public class UploadController : ApiController

       2: {

       3:     public async Task<List<string>> PostMultipartStream()

       4:     {

       5:         // Verify that this is an HTML Form file upload request

       6:         if (!Request.Content.IsMimeMultipartContent("form-data"))

       7:         {

       8:             throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);

       9:         }

  

        // Create a stream provider for setting up output streams that saves the output under c:\tmp\uploads

        // If you want full control over how the stream is saved then derive from MultipartFormDataStreamProvider

        // and override what you need.

        MultipartFormDataStreamProvider streamProvider = new MultipartFormDataStreamProvider("c:\\tmp\\uploads");

  

        // Read the MIME multipart content using the stream provider we just created.

        IEnumerable<HttpContent> bodyparts = await Request.Content.ReadAsMultipartAsync(streamProvider);

  

        // The submitter field is the entity with a Content-Disposition header field with a "name" parameter with value "submitter"

        string submitter;

        if (!bodyparts.TryGetFormFieldValue("submitter", out submitter))

        {

            submitter = "unknown";

        }

  

        // Get a dictionary of local file names from stream provider.

        // The filename parameters provided in Content-Disposition header fields are the keys.

        // The local file names where the files are stored are the values.

        IDictionary<string, string> bodyPartFileNames = streamProvider.BodyPartFileNames;

  

        // Create response containing information about the stored files.

        List<string> result = new List<string>();

        result.Add(submitter);

  

        IEnumerable<string> localFiles = bodyPartFileNames.Select(kv => kv.Value);

        result.AddRange(localFiles);

  

        return result;

    }

}

In the above code we added an extension method for getting the value of the submitter form field as a string. That extension method looks like this (don’t worry, we will integrate this better):

       1: public static bool TryGetFormFieldValue(this IEnumerable<HttpContent> contents, string dispositionName, out string formFieldValue)

       2: {

       3:     if (contents == null)

       4:     {

       5:         throw new ArgumentNullException("contents");

       6:     }

       7:  

       8:     HttpContent content = contents.FirstDispositionNameOrDefault(dispositionName);

       9:     if (content != null)

      10:     {

      11:         formFieldValue = content.ReadAsStringAsync().Result;

      12:         return true;

      13:     }

      14:  

      15:     formFieldValue = null;

      16:     return false;

      17: }

Hosting the Controller

In this example we build a simple console application for hosting the ApiController while setting the MaxReceivedMessageSize and TransferMode in the HttpSelfHostConfiguration. We then use a simple HTML form to point to the ApiController so that we can use a browser to upload a file. That HTML form can be hosted anywhere – here we just drop it into the folder C:\inetpub\wwwroot\Samples and serve it using IIS.

       1: class Program

       2: {

       3:     static void Main(string[] args)

       4:     {

       5:         var baseAddress = "http://localhost:8080/";

       6:         HttpSelfHostServer server = null;

       7:  

       8:         try

       9:         {

            // Create configuration

            var config = new HttpSelfHostConfiguration(baseAddress);

  

            // Set the max message size to 1M instead of the default size of 64k and also

            // set the transfer mode to 'streamed' so that don't allocate a 1M buffer but 

            // rather just have a small read buffer.

            config.MaxReceivedMessageSize = 1024 * 1024;

            config.TransferMode = TransferMode.Streamed;

  

            // Add a route

            config.Routes.MapHttpRoute(

              name: "default",

              routeTemplate: "api/{controller}/{id}",

              defaults: new { controller = "Home", id = RouteParameter.Optional });

  

            server = new HttpSelfHostServer(config);

  

            server.OpenAsync().Wait();

  

            Console.WriteLine("Hit ENTER to exit");

            Console.ReadLine();

        }

        finally

        {

            if (server != null)

            {

                server.CloseAsync().Wait();

            }

        }

    }

}

Trying it Out

Start the console application, then point your browser at the HTML form, for example http://localhost/samples/uploadsample.html.

Once you have uploaded a file you will find it in the folder c:\\tmp\uploads and also get a content negotiated result which if asking for JSON looke like this:

["henrik","c:\\tmp\\uploads\\sample.random"]

Have fun!

Henrik

del.icio.us Tags: asp.net,webapi,mvc,rest,http

Async Streaming in ASP.NET Web API

Henrik F Nielsen — Fri, 24 Feb 2012 11:03:00 GMT

ASP.NET Web API supports asynchronous actions using the new Task-based programming model introduced in .NET 4. In .NET 4.5 the model has improved further and in addition is supported directly from C# using the new async and await keywords (see “Visual Studio Asynchronous Programming” for background).

What this means is that you can have actions that look like this:

        1: public Task<string[]> GetContent(string topic)

       2: {

       3:     ...

       4: }

as well as

       1: public Task<HttpResponseMessage> GetContent(string topic)

       2: {

       3:     ...

       4: }

and even this

       1: public Task GetContent(string topic)

       2: {

       3:     ...

       4: }

The benefit of writing Task-based actions is that you don’t block a thread on the server even if things take a long time to complete and that allows the server to scale better and be more responsive.

In the blog “Push and Pull Streams using HttpClient” we described how to use push and pull streams with HttpContent. As HttpRequestMessage, HttpResponseMesssage, and HttpContent apply to both client side (as part of HttpClient) as well as server side (as part of ASP.NET Web API) the push vs. pull model described in that blog applies equally well to the server side.

In this blog we expand on the topic of reading request content using asynchronous streaming within an action.

Reading Request Body Asynchronously

In the first scenario we implement a POST action that takes a JSON entity body and writes it to another stream. Note that the action returns Task<HttpResponseMessage> which means that the action completes when the task completes. The important part of this action is line 11 where we copy the content from the request to another stream (we describe the SummaryStream below):

       1: public Task<HttpResponseMessage> PostContent()

       2: {

       3:     // Verify that this is JSON content

       4:     if (!Request.Content.IsJsonContent())

       5:     {

       6:         throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);

       7:     }

       8:  

       9:     // Copy content asynchronously to our SummaryStream and return the continuation task

      10:     SummaryStream summaryStream = new SummaryStream();

      11:     return Request.Content.CopyToAsync(summaryStream).ContinueWith(

      12:         (readTask) =>

      13:         {

      14:             summaryStream.Close();

      15:  

      16:             // Check whether we completed successfully and generate response

      17:             if (readTask.IsCompleted)

      18:             {

      19:                 return new HttpResponseMessage(HttpStatusCode.OK);

      20:             }

      21:             else

      22:             {

      23:                 return new HttpResponseMessage(HttpStatusCode.BadRequest);

      24:             }

      25:         });

      26: }

This is essentially all you have to do – the rest is just filler but if you want to dive in a little more then keep going! The first thing the action does is checking that the content is actually JSON. To do this we use a little extension method that we have added for this purpose:

       1: /// <summary>

       2: /// Determines whether the specified content has an JSON media type of either 

       3: /// <c>application/json</c> or <c>text/json</c>. 

       4: /// </summary>

       5: /// <param name="content">The content.</param>

       6: /// <returns>

       7: /// <c>true</c> if the specified content is XML with a media type of either <c>application/json</c> or <c>text/json</c>.

       8: /// </returns>

       9: public static bool IsJsonContent(this HttpContent content)

      10: {

      11:     if (content == null)

      12:     {

      13:         throw new ArgumentNullException("content");

      14:     }

      15:  

      16:     if (content.Headers == null || content.Headers.ContentType == null)

      17:     {

      18:         return false;

      19:     }

      20:  

      21:     MediaTypeHeaderValue contentType = content.Headers.ContentType;

      22:     return

      23:         contentType.MediaType.Equals("application/json", StringComparison.OrdinalIgnoreCase) ||

      24:         contentType.MediaType.Equals("text/json", StringComparison.OrdinalIgnoreCase);

      25: }

To see what is going on when reading the content we create a special stream called SummaryStream that simply indicates when it gets called by writing to the console. You don’t have to use this but if you want to try it out then it looks like this:

       1: /// <summary>

       2: /// Sample stream that write how many characters were written to it.

       3: /// </summary>

       4: internal class SummaryStream : DelegatingStream

       5: {

       6:     public SummaryStream()

       7:         : base(Stream.Null)

       8:     {

       9:     }

  

    public override void Flush()

    {

        Console.WriteLine("Flushing stream...");

        _innerStream.Flush();

    }

  

    public override void Write(byte[] buffer, int offset, int count)

    {

        Console.WriteLine("Writing {0} bytes SYNChronously...", count);

        _innerStream.Write(buffer, offset, count);

    }

  

    public override void WriteByte(byte value)

    {

        Console.WriteLine("Writing a single byte SYNChronously...");

        _innerStream.WriteByte(value);

    }

  

    public override IAsyncResult BeginWrite(byte[] buffer, int offset, int count, AsyncCallback callback, object state)

    {

        Console.WriteLine("Writing {0} bytes ASYNChronously...", count);

        return _innerStream.BeginWrite(buffer, offset, count, callback, state);

    }

  

    public override void Close()

    {

        Console.WriteLine("Closing stream...");

        base.Close();

    }

}

The DelegatingStream which SummaryStream derives from is nothing more than delegating every call to the inner stream. It looks like this:

       1: /// <summary>

       2: /// Utility <see cref="Stream"/> which delegates everything to an inner <see cref="Stream"/>

       3: /// </summary>

       4: internal abstract class DelegatingStream : Stream

       5: {

       6:     protected Stream _innerStream;

       7:  

       8:     protected DelegatingStream(Stream innerStream)

       9:     {

        Contract.Assert(innerStream != null);

        _innerStream = innerStream;

    }

  

    public override bool CanRead

    {

        get { return _innerStream.CanRead; }

    }

  

    public override bool CanSeek

    {

        get { return _innerStream.CanSeek; }

    }

  

    public override bool CanWrite

    {

        get { return _innerStream.CanWrite; }

    }

  

    public override long Length

    {

        get { return _innerStream.Length; }

    }

  

    public override long Position

    {

        get { return _innerStream.Position; }

        set { _innerStream.Position = value; }

    }

  

    public override int ReadTimeout

    {

        get { return _innerStream.ReadTimeout; }

        set { _innerStream.ReadTimeout = value; }

    }

  

    public override bool CanTimeout

    {

        get { return _innerStream.CanTimeout; }

    }

  

    public override int WriteTimeout

    {

        get { return _innerStream.WriteTimeout; }

        set { _innerStream.WriteTimeout = value; }

    }

  

    protected override void Dispose(bool disposing)

    {

        if (disposing)

        {

            _innerStream.Dispose();

        }

        base.Dispose(disposing);

    }

  

    public override long Seek(long offset, SeekOrigin origin)

    {

        return _innerStream.Seek(offset, origin);

    }

  

    public override int Read(byte[] buffer, int offset, int count)

    {

        return _innerStream.Read(buffer, offset, count);

    }

  

    public override IAsyncResult BeginRead(byte[] buffer, int offset, int count, AsyncCallback callback, object state)

    {

        return _innerStream.BeginRead(buffer, offset, count, callback, state);

    }

  

    public override int EndRead(IAsyncResult asyncResult)

    {

        return _innerStream.EndRead(asyncResult);

    }

  

    public override int ReadByte()

    {

        return _innerStream.ReadByte();

    }

  

    public override void Flush()

    {

        _innerStream.Flush();

    }

  

    public override void SetLength(long value)

    {

        _innerStream.SetLength(value);

    }

  

    public override void Write(byte[] buffer, int offset, int count)

    {

        _innerStream.Write(buffer, offset, count);

    }

  

    public override IAsyncResult BeginWrite(byte[] buffer, int offset, int count, AsyncCallback callback, object state)

    {

        return _innerStream.BeginWrite(buffer, offset, count, callback, state);

    }

  

    public override void EndWrite(IAsyncResult asyncResult)

    {

        _innerStream.EndWrite(asyncResult);

    }

  

    public override void WriteByte(byte value)

    {

        _innerStream.WriteByte(value);

    }

}

Trying It Out

To run the sample we start it in self host as usual.

Note: By default the max message size in self host is 64k so to accept larger request sizes we set it to 1M and also turn on streaming so that we don’t buffer the whole thing in memory at any point in time.

       1: static void Main(string[] args)

       2: {

       3:     var baseAddress = "http://localhost:8080/";

       4:     HttpSelfHostServer server = null;

       5:  

       6:     try

       7:     {

       8:         // Create configuration

       9:         var config = new HttpSelfHostConfiguration(baseAddress);

  

        // Set the max message size to 1M instead of the default size of 64k and also

        // set the transfer mode to 'streamed' so that don't allocate a 1M buffer but 

        // rather just have a small read buffer.

        config.MaxReceivedMessageSize = 1024 * 1024;

        config.TransferMode = TransferMode.Streamed;

  

        // Add a route

        config.Routes.MapHttpRoute(

          name: "default",

          routeTemplate: "api/{controller}/{id}",

          defaults: new { controller = "Home", id = RouteParameter.Optional });

  

        server = new HttpSelfHostServer(config);

  

        server.OpenAsync().Wait();

  

        Console.WriteLine("Hit ENTER to exit");

        Console.ReadLine();

    }

    finally

    {

        if (server != null)

        {

            server.CloseAsync().Wait();

        }

    }

}

We then use Fiddler to submit an HTTP POST request with an JSON request body. That is, we open the Composer tab and fill in a POST request with whatever JSON content you want to submit:

The resulting output to the console shows that the data is written asynchronously to the SummaryStream:

Have fun!

Henrik

del.icio.us Tags: asp.net,webapi,mvc,rest,http

Using ASP.NET Web API with ASP.NET Web Forms

Henrik F Nielsen — Thu, 23 Feb 2012 21:30:43 GMT

Several of the ASP.NET Web API tutorials (see for example “Your First ASP.NET Web API”) show how you can use ASP.NET Web API with MVC applications. However, you can equally well add a Web API to a Web Form enabling the same Web API support as in MVC applications.

To show how you do this, let’s build a Web Site from scratch and then add a Web API to it. We use the Visual Web Developer 2010 Express version of Visual Studio with ASP.NET Web API Beta installed.

First we create a new ASP.NET Empty Web Site and give it a reasonable name:

Then we add an ASP.NET Folder to hold our Web API implementation. We do this by adding an App_Code folder as follows:

Next we add a Web API Controller Class within the App_Code folder using Add New Item.

Note: Make sure the name ends in “Controller” and not “Controller1” or similar.

The newly created ValuesController illustrates a very basic Web API with support for HTTP GET, POST, PUT, and DELETE.

Note: You may have to fix the namespace so that it doesn’t start with a "dot “.” (this is a bug):

Let’s leave the default ValuesController for now and instead add a route to the global route table so that we can route messages to the Web API.

To do this, open the file Global.asax in the project. Here we need to add two things:

First we add a couple of import statements for System.Web.Routing and System.Web.Http to get the right name places
Second we add a route entry in the global route table in the Application_Start method

The two edits together look like this:

        1: <%@ Application Language="C#" %>

       2: <%@ Import Namespace="System.Web.Routing" %>

       3: <%@ Import Namespace="System.Web.Http" %>

       4:  

       5: <script runat="server">

       6:  

       7:     void Application_Start(object sender, EventArgs e) 

       8:     {

       9:         RouteTable.Routes.MapHttpRoute(

      10:             name: "DefaultApi",

      11:             routeTemplate: "api/{controller}/{id}",

      12:             defaults: new { id = System.Web.Http.RouteParameter.Optional }

      13:         );

      14:     }

You are now ready to try it out so hit F5 and start it!

To see what is going on you can use your browser to send a request to your controller. Typically you would use JavaScript AJAX calls (see for example the tutorial “Your First ASP.NET Web API”) to invoke the ApiController but for GET you can also just use your browser and point it at the address “api/values” relative to the base address of your Web Site.

For example, if your Web Site is running as

http://localhost:47836/WebSite1/

then your values controller will be answering at

http://localhost:47836/WebSite1/api/values

In IE the default response will be returned as application/json which IE will ask whether you want to open or save. If you save the result and open it then see the JSON result

["value1","value2"]

Alternatively, if you use the address

http://localhost:47836/WebSite1/api/values/1

then you will see only the first value

"value"

That’s it – you now have a Web API running in your Web Site.

Have fun!

Henrik

del.icio.us Tags: asp.net,web,mvc,webapi,rest

ASP.NET Web API and HttpClient Available on NuGet

Henrik F Nielsen — Mon, 20 Feb 2012 19:35:09 GMT

If you don’t need the full ASP.NET MVC 4 installer with project and item templates for Visual Studio 2010 then you can download the various ASP.NET Web API packages directly from NuGet (see list of packages below). If you are new to NuGet then it is a Visual Studio extension that makes it easy to add, remove, and update libraries and tools in Visual Studio projects that use the .NET Framework (check out the NuGet overview).

NuGet Packages

Here are the relevant NuGet packages depending on what you want to build:

Web API hosted in ASP.NET: AspNetWebApi
Self-hosted Web API: AspNetWebApi.Selfhost
HttpClient including XML and JSON formatters: System.Net.Http.Formatting
JsonValue for navigating and manipulating JSON: System.Json

Additional Information

Release notes for installation instructions and to find out what’s new.
ASP.NET Web API home complete with docs, samples, tutorials and videos.

Have fun!

Henrik

del.icio.us Tags: asp.net,webapi,mvc,rest,httpclient

Using MongoDB with ASP.NET Web API

Henrik F Nielsen — Sun, 19 Feb 2012 06:53:47 GMT

MongoDB is a popular NoSQL database that makes it a great backend for Web APIs which lend themselves towards a document store rather than a relational store. In this blog we show how you can use MongoDB with ASP.NET Web API to build an ApiController with support for HTTP GET, PUT, POST, and DELETE. Our sample controller is a simple contact manager Web API which builds on many of the same concepts as in the tutorial “Creating a Web API that Supports CRUD Operations” so I would recommend that you skim that for background.

In particular, the URI patterns are very similar to that of the tutorial:

Action	HTTP Method	Relative URI
Get a list of all contacts	GET	/api/contacts
Get a filtered list of all contacts	GET	/api/contacts?$top=2
Get a contacts by ID	GET	/api/contacts/id
Create a new contact	POST	/api/contacts
Update a contact	PUT	/api/contacts/id
Delete a contact	DELETE	/api/contacts/id

However, there are two differences worth noting:

We host the sample ApiController in selfhost with a base URI of http://localhost:8080
We have added support for OData-style queries on GET requests so that you can filter the set of contacts returned using $top, $skip, etc.

But I am getting ahead of myself – let’s get back to building the controller…

Note: Please see List of ASP.NET Web API and HttpClient Samples for the complete sample solution. Note that you still have to set up the MongoDB as described in the next section.

Prerequisites

Before you start you obviously need to download and install MongoDB. I followed the quick start instructions which has all the information you need and got me going in 10 minutes or so.

Second you need the MongoDB driver for C#. Here we use the MongoDB C# driver provided by 10gen and which is available as a NuGet package.

As an optional third step once your MongoDB installation has been set up you can use a tool like MongoVUE which allows you to browse and interact directly with the data in a MongoDB database.

Defining the Contact Type

We use the same basic Contact type used in “Creating a Web API that Supports CRUD Operations” but decorate it with an BsonIdAttribute (see the Mongo C# driver reference documentation for details) indicating which field is the id:

        1: public class Contact

       2: {

       3:     [BsonId]

       4:     public string Id { get; set; }

       5:  

       6:     public string Name { get; set; }

       7:  

       8:     public string Phone { get; set; }

       9:  

      10:     public string Email { get; set; }

      11:  

      12:     public DateTime LastModified { get; set; }

      13: }

Using MongoVUE we can see how Contact instances show up in the MongoDB database. The sample Contact data we use in this sample looks something like this

Defining the Contact Repository

We use the common repository pattern to interact with the backend data source so the next step is to define the shape of the repository. We do this by defining an IContactRepository interface which we then implement so that it targets MongoDB. First, the interface is defined like this:

       1: public interface IContactRepository

       2: {

       3:     IEnumerable<Contact> GetAllContacts();

       4:  

       5:     Contact GetContact(string id);

       6:  

       7:     Contact AddContact(Contact item);

       8:  

       9:     bool RemoveContact(string id);

      10:  

      11:     bool UpdateContact(string id, Contact item);

      12: }

Implementing the Repository

It’s when implementing IContactRepository that we use the MongoDB C# driver API to connect to the data and to do the various CRUD operations defined by IContactRepository. First we set up the connection to database and get a Contact collection. By default we use the connection string “mongodb://localhost:27017” pointing at the localhost database we set up above.

For the purpose of this sample we also reset the database and add some default entries so we have something to start with. Note that the classes MongoServer, MongoDatabase, and MongoCollection<T> are all thread-safe so they can be used simultaneously by separate threads.

       1: public ContactRepository(string connection)

       2: {

       3:     if (string.IsNullOrWhiteSpace(connection))

       4:     {

       5:         connection = "mongodb://localhost:27017";

       6:     }

       7:  

       8:     _server = MongoServer.Create(connection);

       9:     _database = _server.GetDatabase("Contacts", SafeMode.True);

      10:     _contacts = _database.GetCollection<Contact>("contacts");

      11:  

      12:     // Reset database and add some default entries

      13:     _contacts.RemoveAll();

      14:     for (int index = 1; index < 5; index++)

      15:     {

      16:         Contact contact1 = new Contact

      17:         {

      18:             Email = string.Format("test{0}@example.com", index),

      19:             Name = string.Format("test{0}", index),

      20:             Phone = string.Format("{0}{0}{0} {0}{0}{0} {0}{0}{0}{0}", index)

      21:         };

      22:         AddContact(contact1);

      23:     }

      24: }

We now add the actual CRUD implementations as follows:

       1: public IEnumerable<Contact> GetAllContacts()

       2: {

       3:     return _contacts.FindAll();

       4: }

       5:  

       6: public Contact GetContact(string id)

       7: {

       8:     IMongoQuery query = Query.EQ("_id", id);

       9:     return _contacts.Find(query).FirstOrDefault();

}

  

public Contact AddContact(Contact item)

{

    item.Id = ObjectId.GenerateNewId().ToString();

    item.LastModified = DateTime.UtcNow;

    _contacts.Insert(item);

    return item;

}

  

public bool RemoveContact(string id)

{

    IMongoQuery query = Query.EQ("_id", id);

    SafeModeResult result = _contacts.Remove(query);

    return result.DocumentsAffected == 1;

}

  

public bool UpdateContact(string id, Contact item)

{

    IMongoQuery query = Query.EQ("_id", id);

    item.LastModified = DateTime.UtcNow;

    IMongoUpdate update = Update

        .Set("Email", item.Email)

        .Set("LastModified", DateTime.UtcNow)

        .Set("Name", item.Name)

        .Set("Phone", item.Phone);

    SafeModeResult result = _contacts.Update(query, update);

    return result.UpdatedExisting;

}

Creating the Contact Controller

That’s it for the repository so we can now implement the actual ApiController. We base the implementation on the common pattern for supporting GET, PUT, POST, and DELETE as follows:

       1: public class ContactsController : ApiController

       2: {

       3:     private static readonly IContactRepository _contacts = new ContactRepository();

       4:  

       5:     public IQueryable<Contact> Get()

       6:     {

       7:         return _contacts.GetAllContacts().AsQueryable();

       8:     }

       9:  

    public Contact Get(string id)

    {

        Contact contact = _contacts.GetContact(id);

        if (contact == null)

        {

            throw new HttpResponseException(HttpStatusCode.NotFound);

        }

  

        return contact;

    }

  

    public Contact Post(Contact value)

    {

        Contact contact = _contacts.AddContact(value);

        return contact;

    }

  

    public void Put(string id, Contact value)

    {

        if (!_contacts.UpdateContact(id, value))

        {

            throw new HttpResponseException(HttpStatusCode.NotFound);

        }

    }

  

    public void Delete(string id)

    {

        if (!_contacts.RemoveContact(id))

        {

            throw new HttpResponseException(HttpStatusCode.NotFound);

        }

    }

}

Note: We use IQueryable<Contact> as the return type of the Get() method. This enables automatic query-support using OData query syntax including $top and $skip. That is, if you use a URI like “/api/contacts?$top=2” with a “?$top=2” query component then you will only get the first two entries back.

Hosting the Controller

Now that we have the controller we can either host it in ASP or as selfhost. Here we use selfhost to host the controller in a simple console application but it would work exactly the same if hosted in ASP. As usual we create a HttpSelfHostConfiguration, add a route, then create a server, and start it:

       1: static void Main(string[] args)

       2: {

       3:     HttpSelfHostServer server = null;

       4:     try

       5:     {

       6:         // Set up server configuration

       7:         HttpSelfHostConfiguration config = new HttpSelfHostConfiguration("http://localhost:8080");

       8:  

       9:         config.Routes.MapHttpRoute(

            name: "DefaultApi",

            routeTemplate: "api/{controller}/{id}",

            defaults: new { id = RouteParameter.Optional }

        );

  

        // Create server

        server = new HttpSelfHostServer(config);

  

        // Start listening

        server.OpenAsync().Wait();

  

        Console.WriteLine("Hit ENTER to exit...");

        Console.ReadLine();

  

    }

    finally

    {

        if (server != null)

        {

            // Stop listening

            server.CloseAsync().Wait();

        }

    }

}

Note: In order to successfully start the selfhost server you have to run as admin (or configure http.sys with the appropriate URI prefix using netsh).

Trying out the Controller

When running the controller can be accessed using any HTTP client. In the full sample I show how to use HttpClient to do GET and POST but Fiddler is often very useful for trying out various combinations of GET, PUT, POST, and DELETE by using the Composer tab manually to create requests. For example, you can create a POST request like this to insert a new Contact and then hit Execute:

Similarly you can create a GET request to ask for the two first entries which will yield a result like this where the response body contains the first two Contacts:

As mentioned above, when you modify the data you can track the contents of the MongoDB database using MongoVUE.

Have fun!

Henrik

del.icio.us Tags: asp.net,webapi,mvc,rest,httpclient,mongodb

Using JSON.NET with ASP.NET Web API

Henrik F Nielsen — Sat, 18 Feb 2012 00:10:14 GMT

Json.Net is a popular framework for working with JSON. In particular, it has a bunch of features that are not supported by the DataContractJsonSerializer such as being much more flexible in what kind of types it can serialize and exactly how they should be serialized. The ASP.NET Web API supports an open-ended set of formatters that can read and write data to and from any media type you want to support. For example, if you want to support the vCard format which has the media type text/vcard (previously it was text/directory) media type then you can write a formatter for vCard and register it for the media type (or types) in question.

Note: JSON.NET is now an integral part of ASP.NET Web API so you can just us it out of the box.

This sample shows how to hook in Json.Net as the default formatter replacing the built in DataContractJsonSerializer formatter (in the beta bits DataContractJsonSerializer is the default formatter). There are already a bunch of Json.Net formatters provided by the community that may well be more full-featured but this should allow you to get started.

Building the Formatter

The first thing we do is building the formatter. The key part of the formatter is to provide support for reading and writing content of a given media type. The sample formatter derives from the base MediaTypeFormatter class; we are working on a buffered media type formatter that will help working with lots of small reads and writes but for this sample the goal is to keep things simple. The formatter looks like this:

        1: public class JsonNetFormatter : MediaTypeFormatter

       2: {

       3:     private JsonSerializerSettings _jsonSerializerSettings;

       4:  

       5:     public JsonNetFormatter(JsonSerializerSettings jsonSerializerSettings)

       6:     {

       7:         _jsonSerializerSettings = jsonSerializerSettings ?? new JsonSerializerSettings();

       8:  

       9:         // Fill out the mediatype and encoding we support

        SupportedMediaTypes.Add(new MediaTypeHeaderValue("application/json"));

        Encoding = new UTF8Encoding(false, true);

    }

  

    protected override bool CanReadType(Type type)

    {

        if (type == typeof(IKeyValueModel))

        {

            return false;

        }

  

        return true;

    }

  

    protected override bool CanWriteType(Type type)

    {

        return true;

    }

  

    protected override Task<object> OnReadFromStreamAsync(Type type, Stream stream, HttpContentHeaders contentHeaders, FormatterContext formatterContext)

    {

        // Create a serializer

        JsonSerializer serializer = JsonSerializer.Create(_jsonSerializerSettings);

  

        // Create task reading the content

        return Task.Factory.StartNew(() =>

        {

            using (StreamReader streamReader = new StreamReader(stream, Encoding))

            {

                using (JsonTextReader jsonTextReader = new JsonTextReader(streamReader))

                {

                    return serializer.Deserialize(jsonTextReader, type);

                }

            }

        });

    }

  

    protected override Task OnWriteToStreamAsync(Type type, object value, Stream stream, HttpContentHeaders contentHeaders, FormatterContext formatterContext, TransportContext transportContext)

    {

        // Create a serializer

        JsonSerializer serializer = JsonSerializer.Create(_jsonSerializerSettings);

  

        // Create task writing the serialized content

        return Task.Factory.StartNew(() =>

        {

            using (JsonTextWriter jsonTextWriter = new JsonTextWriter(new StreamWriter(stream, Encoding)) { CloseOutput = false })

            {

                serializer.Serialize(jsonTextWriter, value);

                jsonTextWriter.Flush();

            }

        });

    }

}

Building a Sample ApiController

Next we need a controller to try things out. For illustrative purposes we create a type that would not serialize well with DataContractJsonSerializer but other than that this is a completely vanilla controller that knows nothing about serialization:

       1: public class HomeController : ApiController

       2: {

       3:     public HomeInfo Get()

       4:     {

       5:         return new HomeInfo();

       6:     }

       7: }

       8:  

       9: public class HomeInfo

      10: {

      11:     private readonly DateTime _created = DateTime.UtcNow;

      12:     private readonly Dictionary<int, string> _colorMap = new Dictionary<int, string>

      13:     {

      14:         { 1, "blue"},

      15:         { 2, "red" },

      16:         { 3, "green" },

      17:         { 4, "black" },

      18:         { 5, "white" },

      19:     };

      20:  

      21:     public DateTime Created { get { return _created; } }

      22:  

      23:     public IDictionary<int, string> ColorMap { get { return _colorMap; } }

      24: }

Hosting the Controller

Now that we have the controller we can host it in either ASP or as selfhost. In this case we selfhost the controller in a simple console application but it would work exactly the same if hosted in ASP.

The first part is to configure the selfhost server and injecting the JsonNetFormatter as the first formatter in the configuration so that it becomes the default formatter. We also configure Json.Net to serialize DateTime types using ISO 8601 format instead of the more esoteric "/Date(1240718400000)/” format. The part of the console app that configures and starts the server looks like this:

       1: // Set up server configuration

       2: HttpSelfHostConfiguration config = new HttpSelfHostConfiguration("http://localhost:8080");

       3: config.Routes.MapHttpRoute("Default", "{controller}", new { controller = "Home" });

       4:  

       5: // Create Json.Net formatter serializing DateTime using the ISO 8601 format

       6: JsonSerializerSettings serializerSettings = new JsonSerializerSettings();

       7: serializerSettings.Converters.Add(new IsoDateTimeConverter());

       8: config.Formatters[0] = new JsonNetFormatter(serializerSettings);

       9:  

      10: // Create server

      11: server = new HttpSelfHostServer(config);

      12:  

      13: // Start listening

      14: server.OpenAsync().Wait();

Note: In order to successfully start the selfhost server you have to run as admin (or configure http.sys with the appropriate URI prefix using netsh).

Trying it Out

Once the controller is running, we can access it using any HTTP client. In this case we use HttpClient to access it and print out the result to the console. If we put both the server configuration and the client in the same Main then we get something like this:

       1: class Program

       2: {

       3:     static void Main(string[] args)

       4:     {

       5:         HttpSelfHostServer server = null;

       6:         try

       7:         {

       8:             // Set up server configuration

       9:             HttpSelfHostConfiguration config = new HttpSelfHostConfiguration("http://localhost:8080");

            config.Routes.MapHttpRoute("Default", "{controller}", new { controller = "Home" });

  

            // Create Json.Net formatter serializing DateTime using the ISO 8601 format

            JsonSerializerSettings serializerSettings = new JsonSerializerSettings();

            serializerSettings.Converters.Add(new IsoDateTimeConverter());

            config.Formatters[0] = new JsonNetFormatter(serializerSettings);

  

            // Create server

            server = new HttpSelfHostServer(config);

  

            // Start listening

            server.OpenAsync().Wait();

  

            // Create HttpClient, do an HTTP GET on the controller, and show the output

            HttpClient client = new HttpClient();

            client.GetAsync("http://localhost:8080").ContinueWith(

                (requestTask) =>

                {

                    // Get HTTP response from completed task.

                    HttpResponseMessage response = requestTask.Result;

  

                    // Check that response was successful or throw exception

                    response.EnsureSuccessStatusCode();

  

                    // Read response asynchronously as string and write out

                    response.Content.ReadAsStringAsync().ContinueWith(

                        (readTask) =>

                        {

                            Console.WriteLine(readTask.Result);

                        });

                });

  

            Console.WriteLine("Hit ENTER to exit...");

            Console.ReadLine();

  

        }

        finally

        {

            if (server != null)

            {

                // Stop listening

                server.CloseAsync().Wait();

            }

        }

    }

}

The resulting output written to the console is

{"Created":"2012-02-18T00:54:06.8447642Z","ColorMap":{"1":"blue","2":"red","3":"green","4":"black","5":"white"}}

Note the ISO date and the nice serialization of the dictionary!

Henrik

del.icio.us Tags: asp.net,webapi,mvc,rest,httpclient