Yesterday I was having a conversation with Anil Ruia who happens to be the ARR (Application Request Routing) developer and based on customer feedback we discussed the idea of using ARR in the context of Remote Management in IIS which solves a question that several people asked me before and thought it would be fun to try it out.

Basically the question that I got asked was "Can I have a single entry-point exposed for Remote Management?", or in other words "Can I provide users with remote administration giving them a single server name like management.myhostingcompany.com, instead of having to give them the specific machine name where their site lives?". So far the answer to these questions was "not easily", however with the use of ARR and URL Rewriter we will see how easy it is to achieve this.

The only thing you need for this to work is install the new URL Rewrite and ARR Module both available here http://blogs.iis.net/bills/archive/2008/07/09/new-iis7-releases-url-rewrite-application-routing-and-load-balancing-and-powershell-cmd-lets.aspx.

Background

The Web Management Service (WMSvc) is the service that enables remote administration for IIS 7.0 Manager, providing an HTTPS end-point that exposes functionality similar to Web Services to manage the Web Server (IIS) remotely. This service uses HTTPS for its communication and exposes several configuration options that support giving access to Non-Windows Users (What we call IIS Manager Users), provide a list of IP Restrictions, support only local connections and many more that can be managed using the Management Service feature inside IIS Manager.

To enable remote administration typically you need to: 1) Configure a valid Certificate for SSL, 2) Allow Remote Connections and 3) Start the WMSvc Service, all of which can be performed in IIS Manager. Once you have successfully enabled the remote service you should be able to go to a different machine and be able to connect remotely.

Note: If you are using Windows Vista, Windows XP, or Windows 2003 to connect to a Windows Server 2008 you need to download and install the client to do this: http://www.iis.net/downloads/default.aspx?tabid=34&g=6&i=1626

However, one of the drawbacks is that in order to be able to connect to a Web Site, the end-user needs to know the machine name, as well as the name of the Web Site they will be connecting to, which sometimes it would be better to be dynamic. The following image shows the information required to enter when connecting to a Web Site. Note that if connecting to an application you will also need to enter the name of the application.

However, this can potentially reduce the flexibility for deployment options, since now your customers have specific knowledge of the physical machine and will limit the flexibility of moving the site to different machines or even changing the name of the site where it is being hosted.

ARR and URL Rewrite to the rescue.

ARR has several very interesting capabilities that are really useful for this scenario. First, we can configure it to act as a proxy and basically forward the requests to another server where they actually get processed. This is the simplest configuration option and what it allows you is to have something similar to the next image:

To set up this configuration where a front-end management server forwards the IIS Remote Management requests to another server running WMSVC you have to:

1. Install ARR and URL Rewrite in the Server that is intended to be used as the front-end for management requests. Lets call this ServerA.
2. Create a new Web Site.
   1. Navigate to IIS Manager->Site
   2. Click Add Web Site.
   3. In the dialog set: Site name:ManagementSite, Binding: https, port: 8172 and choose a valid SSL certificate, specify a phisical path. Click OK
3. Configure URL Rewrite to Route requests to the IIS Management Service running in the other computer.
   1. Navigate to IIS Manager->Sites->Management Site->URL Rewrite Module
   2. Click Add Rule
   3. Set: Name: RouteWMSvc, Pattern:.*, Rewrite URL:https://<RemoteServer>:8172/{R:0}, Stop Processing rules: Checked.
   4. This should generate a web.config with similar content (note that my backend, ie the RemoteServer in my case is carlosag2-iis below):

      <configuration>
          <system.webServer>
              <rewrite>
                  <rules>
                      <rule name="RouteWMSvc" stopProcessing="true">
                          <match url=".*" />
                          <action type="Rewrite" url="https://carlosag2-iis:8172/{R:0}" />
                      </rule>
                  </rules>
              </rewrite>
          </system.webServer>
      </configuration>

4. Now you can run IIS Manager in any client machine, specify the ServerA as the machine name and specify any web site in the remote RemoteServer, the result will be that all requests will be forwarded to the WMSvc running in the remote server.

Now, that is interesting and the scenario it allows you to do is potentially have WMSvc IP Request Filtering in the RemoteServer and only allow calls from the Management Server where you can do further configuration. Note that this also means that you can have a single public SSL Certificate in the management server and use privately issued certificates (or potentially even self-signed certificates in the remoteserver since you can control installing the certificate into the management server). It also means that the customers no longer use the physical name of the RemoteServer machine but instead connect to the Management Server allowing you to completely move them to another machine and not have to update your clients.

Troubleshooting: If you are having troubles testing this, the best thing to do is enable Failed Request Tracing in the ManagementSite, which will tell you exactly what is going on. For example you will get entries like:

Warning: ModuleName="ApplicationRequestRouting", Notification="EXECUTE_REQUEST_HANDLER", HttpStatus="502", HttpReason="Bad Gateway", HttpSubStatus="3", ErrorCode="2147954575", ConfigExceptionInfo=""

If you lookup the ErrorCode, it is actually: ERROR_WINHTTP_SECURE_FAILURE, this means that you have most likely issues with the certificate. In my case, just to test this what I did is generate a self-signed certificate in the RemoteServer with the name of the machine (carlosag2-iis) and then I installed that certificate using the MMC certificates snap-in in the management server into the Trusted Root Certification Authority. Disclaimer Warning!! this is something you should only do for testing purposes or if you know what you are doing.

More Advanced Stuff... Dynamically choosing the machine

Now, trying to push the capabilities of this I decided to solve another requests that we've heard which is closely related "Can I have a single management server and dynamically route the requests to the machine where a particular site lives?"

The following picture represents this, where the Management Server dynamically resolves the server that it should talk to using the URL Rewrite Maps functionality.

Turns out this is really simple using URL Rewrite, basically you can write a Rewrite Rule that matches the Site name that is included as part of the Query String and use the Rewrite Maps support for figuring out the machine where this site lives. The following shows such a rule:

Basically, URL Rewrite matches every request and uses the condition entry to parse the Query String and find the Site name within it. With it, it and using the Map ServersTable to resolve the machine name based on the Site name it rewrites the request to the machine where its currently located. This makes it basically route "https://localhost:8172/Service.axd?...&Site=CarlosAgWebSite" into https://carlosag2-iis:8172/Service.axd?...&Site=CarlosAgWebSite. The end result is that can dynamically at any time just update this table and make ARR route the requests to the right machine giving you complete flexibility on the deployment of sites.

One thing to note is that URL Rewrite is one of the ways you can make the ARR scenario work, however, you could also write your own module that uses any dynamic behavior such as going to a database or a provisioning system or anything else and rewrite the URL programmatically in a way that ARR will understand it and do the routing automatically.

Also, worth to mention that ARR has way more features than just this, making it possible to load-balance requests and many more interesting stuff that I will try to get back in a future post.

With all this you can imagine several benefits, such as single public end-point for remote management of multiple servers, only one valid certificate is needed facing public machines, you can relocate sites at your own will since customers will never really know the real machine name where their site lives, you can use a similar technique to rewrite even the Site Name and give them some friendly name such as their user name or whatever.

Acknowledgements: I want to thank Anil Ruia and Daniel Vasquez Lopez who helped figuring out a few issues during this blog and Ruslan Yakushev and Won Yoo for reviewing its technical accuracy.

After many years I decided that it is time to rewrite my Web site using Razor. A bit of history, I started it around 2003 using ASP.NET 1.1. When .NET 2.0 came around in 2005 I migrated to it and it was great being able to leverage features like MasterPages, Themes, Sitemaps, and many other features. Honestly it is a pretty simple Web site, with mostly content, so very few controls, Sitemap, my own custom Menu control, and a couple more. Last week it was moved to use .NET 4.0 and it feels its about time to go back and update it a bit, both in look and features. So this (if time permits) will be the first of a series of migration notes that I discover as I move it to use ASP.NET Razor (aka WebPages). Do note that this is not meant to be a best practice in anyway, I would never claim I can make such a thing, these will be only my personal notes as I discover more details in ASP.NET WebPages features and as I move my own implementation to use them.

So with that, one of the first things I faced during this migration, was the use of a Sitemap control (asp:SiteMapPath) in my MasterPage (future post about moving from MasterPages coming). I knew about Sitemap API, so I just decided to write a simple Sitemap helper that I can now use anywhere in Razor. The code is pretty simple, it basically generates an unordered list of links using <ul> and <li> with <a> inside, and used CSS to layout them in a way that I liked.

SitemapPath Control in WebForms

The original code I was using in my MasterPage looked like the following:

<asp:SiteMapPath CssClass="HeaderText" runat="server" ID="siteMap" ShowToolTips="true" NodeStyle-ForeColor="White" CurrentNodeStyle-Font-Bold="true" />

And generated the following markup:

<span id="siteMap" class="HeaderText"><a href="#siteMap_SkipLink"><img alt="Skip Navigation Links" height="0" width="0" src="http://blogs.msdn.com/WebResource.axd?d=S2jbW9E-HYlS0UQoRCcsm94KUJelFI6yS-CQIkFvzT6fyMF-zCI4oIF9bSrGjIv4IvVLF9liJbz7Om3voRpNZ8yQbW3z1KfqYr4e-0YYpXE1&amp;t=634219272564138624" style='border-width:0px;' /></a><span><a title='Home' href='/' style='color:White;'>Home</a></span><span> &gt; </span><span><a title='Free tools for download' href='/Tools/' style='color:White;'>Tools</a></span><span> &gt; </span><span style='color:White;font-weight:bold;'>Code Translator</span><a id='siteMap_SkipLink'></a></span>

Which looks like the following in the browser:

I used some CSS to set the color, and background and other stuff, but still to set the last item to bold required me to use a property in the Sitemap to get it to look the way I wanted.

My Sitemap Helper in Razor

Since I was familiar with the Sitemap API and my goal was to change as “little” as possible as part of this first migration, I decided to write a Sitemap helper that I can use in my Layout pages. The code in the Page is as simple as it gets, you just call @Helpers.Sitemap() and that’s it (added the Div below to get some context in the markup, but that was already there with the SitemapPath control anyway):

<div class="bannerPath">
@Helpers.Sitemap()
</div>

This new helper version generates the markup below. I don’t know about you, but I can sure make more sense of what it says, and I imagine Search Engines will as well, I decided to use more semantically correct markup using a <nav> to signal navigational section and use a list of links.

<nav>
    <ul class="siteMap">
        <li><a href="http://blogs.msdn.com/" title="Home">Home</a>&nbsp;&gt;&nbsp;</li>
        <li><a href="http://blogs.msdn.com/Tools/" title="Free tools for download">Tools</a>&nbsp;&gt;&nbsp;</li>
        <li><span>Code Translator</span></li>
    </ul>
</nav>

And it looks like the following in the browser (I decided to remove the underlining, and have more padding, and a new font, but all of that is CSS):

The Sitemap helper code

The code to do the Sitemap was pretty simple, just use the SiteMap API to get the current node. Since I’m picky and I wanted to generate the markup in the “right” order (note you could use CSS to float them to the right instead), I used a Stack to push the nodes while traversing them up. Finally just generate the <li>.

To make it look the way I wanted I used the following CSS:

.siteMap

  { float:right; font-size:11px; color:White; display:inline; margin-top:3px; margin-bottom:3px; margin-left:0px; margin-right:10px; } .siteMap li,span { float:left; list-style-type:none; padding-left:5px; border-width:0px;} .siteMap span { font-weight:bold; } .siteMap a,a.Visited { color:White; text-decoration:none; }

Conclusion

SitemapPath control gives you a really easy way to put together a navigation control based on the Sitemap APIs (and the Web.Sitemap file in my case). Creating a simple ASP.NET Razor helper is actually pretty easy since all the functionality needed is there in the base API’s and although it required some code (20 lines of code) now I feel like I have more control over my markup, can style it in anyway I want using CSS and have cleaner markup rendered.

I’m sure there are better ways to do this, but as I said, the goal of this first pass is to push my site soon with as little changes possible while keeping the same functionality first.

In the URL Rewrite forum somebody posted the question "are redirects bad for search engine optimization?". The answer is: not necessarily, Redirects are an important tool for Web sites and if used in the right context they actually are a required tool. But first a bit of background.

What is a Redirect?

A redirect in simple terms is a way for the server to indicate to a client (typically a browser) that a resource has moved and they do this by the use of an HTTP status code and a HTTP location header. There are different types of redirects but the most common ones used are:

• 301 - Moved Permanently. This type of redirect signals that the resource has permanently moved and that any further attempts to access it should be directed to the location specified in the header
• 302 - Redirect or Found. This type of redirect signals that the resource is temporarily located in a different location, but any further attempts to access the resource should still go to the same original location.

Below is an example on the response sent from the server when requesting http://www.microsoft.com/SQL/

So what do redirects mean for SEO?

One of the most important factors in SEO is the concept called organic linking, in simple words it means that your page gets extra points for every link that external Web sites have linking to your page. So now imagine the Search Engine Bot is crawling an external Web site and finds a link pointing to your page (example.com/some-page) and when it tries to visit your page it runs into a redirect to another location (say example.com/somepage). Now the Search Engine has to decide if it should add the original "some-page" into its index as well as if it should "add the extra points" to the new location or to the original location, or if it should just ignore it entirely. Well the answer is not that simple, but a simplification of it could be:

• if you return a 301 (Permanent Redirect) you are telling the search engine that the resource moved to a new location permanently so that all further traffic should be directed to that location. This clearly means that the search engine should ignore the original location (some-page) and index the new location (somepage), and that it should add all the "extra points" to it, as well as any further references to the original location should now be "treated" as if it was the new one.
• if you return a 302 (Temporary Redirect) the answer can depend on search engines, but its likely to decide to index the original location and ignore the new location at all (unless directly linked in other places) since its only temporary and it could at any given point stop redirecting and start serving the content from the original location. This of course makes it very ambiguous on how to deal with the "extra points" and likely will be added to the original location and not the new destination.

Enter IIS SEO Toolkit

IIS Search Optimization Toolkit has a couple of rules that look for different patterns related to Redirects. The Beta version includes the following:

1. The redirection did not include a location header. Believe it or not there are a couple of applications out there that does not generate a location header which completely breaks the model of redirection. So if your application is one of them, it will let you know.
2. The redirection response results in another redirection. In this case it detected that your page (A) is linking to another page (B) which caused a redirection to another page (C) which resulted in another redirection to yet another page (D). In this case it is trying to let you know that the number of redirects could significantly impact the SEO "bonus points" since the organic linking could be all broken by this jumping around and that you should consider just linking from (A) to (D) or whatever actual end page is supposed to be the final destination.
3. The page contains unnecessary redirects. In this case it detected that your page (A) is linking to another page (B) in your Web site that resulted in a redirect to another page (C) within your Web site. Note that this is an informational rule, since there are valid scenarios where you would want this behavior, such as when tracking page impressions, or login pages, etc. but in many cases you do not need them since we detect that you own the three pages we are suggesting to look and see if it wouldn't be better to just change the markup in (A) to point directly to (C) and avoid the (B) redirection entirely.
4. The page uses a refresh definition instead of using redirection. Finally related to redirection, IIS SEO will flag when it detects that the use of the refresh meta-tag is being used as a mean for causing a redirection. This is a practice that is not recommended since the use of this tag does not include any semantics for search engines on how to process the content and in many cases is actually consider to be a tactic to confuse search engines, but I won't go there.

So how does it look like? In the image below I ran Site Analysis against a Web site and it found a few of these violations (2 and 3).

Notice that when you double click the violations it will tell you the details as well as give you direct access to the related URL's so that you can look at the content and all the relevant information about them to make the decision. From that menu you can also look at which other pages are linking to the different pages involved as well as launch it in the browser if needed.

Similarly with all the other violations it tries to explain the reason it is being flagged as well as recommended actions to follow for each of them.

IIS Search Engine Optimization Toolkit can also help you find all the different types of redirects and the locations where they are being used in a very easy way, just select Content->Status Code Summary in the Dashboard view and you will see all the different HTTP Status codes received from your Web site. Notice in the image below how you can see the number of redirects (in this case 18 temporary redirects and 2 permanent redirects). You can also see how much content they accounted for, in this case about 2.5 kb (Note that I've seen Web sites generate a large amount of useless content in redirect traffic, speaking of spending in bandwidth). You can double click any of those rows and it will show you the details of the URL's that returned that and from there you can see who links to them, etc.

So what should I do?

1. Know your Web site. Run Site Analysis against your Web site and see all the different redirects that are happening.
2. Try to minimize redirections. If possible with the knowledge gain on 1, make sure to look for places where you can update your content to reduce the number of redirects.
3. Use the right redirect. Understand what is the intent of the redirection you are trying to do and make sure you are using the right semantics (is it permanent or temporary). Whenever possible prefer Permanent Redirects 301.
4. Use URL Rewrite to easily configure them. URL Rewrite allows you to configure a set of rules using both regular expressions and wildcards that live along with your application (no-administrative privileges required) that can let you set the right redirection status code. A must for SEO. More on this on a future blog.

Summary

So going back to the original question: "are redirects bad for Search Engine Optimization?". Not necessarily, they are an important tool used by Web application for many reasons such as:

• Canonicalization. Ensure that users are accessing your site with www. or without www. use permanent redirects
• Page impressions and analytics. Using temporary redirects to ensure that the original link is preserved and counters work as expected.
• Content reorganization. Whether you are changing your host due to a brand change or just renaming a page, you should make sure to use permanent redirects to keep your page rankings.
• etc

Just make sure you don't abuse them by having redirects to redirects, unnecessary redirects, infinite loops, and use the right semantics.

Well, today is my very first post ever.

Soon I hope to have time to start writing something that hopefully is interesting at all. Some of the first ideas I intend to dump here are about the tools that I posted in my site so that people trying to use them understand a little bit more about the problem I was trying to solve, the rationale behind them and the design issues I ran into and other stuff. Some of those tools include a Chart Control for ASP.NET, a library to generate Excel Xml Workbooks, C# and VB.NET Code translation, Code colorizer, and other random tools.

For now, this is just my meaningless boring post to say ‘hello' and thanks for the people who kept 'bugging' [:)] me to finally do this.

Today somebody was running the IIS SEO Toolkit and using the Site Analysis feature flagged a lot of violations about "The page contains multiple canonical formats.". The reason apparently is that he uses Query String parameters to pass contextual information or other information between pages. This of course yield the question: Does that mean in general query strings are bad news SEO wise?

Well, the answer is not necessarily.

I will start by clarifying that this violation in Site Analysis means that our algorithm detected that those two URL's look like the same content, note that we make no assumptions based on the URL (including Query String parameters). This kind of situation is bad for a couple of reasons:

1. Based on the fact they look like the same page Search Engines will probably choose one of them and index it as the real content and will discard the other one. The problem is that you are leaving this decision to Search Engines which means some might choose the wrong version and end up using the one with Query String parameters instead of the clean one (not-likely though). Or even worse they might end up indexing both of them as if they were different.
2. When other Web sites look at your content and add links to it, some of them might end up using the URL with different Query String parameters and some of them not. What this means is that the organic linking will not give you the benefits that you would if this was not the case. Remember Search Engines add you "extra" points when somebody external references your page but now you'll be splitting the earnings with "two pages" instead of a single canonical form.

Query String by themselves do not pose a terrible threat to SEO, most modern Search Engines deal OK with Query Strings, however its the organic linking and the potential abuse of Query Strings that could give you headaches.

Remember, Search Engines should make no assumptions based on the fact it is a single "page" that serves tons of content through a single Absulte Path and the use of Query Strings. This is typical in many cases such as when using index.php, where pretty much every page on the site is served by the same resource and just using variations of Query Strings or path information.

So what should I do?

Well, there are several things you could do, but probably one of the easiest is to just tell Search Engines (more specifically crawlers or bots) to not index pages that have the different Query String variations that really are meant only for the application to pass state and not to specify different content. This can be done using the Robots Exclusion Protocol and use the wildcard matching to specify to not follow any URL's that contain a '?'. Note that you should make sure you are not blocking URL's that actually are supposed to be indexed. For this you can use the Site Analysis feature to run it again and it will flag an informational message for each URL that is not visited due to the robots exclusion file.

In summary, try to keep canonical formats yourself, don't leave any guesses to Search Engines cause some of them might get it wrong. There are new ways of specifying the canonical form in your markup but it is "very recent" (as in 2009) and some Search Engines do not support it (I believe the top three do, though) using the new rel="canonical":

In the Beta 2 version of IIS SEO Toolkit we will support this tag and have better detection of this canonical issues. So stay tuned.

Other ways to solve this is to use URL Rewrite so that you can easily redirect or rewrite your URL's to get rid of the Query Strings and use more SEO friendly URL's.

Today I was playing a bit with Visual Studio 2008 and was surprised to see that I was not getting IntelliSense in my web.config. As you might already know IntelliSense in Xml in Visual Studio is implemented by using a set of schemas that are stored in a folder inside the VS folder, something like: \Program Files\Microsoft Visual Studio 9.0\Xml\Schemas. After looking to the files it was easy to understand what was going on, turns out I was developing using .NET 2.0 settings and Visual Studio now ships different schemas for Web.config files depending on the settings that you are using: DotNetConfig.xsd, DotNetConfig20.xsd and DotNetConfig30.xsd.

As I imagine I looked into the DotNetConfig.xsd and it indeed has all the definitions for the system.webServer sections as well as the DotNetConfig30.xsd. However, DotNetConfig20.xsd does not include the section details, only its definition, so to fix your IntelliSense you can just open DotNetConfig.xsd, select the entire section from:

<xs:element name="system.webServer" vs:help="configuration/system.webServer">...</xs:element>

and just replace the entry in DotNetConfig20.xsd. You might also want to copy the system.applicationHost section and add it to the DotNetConfig20.xsd since it does not include it as well.

During this PDC I attended Ian's presentation about WPF and Silverlight where he demonstrated the high degree of compatibility that can be achieved between a WPF desktop application and a Silverlight application. One of the differences that he demonstrated was when your application consumed Web Services since Silverlight applications execute in a sandboxed environment they are not allowed to call random Web Services or issue HTTP requests to servers that are not the originating server, or a server that exposes a cross-domain manifest stating that it is allowed to be called by clients from that domain.

Then he moved to show how you can work around this architectural difference by writing your own Web Service or HTTP end-point that basically gets the request from the client and using code on the server just calls the real Web Service. This way the client sees only the originating server and it allows the call to succeed, and the server can freely call the real Web Service. Funny enough while searching for a Quote Service I ran into an article from Dino Esposito in MSDN magazine  where he explains the same issue and also exposes a "Compatibility Layer" which again is just code (more than 40 lines of code) to act as proxy to call a Web Service (except he uses the JSON serializer to return the values).

The obvious disadvantage is that this means you have to write code that only forwards the request and returns the response acting essentially as a proxy. Of course this can be very simple, but if the Web Service you are trying to call has any degree of complexity where custom types are being sent around, or if you actually need to consume several methods exposed by it, then it quickly becomes a big maintenance nightmare trying to keep them in sync when they change and having to do error handling properly, as well as dealing with differences when reporting network issues, soap exceptions, http exceptions, etc.

So after looking at this, I immediately thought about ARR (Application Request Routing) which is a new extension for IIS 7.0 (see http://www.iis.net/extensions) that you can download for free from IIS.NET for Windows 2008, that among many other things is capable of doing this kind of routing without writing a single line of code.

This blog tries to show how easy it is to implement this using ARR. Here are the steps to try this: (below you can find the software required), note that if you are only interested in what is really new just go to 'Enter ARR' section below to see the configuration that fix the Web Service call.

1. Create a new Silverlight Project (linked to an IIS Web Site)
1. Launch Visual Web Developer from the Start Menu
2. File->Open Web Site->Local IIS->Default Web Site. Click Open
3. File->Add->New Project->Visual C#->Silverlight->Silverlight Application
4. Name:SampleClient, Locaiton:c:\Demo,  Click OK
5. On the "Add Silverlight Application" dialog choose the "Link this Silverlight control into an existing Web site", and choose the Web site in the combo box.
6. This will add a SampleClientTestPage.html to your Web site which we will run to test the application.
2. Find a Web Service to consume
1. In my case I searched using http://live.com for a Stock Quote Service which I found one at http://www.webservicex.net/stockquote.asmx
3. Back at our Silverlight project, add a Service Reference to the WSDL
1. Select the SampleClient project in the Solution Explorer window
2. Project->Add Service Reference and type http://www.webservicex.net/stockquote.asmx in the Address and click Go
3. Specify a friendly Namespace, in this case StockQuoteService
4. Click OK
4. Add a simple UI to call the Service
1. In the Page.xaml editor type the following code inside the <UserControl></UserControl> tags:
    <Grid x:Name="LayoutRoot" Background="Azure">
        <Grid.RowDefinitions>
            <RowDefinition Height="30" />
            <RowDefinition Height="*" />
        </Grid.RowDefinitions>
        <Grid.ColumnDefinitions>
            <ColumnDefinition Width="50" />
            <ColumnDefinition Width="*" />
            <ColumnDefinition Width="50" />
        </Grid.ColumnDefinitions>
        <TextBlock Grid.Column="0" Grid.Row="0" Text="Symbol:" />
        <TextBox Grid.Column="1" Grid.Row="0" x:Name="_symbolTextBox" />
        <Button Grid.Column="2" Grid.Row="0" Content="Go!" Click="Button_Click" />
        <ListBox Grid.Column="0" Grid.Row="1" x:Name="_resultsListBox" Grid.ColumnSpan="3"
                 ItemsSource="{Binding}">
            <ListBox.ItemTemplate>
                <DataTemplate>
                    <StackPanel Orientation="Horizontal">
                        <TextBlock Text="{Binding Path=Name}" FontWeight="Bold" Foreground="DarkBlue" />
                        <TextBlock Text=" = " />
                        <TextBlock Text="{Binding Path=Value}" />
                    </StackPanel>
                </DataTemplate>
            </ListBox.ItemTemplate>
        </ListBox>
    </Grid>
2. Right click the Button_Click text above and select the "Navigate to Event Handler" context menu.
3. Enter the following code to call the Web Service
    private void Button_Click(object sender, RoutedEventArgs e)
    {
        var service = new StockQuoteService.StockQuoteSoapClient();
        service.GetQuoteCompleted += service_GetQuoteCompleted;
        service.GetQuoteAsync(_symbolTextBox.Text);
    }
4. Now, since we are going to use XLINQ to parse the result of the Web Service which is an XML then we need to add the reference to System.Xml.Linq by using the Project->Add Reference->System.Xml.Linq.
5. Finally, add the following function to handle the result of the Web Service
    void service_GetQuoteCompleted(object sender, StockQuoteService.GetQuoteCompletedEventArgs e)
    {
        var el = System.Xml.Linq.XElement.Parse(e.Result);
        _resultsListBox.DataContext = el.Element("Stock").Elements();
    }
5. Compile the application. Build->Build Solution.
6. At this point we are ready to test our application, to run it just navigate to http://localhost/SampleClientTestPage.html or simply select the SampleClientTestPage.html in the Solution Explorer and click View In Browser.
7. Enter a Stock Symbol (say MSFT) and press Go!, Verify that it breaks. You will see a small "Error in page" with a Warning icon in the status bar. If you click that and select show details you will get a dialog with the following message:

Message: Unhandled Error in Silverlight 2 Application An exception occurred during the operation, making the result invalid. 

Enter Application Request Routing and IIS 7.0

1. Ok, so now we are running into the cross-domain issue, and unfortunately we don't have a cross-domain here is where ARR can help us call the service without writing more code
2. Modify the Web Service configuration to call a local Web Service instead
1. Back in Visual Web Developer, open the file ServiceReferences.ClientConfig
2. Modify the address="http://www.webservicex.net/stockquote.asmx" to be instead address="http://localhost/stockquote.asmx", it should look like:
    <client>
        <endpoint address="http://localhost/stockquote.asmx"
            binding="basicHttpBinding" bindingConfiguration="StockQuoteSoap"
            contract="StockQuoteService.StockQuoteSoap" name="StockQuoteSoap" />
    </client>
3. This will cause the client to call the Web Service in the same originating server, now we can configure ARR/URL Rewrite rule to route the Web Service requests to the original end-point
1. Add a new Web.config to the http://localhost project (Add new item->Web.config)
2. Add the following content:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <system.webServer>
        <rewrite>
            <rules>
                <rule name="Stock Quote Forward" stopProcessing="true">
                    <match url="^stockquote.asmx$" />
                    <action type="Rewrite" url="http://www.webservicex.net/stockquote.asmx" />
                </rule>
            </rules>
        </rewrite>
    </system.webServer>
</configuration>
4. This rule basically uses regular expression to match the requests for StockQuote.asmx and forwards them to the real Web Service.
5. Compile everything by running Build->Rebuild Solution
6. Back in your browser refresh the page to get the new, enter MSFT in the symbol and press Go!
7. And Voila!!! everything works.

Summary

One of the features offered by ARR is to provide proxy functionality to forward requests to another server. One of the scenarios where this functionality is useful is when using it from clients that cannot make calls directly to the real data, this includes Silverlight, Flash and AJAX applications. As shown in this blog, by just using a few lines of XML configuration you can enable clients to call services in other domains without having to write hundreds of lines of code for each method. It also means that I get the original data and that if the WSDL were to change I do not need to update any wrappers. Additionally if using REST based services you could use local caching in your server relying on Output Cache and increase the performance of your applications significantly (again with no code changes).

Software used

Here is the software I installed to do this sample(amazing that all of it is completely free):

1. Install Visual Web Developer 2008 Express
2. Install Silverlight Tools for Visual Studio 2008 SP 1
3. Install Application Request Routing for IIS 7.

One question that I've been asked several times is: "Is it possible to schedule the IIS SEO Toolkit to run automatically every night?". Other related questions are: "Can I automate the SEO Toolkit so that as part of my build process I'm able to catch regressions on my application?", or "Can I run it automatically after every check-in to my source control system to ensure no links are broken?", etc.

The good news is that the answer is YES!. The bad news is that you have to write a bit of code to be able to make it work. Basically the SEO Toolkit includes a Managed code API to be able to start the analysis just like the User Interface does, and you can call it from any application you want using Managed Code.

In this blog I will show you how to write a simple command application that will start a new analysis against the site provided in the command line argument and process a few queries after finishing.

IIS SEO Crawling APIs

The most important type included is a class called WebCrawler. This class takes care of all the process of driving the analysis. The following image shows this class and some of the related classes that you will need to use for this.

The WebCrawler class is initialized through the configuration specified in the CrawlerSettings. The WebCrawler class also contains two methods Start() and Stop() which starts the crawling process in a set of background threads. With the WebCrawler class you can also gain access to the CrawlerReport through the Report property. The CrawlerReport class represents the results (whether completed or in progress) of the crawling process. It has a method called GetUrls() that returns an instance to all the UrlInfo items. A UrlInfo is the most important class that represents a URL that has been downloaded and processed, it has all the metadata such as Title, Description, ContentLength, ContentType, and the set of Violations and Links that it includes.

Developing the Sample

1. Start Visual Studio.
2. Select the option "File->New Project"
3. In the "New Project" dialog select the template "Console Application", enter the name "SEORunner" and press OK.
4. Using the menu "Project->Add Reference" add a reference to the IIS SEO Toolkit Client assembly "c:\Program Files\Reference Assemblies\Microsoft\IIS\Microsoft.Web.Management.SEO.Client.dll".
5. Replace the code in the file Program.cs with the code shown below.
6. Build the Solution

If you are not using Visual Studio, you can just save the contents above in a file, call it SEORunner.cs and compile it using the command line:

C:\Windows\Microsoft.NET\Framework\v3.5\csc.exe /r:"c:\Program Files\Reference Assemblies\Microsoft\IIS\Microsoft.Web.Management.SEO.Client.dll" /optimize+ SEORunner.cs

After that you should be able to run SEORunner.exe and pass the URL of your site as a argument, you will see an output like:

Processed - Remaining - Download Size
       56 -       149 -      0.93 MB
      127 -       160 -      2.26 MB
      185 -       108 -      3.24 MB
      228 -        72 -      4.16 MB
      254 -        48 -      4.98 MB
      277 -        36 -      5.36 MB
      295 -        52 -      6.57 MB
      323 -        25 -      7.53 MB
      340 -         9 -      8.05 MB
      358 -         1 -      8.62 MB
      362 -         0 -      8.81 MB
Crawling complete!!!

----------------------------
 Overview
----------------------------
Start URL:  http://www.carlosag.net/
Start Time: 11/16/2009 12:16:04 AM
End Time:   11/16/2009 12:16:15 AM
URLs:       362
Links:      3463
Violations: 838

----------------------------
 Status Code summary
----------------------------
                  OK -   319
    MovedPermanently -    17
               Found -    23
            NotFound -     2
 InternalServerError -     1

----------------------------
 Broken links
----------------------------
http://www.carlosag.net/downloads/ExcelSamples.zip

The most interesting method above is RunAnalysis, it creates a new instance of the CrawlerSettings and specifies the start URL. Note that it also specifies that we should consider internal all the pages that are hosted in the same directory or subdirectories. We also set the a unique name for the report and use the same directory as the IIS SEO UI uses so that opening IIS Manager will show the reports just as if they were generated by it. Then we finally call Start() which will start the number of worker threads specified in the WebCrawler::WorkerCount property. We finally just wait for the WebCrawler to be done by querying the IsRunning property.

The remaining methods just leverage LINQ to perform a few queries to output things like a report aggregating all the URLs processed by Status code and more.

Summary

As you can see the IIS SEO Toolkit crawling APIs allow you to easily write your own application to start the analysis against your Web site which can be easily integrated with the Windows Task Scheduler or your own scripts or build system to easily allow for continuous integration.

Once the report is saved locally it can then be opened using IIS Manager and continue further analysis as with any other report. This sample console application can be scheduled using the Windows Task Scheduler so that it can run every night or at any time. Note that you could also write a few lines of PowerShell to automate it without the need of writing C# code and do that by only command line, but that is left for another post.

One of my favorites features in the IIS Search Engine Optimization (SEO) Toolkit is what we called Report Comparison. Report Comparison basically allows you to compare two different versions of the results of crawling the same site to see what changed in between. This is a really convenient way to track not only changes in terms of SEO violations but also to be able to compare any attributes on the pages such as Title, Heading, Description, Links, Violations, etc.

How to access the feature

There are a couple of ways to get to this feature.

1) Use the Compare Reports task. While in the Site Analysis Reports listing you can select two reports by using Ctrl+Click, and if both reports are compatible (e.g. they use the same Start URL) the task "Compare Reports" will be shown. Just clicking on that will get you the comparison.

2) Use the Compare to another report menu item. While in the Dashboard view of a Report you can use the "Report->Compare To Another Report" menu item which will show a dialog where you can either select an existing report or even start a new analysis to compare with.

Report Comparison Page

In both cases you will get the Report Comparison Page displaying the results as shown in the next image.

The Report Comparison page includes a couple of "sections" with data. At the very top it includes links showing the Name and the Date when the reports were ran. If you click on them it will open the report directly just as if you had used the Site Analysis report listing view.

The next sections shows a lot of interesting built-in data such as:

Whenever you click the links you get a query dialog that you can customize just as any Query in the Query builder, where you can Add/Remove columns, add filters, etc.

My favorite one is the "Modified URLs" source when you actually can add filters that compare URLs coming from the two different reports.

Note that when you double click or "right-click –> Compare Details" any of the rows you get a side-by-side comparison of everything in the URL:

Again, you can use any of the tabs to see side-by-side things like the Content of the pages or the Links both versions have or the violations, or pretty much everything that you can see for just one.

Finally, you can also right click on the Query dialog and choose "Compare Contents". This will launch whatever File Comparison tool you have configured using the "Edit Feature Settings". In this case I have configured WinDiff.exe which shows something like:

Summary

As you can see Report Comparison offers is a powerful feature that allows you to keep track of changes between two different reports. This easily allows you to understand over time how your site has been affected by changes. For Site managers it will allow them to query and maintain a history with all the changes. You can imagine that using an automated build process that runs IIS SEO Toolkit crawling whenever a build is made that keeps the report stored somewhere and potentially annotate it with the build number you could even keep a correlation of changes in code with Web site crawling.

Well, I’m really excited to be heading to PDC’05, it sure promise to be an interesting one where we can finally show off so many new technologies we have been working so hard during this past few months to create.

I will be there along with many of my teammates from IIS trying to show off the really cool new features that we have built into the next version of IIS which I'm sure will rock your mind if you are a Web lover like me.

So I really hope to meet a lot of you guys out there and try to answer any questions you might have.