While creating the new administration stack in IIS 7, we were looking into the different ways users could manipulate the server configuration as well as the new runtime information available in IIS 7 (Internally we call this RSCA-Runtime State and Control API) from managed code, and we realized we needed to provide a simpler and more straight forward API that developers could consume from managed code. Microsoft.Web.Administration is the answer to this problem. This API is designed to be simple to code against in an “intellisense-driven” sort of way. At the root level a class called ServerManager exposes all the functionality you will need.

To show the power and simplicity of this API, let’s look at some samples below. To try this samples just create a new Console Application in Visual Studio and add a reference to Microsoft.Web.Administration.dll that can be found at IIS directory (%WinDir%\System32\InetSrv).
Please note that the following code is based on Windows Vista Beta 2 code and will likely change for the release candidate versions of Windows Vista since we have planned several enhancements to simplify the API and expose more features into it.
The following picture shows the main objects (excluding Configuration related classes).
 

Creating a Site
 
This basically creates an instance of the ServerManager class and uses the Add method in the Sites collection to create a new site named "NewSite" listening at port 8080 using http protocol and content files are at d:\MySite.
One thing to note is that calling Update is a requirement since that is the moment when we persist the changes to the configuration store.
After running this code you have now a site that you can browse using http://localhost:8080

Adding an Application to a site
Creating a Virtual Directory

Runtime State and Control

Now, moving on to the new Runtime state and control information we also expose in this objects information about their current state as well as the ability to modify them. For example, we expose the list of W3WP processes running (Worker processes) and what I think is really cool, we even expose the list of requests currently running. Stopping a Web Site
Recyciling an Application Pool
Getting the list of executing requests Another big thing on this API is the ability to edit the “.config” files using a simple API, this includes the ability of modifying the main applicationHost.config file from IIS, web.config files from asp.net as well as machine.config and other config files (such as administration.config). However I will talk about them in a future post.

Today I was planning on talking about the configuration classes that I purposedly skipped in my last post, but I realized it would be better to explain a little bit more about the new configuration system used in IIS 7.

First of all, many of you (as me) will be extremely happy to know that the old "monolithic-centralized-admin only" metabase is dead, we have got rid of it for a much better configuration store. Now, before you feel panic, let me assure you that we haven’t just killed it and forget about the thousands of lines of scripts or custom tools built using the old metabase API’s (such as ABO), for that we have created something we called ABOMapper which will allow all of those applications to keep running transparently, since it will auto-magically translate the old calls to the metabase to actually modify the new configuration system.

So what is this new configuration system? Well for those of you who have been working with ASP.NET for the past years, you will feel right at home and happy to know that we are moving to used the exact same concept as ASP.NET does .config files.

ApplicationHost.config

At the root level we have a file called ApplicationHost.config that lives in the same directory of IIS (typically <windows>\System32\InetSrv\ directory). This is the main configuration file for IIS, this is where we store things like the list of sites, applications, virtual directories, general settings, logging, caching, etc.

This file has two main groups of settings:

• system.applicationHost: Contains all the settings for the activation service, basically things like the list of application pools, the logging settings, the listeners and the sites. These settings are centralized and can only be defined within applicationHost.config.
• system.webServer: Contains all the settings for the Web server, such as the list of modules and isapi filters, asp, cgi and others. These settings can be set in applicationHost.config as well as any web.config (provided the Override Mode settings are set to allow)

Administration.config

This is also a file located in the IIS directory where we store delegation settings for the UI, including the list of modules (think of it as a UI Add-in) available, and other things like administrators.

Web.config

Finally the same old web.config from asp.net has gotten smarter and now you will be able to include server settings along with your asp.net settings.

Why is this important?

Well, as I said at the beginning the old metabase could only be accessed by administrators, so in order for someone to change a settings as simple as the default document for a specific application (say you want to change it to be index.aspx), you would need to be an administrator or call the administrator to do the changes.
With this new distributed configuration system I can now safely modify the web.config within my application and have it my own way without disturbing anyone else. Furthermore, since it lives in my own web.config along with the content of my application I can safely XCopy the whole application and now even the web server settings are ready. No longer the case of going to InetMgr and start setting everything manually or creating a bunch of scripts to do that.

So how does this actually looks like:

In applicationHost.config my Sites section looks as follows: This basically defines a site that has a root application with a virtual directory that points to \inetpub\wwwroot. This site is listening on any IP address on port 80.
Say I wanted to add a new application and make it listen also in port 8080. Just by adding the previous markup, I can now browse to http://localhost:8080/MyApp
IIS Settings in web.config
More interesting I can now add a file called web.config to c:\MyApp\web.config, and set the content to be: And with this change, my application now will respond using index.aspx whenever /MyApp is requested.
You can extrapolate from this that all the IIS settings for your application including authentication, authorization, asp and cgi settings, the list of modules, custom errors, etc can be configured within your web.config and never have to request changes to administrators again.

Of course this brings the question, isn’t this insecure? The answer is no, by default all the IIS sections (except DefaultDocuments) is locked at the applicationHost.config, meaning no one can change them within their web.config unless explicitly changed by the administrator. The cool thing is that the administrator can change it and customize it per application allowing certain apps to change settings while preventing others from doing it. All this can be done through plain config using Notepad or using the very cool NEW InetMgr (which I will blog about it later)
Finally, the following image shows the hierarchy of config files for each url.

Now that I have shown a high level overview of how configuration works in IIS 7, I will finally blog about the API to actually change this settings programmatically using Managed code and Microsoft.Web.Administration.dll

In the past few days I've been reading a bit about SEO and trying to understand more about what makes a Web Site be SEO (Search-Engine-Optimized) and what are some of the typical headaches when trying to achieve that as well as how we can implement them in IIS.

Today I decided to post how you can make your Web Site running IIS 7.0 a bit "friendlier" to Search Engines without having to modify any code in your application. Being SEO is a big statement since it can include several things, so for now I will scope the discussion to 3 things that can be easily addressed using the IIS URL Rewrite Module:

1. Canonicalization
2. Friendly URL's
3. Site Reorganization

1) Canonicalization

Basically the goal of canonicalization is to ensure that the content of a page is only exposed as a unique URI. The reason this is important is because even though for humans it's easy to tell that http://www.carlosag.net is the same as http://carlosag.net, many search engines will not make any assumptions and keep them as two separate entries, potentially splitting the rankings of them lowering their relevance. Another example of this is http://www.carlosag.net/default.aspx and http://www.carlosag.net/. You can certainly minimize the impact of this by writing your application using the canonical forms of your links, for example in your links you can always link to the right content for example: http://www.carlosag.net/tools/webchart/ and remove the default.aspx, however that only accounts for part of the equation since you cannot assume everyone referencing your Web Site will follow this carefully, you cannot control their links.

This is when URL Rewrite comes into play and truly solves this problem.

Host name.

URL Rewrite can help you redirect when the users type your URL in a way you don't unnecessarily want them to, for example just carlosag.net. Choosing between using WWW or not is a matter of taste but once you choose one you should ensure that you guide everyone to the right one. The following rule will automatically redirect everyone using just carlosag.net to www.carlosag.net. This configuration can be saved in the Web.config file in the root of your Web Site.Note that I'm only including the XML in this blog, however I used IIS Manager to generate all of these settings so you don't need to memorize the XML schema since the UI includes several friendly capabilities to generate all of these..

Note that one important thing is to use Permanent redirects (301) , this will ensure that if anybody links your page using a non-WWW link when the search engine bot crawls their Web Site it will identify the link as permanently moved and it will treat the new URL as the correct address and it will not index the old URL, which is the case when using Temporary (302) redirects. The following shows how the response of the server looks like:

Default Documents

IIS has a feature called Default Document that allows you to specify the content that should be processed when a user enters a URL that is mapped to a directory and not an actual file. In other words, if the user enters http://www.carlosag.net/tools/ then they will actually get the content as if they entered http://www.carlosag.net/tools/default.aspx. That is all great, the problem is that this feature only works one way by mapping a Directory to a File, however it does not map the File to the Document, this means that if some of your links or other users enter the full URL, then search engines will see two different URL's. To solve that problem we can use a configuration very similar to the rule above, following is a rule that will redirect the default.aspx to the canonical URL (the folder).

This again, uses a Permanent redirect to extract everything before Default.aspx and redirect it to the "parent" URL path, so for example, if the user enters http://www.carlosag.net/Tools/WindowsLiveWriter/default.aspx it will be redirected to http://www.carlosag.net/Tools/WindowsLiveWriter/ as well as http://www.carlosag.net/Tools/default.aspx to http://www.carlosag.net/Tools/. You can place this rule at the root of your site and it will take care of all the default documents (if you have a default.aspx in every folder)

2) Friendly URL's

Asking your user to remember that www.contoso.com/books.aspx?isbn=0735624410 is the URL for the IIS Resource Kit is not the nicest thing to do, first of all why do they care about this being an ASPX and the fact that it takes arguments and what not. It seems that providing them with a URL like www.contoso.com/books/IISResourceKit will truly resonate with them and be easier for them to remember and pass along. Most importantly it really doesn't tie you to any Web technology.

With URL Rewrite you can easily build this kind of logic automatically without having to modify your code using Rewrite Maps:

The configuration above includes a rule that uses a Rewrite Map to translate a URL like: http://www.contoso.com/books/IISResourceKit into http://www.contoso.com/books.aspx?isbn=0735624410 automatically. Using maps is a very convenient way to have a "table" of values that can be transformed into any other value to be used in the result URL. Of course there are better ways of doing this when using large catalogs or values that change frequently but is extremely useful when you have a consistent set of values or when you can't make changes to an existing application. Note that since we use Rewrite the end users never see the "ugly-URL" unless they knew it already and typed it, and of course this means you can use the inverse approach to ensure the canonicalization is preserved:

The rule above does the "inverse" by matching the URL books.aspx, extracting the ISBN query string value and doing a lookup in the ISBN table and redirecting the client to the canonical URL, so again if user enters http://www.contoso.com/books.aspx?isbn=0735624410 they will be redirected to http://www.contoso.com/books/IISResourceKit.

This Friendly URL to me is more of a user feature than a SEO feature, however I've read in every SEO guide to reduce the number of parameters in your Query String, however, I have not find yet any document that clearly states if there is truly a limit in the search engine bot's that would truly impact the search relevance. I guess it makes sense that they wouldn't keep track of thousands of links to a catalog.aspx that has zillions of permutations based on hundreds of values in the query string (category, department, price range, etc) even if all of them were linked, but again I don't have any prove.

3) Site Reorganization

One complex tasks that Web Developers face sometimes is trying to reorganize their current Web Site structure, whether its moving a section to a different path, or something as simple as renaming a single file, you need to take into consideration things like, Is this move a temporary thing?, How do I ensure old clients get the new URL?, How do I prevent losing the search engine relevance?. URL Rewrite will help you perform these tasks.

Rename a file

If you rename a file you can very easily just write a Rewrite or Redirect Rule that ensures that your users continue getting the content. If your intent is to never go back to the old name you should use a Redirect Permanent so everyone starts getting the new content with its new "Canonical URL", however, if this could be a temporary thing you should use a Redirect Temporary. Finally a Rewrite is useful if you still want both URL's to continue to be valid (though this breaks the canonicality).

Moving directories

Another common scenario is when you need to move an entire directory to another place of the Web Site. It could also be that based on some criteria (say Mobile browsers or other User Agent) get a different set of pages/images. Either way, URL rewrite helps with this. The following configuration will redirect every call to the /Images directory to the /NewImages directory.

A related scenario is if you wanted to show different smaller images whenever a user of Windows CE was accessing your site, you could have a "img" directory where all the small images are stored and use a rule like the following:

Note, that in this case the use of Rewrite makes sense since we want the small images to look as the original images to the browser and it will save a "round-trip" to it.

Moving multiple files

Another common operation is when you randomly need to relocate pages for whatever reason (such as Marketing Campaigns, Branding, etc). In this case if you have several files that have been moved or renamed you can have a single rule that catches all of those and redirects them accordingly. Similarly, another sample could include an incremental migration from one technology to another where say you are moving from Classic ASP to ASP.NET and as you rewrite some of the old ASP pages into ASPX pages you want to start serving them without breaking any links or the search engine relevance.

Now, you can just keep adding to this table any broken link and specify its new address.

Others

Other potential use of URL Rewrite is when using RIA applications in the browser, whether using things like AJAX, Silverlight or Flash, that are not easy to parse and index by search engines, you could use URL Rewrite to rewrite the URL to static HTML versions of your content, however you should make sure that the content is consistent so you don't misguide users and search engines. For example the following rule will rewrite all the files in the RIAFiles table to their static HTML counterpart but only if the User Agent is the MSNBot or the GoogleBot:

Related to this is that you might want to prevent search engines from crawling certain files (or your entire site), for that, you can use the Robots.txt semantics and use a "disallow", however, you can also use URL Rewrite to prevent this with more functionality such as blocking only a specific user agent:

There are several other things you can do to ensure that your Web Site is friendly with Search Engines, however most of them require changes to your application, but certainly worth the effort, for example:

• Ensure your HTML includes a <title> tag.
• Ensure your HTML includes a <meta name="description".
• Use the correct HTML semantics, use H1 once and only once, use the alt attribute in your <img>, use <noscript> etc.
• Redirect using status code 301 and not 302.
• Provide Site Map's and/or Robots.txt.
• Beware of POST backs and links that require script to run. 

Resources

For this entry I read and used some of the resources at several Web Sites, including:

• http://blogs.msdn.com/webmaster/
• http://webmaster.live.com/
• http://www.robotstxt.org/
• http://www.sitemaps.org/

Today there was a question in the IIS.net Forums asking how to expose two different Internet sites from another site making them look like if they were subdirectories in the main site.

So for example the goal was to have a site: www.site.com expose a www.site.com/company1  and a www.site.com/company2 and have the content from “www.company1.com” served for the first one and “www.company2.com” served in the second one. Furthermore we would like to have the responses cached in the server for performance reasons. The following image shows a simple diagram of this:

 

This sounds easy since its just about routing or proxying every single request to the correct servers, right? Wrong!!! If it only it was that easy. Turns out the most challenging thing is that in this case we are modifying the structure of the underlying URLs and the original layout in the servers which makes relative paths break and of course images, Stylesheets (css), javascripts and other resources are not shown correctly.

To try to clarify this, imagine that a user requests using his browser the page at http://www.site.com/company1/default.aspx, and so based on the specification above the request is proxied/routed to http://www.company1.com/default.aspx on the server-side. So far so good, however, imagine that the markup returned by this HTML turns out to have an image tag like “<img src=/some-image.png />”, well the problem is that now the browser will resolve that relative path using the base path on the original request he made which was http://www.site.com/company1/default.aspx resulting in a request for the image at http://www.site.com/some-image.png instead of the right “company1” folder that would be http://www.site.com/company1/some-image.png .

Do you see it? Basically the problem is that any relative path or for that matter absolute paths as well need to be translated to the new URL structure imposed by the original goal.

So how do we do it then?

Enter URL Rewrite 2.0 and Application Request Routing

URL Rewrite 2.0 includes the ability to rewrite the content of a response as it is getting served back to the client which will allow us to rewrite those links without having to touch the actual application.

Software Required:

• IIS 7.0 or IIS 7.5 (included in Windows Vista, Windows 2008, Windows 7 or Windows 2008 R2).
• Install URL Rewrite 2.0
• Install Application Request Routing

Steps

1. The first thing you need to do is enable Proxy support in ARR.
1. To do that just launch IIS Manager and click the server node in the tree view.
2. Double click the “Application Request Routing Cache” icon
3. Select the “Server Proxy Settings…” task in the Actions panel
4. And Make sure that “Enable Proxy” checkbox is marked. What this will do is allow any request in the server that is rewritten to a server that is not the local machine will be routed to the right place automatically without any further configuration.
2. Configure URL Rewrite to route the right folders and their requests to the right site. But rather than bothering you with UI steps I will show you the configuration and then explain step by step what each piece is doing.
3. Note that for this post I will only take care of Company1, but you can imagine the same steps apply for Company2, and to test this you can just save the configuration file below as web.config and save it in your inetpub\wwwroot\  or in any other site root and you can test it.

Setup the Routing

                <rule name="Route the requests for Company1" stopProcessing="true">
                    <match url="^company1/(.*)" />
                    <conditions>
                        <add input="{CACHE_URL}" pattern="^(https?)://" />
                    </conditions>
                    <action type="Rewrite" url="{C:1}://www.company1.com/{R:1}" />
                    <serverVariables>
                        <set name="HTTP_ACCEPT_ENCODING" value="" />
                    </serverVariables>
                </rule>

The first rule is an inbound rewrite rule that basically captures all the requests to the root folder /company1/*, so if using Default Web Site, anything going to http://localhost/company1/* will be matched by this rule and it will rewrite it to www.company1.com respecting the HTTP vs HTTPS traffic.

One thing to highlight which is what took me a bit of time is the “serverVariables” entry in that rule that basically is overwriting the Accept-Encoding header, the reason I do this is because if you do not remove that header then the response will likely be compressed (Gzip or deflate) and Output Rewriting is not supported on that case, and you will end up with an error message like:

HTTP Error 500.52 - URL Rewrite Module Error.
Outbound rewrite rules cannot be applied when the content of the HTTP response is encoded ("gzip").

Also note that to be able to use this feature for security reasons you need to explicitly enable this by allowing the server variable. See enabling server variables here.

 

Outbound Rewriting to fix the Links

The last two rules just rewrite the links and scripts and other resources so that the URLs are translated to the right structure. The first one rewrites absolute paths, and the last one rewrites the relative paths. Note that if you use relative paths using “..” this will not work, but you can easily fix the rule above, I was too lazy to do that and since I never use those when I create a site it works for me :)

Setting up Caching for ARR

A huge added value of using ARR is that now we can with a couple of clicks enable disk caching so that the requests are cached locally in the www.site.com, so that not every single request ends up paying the price to go to the backend servers.

1. To do that just launch IIS Manager and click the server node in the tree view.
2. Double click the “Application Request Routing Cache” icon
3. Select the “Add Drive…” task in the Actions panel.
4. Specify a directory where you want to keep your cache. Note that this can be any subfolder in your system.
5. Make sure that “Enable Disk Cache” checkbox is marked in the Server Proxy Settings mentioned above.

As easy as that now you will see caching working and your site will act as a container of other servers in the internet. Pretty cool hah! :)

So in this post we saw how with literally few lines of XML, URL Rewrite and ARR we were able to enable a proxy/routing scenario with the ability to rewrite links and furthermore with caching support.

Why I wrote Excel Xml Writer
One day I found myself having to build a Web Application that one of the requirements involved generating a nice Excel Workbook that users could then play with. This required quite some styling and several other features that you can only do with Excel (like setting printer options and document properties).
 
Well, coming from Consulting, this requirement was no strange at all to me, and I had to dealt with this problem many times years ago. However then, it was a different story, Managed code and Xml were not even created, and COM was the only option I had. The only solution was using Excel Automation to build Workbooks that supported all the features I required. Yes, I know HTML could do the trick to just generate a table and set the content type to an Excel Application but this certainly leaves you lacking of control in several Excel features like document properties (Author, custom properties, etc), printer settings and more.
 
Excel Automation
If you ever worked with Excel Automation you know that it is an extremely powerful (and complicated) object model. However, this power does not come for free, every time you create an Excel.Application, you are essentially running a new Excel.exe instance which is nothing but cheap, and for that reason (and many more) you certainly do not want to do that in a Web Application where thousands of users might run the nice “Export to Excel” link and you end up with thousand of processes being created and destroyed.
 
Just to illustrate my point, I created the following sample.
 
C#

---

Colorized by: CarlosAg.CodeColorizer Note: If you want to run the application you need to add a Reference to Microsoft Excel COM library.
 
Well, I ran this really simple application cold in my machine that has Office 2003, and running it took almost 3000 milliseconds. Of course if you run it again it runs in about 1 second. But this solution will just not scale in a Web Application.
 
Another big problem with this approach is the code itself, if you take a close look I had to type almost 20 references to Missing.Value.
 
Solution
Luckily ever since Office XP, Excel supports a new file format called Xml Workbook (or something like that), this allow you to create an Xml document that follows a certain schema and Excel will work as if it were the binary XLS format (though not all features at supported like Charts).
 
Now I have new options, I could just generate the Xml using an XmlDocument or even better using an XmlWriter; but doing so it is quite cumbersome, since you need to understand a lot of Xml, Schemas, and Namespaces and it is quite probably that you will mess up something like closing an element incorrectly or adding the wrong namespace, or prefix, etc.
 
For that reason I thought to build a lightweight fast wrapper to the Excel Xml Workbook schema. This way my application manipulates an object model that looks similar to Excel Automation OM but it is lightweight, 100% managed, and that in the end serialize itself into an Xml using an XmlWriter.
 
This is exactly what Excel Xml Writer is, just a simple object model that generates Xml following the Excel Xml Workbook schema. After almost done with it, I thought I could add the ability to load the Xml as well, so I added that feature. This turned out to be extremely useful when loading Excel worksheets from the Office Web Components, and really cool usage, so that you can embed workbooks in your page, and then use AJAX like technology to post the XMLData property back and load it in the server side to do the actual processing in your Database, etc.
 
You can download it for free at http://www.carlosag.net/Tools/ExcelXmlWriter/Default.aspx.
 
Now, the code to generate the same workbook we just did using my library the code looks like:
 
 

---

Colorized by: CarlosAg.CodeColorizer

Several differences:
1)      You don’t actually need Excel installed in your server to run this program since it does not uses Excel at all, just Xml.
2)      Working set of your application is way smaller than using the Interop libraries
3)      This is more than 100 times faster to run.
4)      Code looks much more simpler.
5)      Since it generates Xml, you can actually stream it directly in an ASP.NET application to the Response.OutputStream without ever saving it into the file system.
6)      This solution will scale to thousands of users since it does not require any creation of processes.
 
Now, even better I decided to write a code generator tool so that you don’t need to write all the styling code and superficial stuff and just focus on the actual data. This tool allows you to open an Excel Xml Workbook that you have created in Excel, and it will generate the C# or VB.NET code that you can use to generate it. This means that you can create the skeleton with all the formatting options in Excel and then just generate the code.
 
 
Conclusion
Don’t get me wrong Excel team did an awesome job with the Automation support for all Office products, however this has been around for several years, but it definitely lacks of support for Web based applications (asside from Office Web Components). Luckily they are addressing this in the next release of Office where they will have some awesome support for server side applications and many exciting stuff.
 
In the mean time you might find really exciting working with Xml in Office since they have great support and it will only get better with time.

The other day I was asked if I knew about a tool that would allow users to easily analyze the IIS Log Files, to process and look for specific data that could easily be automated. My recommendation was that if they were comfortable with using a SQL-like language that they should use Log Parser. Log Parser is a very powerful tool that provides a generic SQL-like language on top of many types of data like IIS Logs, Event Viewer entries, XML files, CSV files, File System and others; and it allows you to export the result of the queries to many output formats such as CSV (Comma-Separated Values, etc), XML, SQL Server, Charts and others; and it works well with IIS 5, 6, 7 and 7.5.

To use it you just need to install it and use the LogParser.exe that is found in its installation directory (on my x64 machine it is located at: C:\Program Files (x86)\Log Parser 2.2).

I also thought on sharing some of my favorite queries. To run them, just execute LogParser.exe and make sure to specify that the input is an IIS Log file (-i:W3C) and for ease of use in this case we will export to a CSV file that can be then opened in Excel (-o:CSV) for further analysis:

Purpose	Query	Sample Output
Number of Hits per Client IP, including a Reverse DNS lookup (SLOW)	SELECT c-ip As Machine,          REVERSEDNS(c-ip) As Name,          COUNT(*) As Hits   FROM c:\inetpub\logs\LogFiles\W3SVC1\*   GROUP BY Machine ORDER BY Hits DESC	Machine Name Hits ::1 CARLOSAGDEV 57 127.0.0.1 MACHINE1 28 127.X.X.X MACHINE2 1
Top 25 File Types	SELECT TOP 25      EXTRACT_EXTENSION(cs-uri-stem) As Extension,      COUNT(*) As Hits  FROM c:\inetpub\logs\LogFiles\W3SVC1\*  GROUP BY Extension  ORDER BY Hits DESC	Extension Hits gif 52127 bmp 20377 axd 10321 txt 460 htm 362
Top 25 URLs	SELECT TOP 25      cs-uri-stem as Url,      COUNT(*) As Hits  FROM c:\inetpub\logs\LogFiles\W3SVC1\*  GROUP BY cs-uri-stem  ORDER By Hits DESC	Url Hits /WebResource.axd 10318 /favicon.ico 8523 /Tools/CodeTranslator/Translate.ashx 6519 /App_Themes/Silver/carlosag.css 5898 /images/arrow.gif 5720
Number of hits per hour for the month of March	SELECT      QUANTIZE(TO_LOCALTIME(TO_TIMESTAMP(date, time)), 3600) AS Hour,      COUNT(*) AS Hits  FROM c:\inetpub\logs\LogFiles\W3SVC1\*  WHERE date>'2010-03-01' and date<'2010-04-01'  Group By Hour	Hour   Hits 3/3/2010 10:00:00 33 3/3/2010 11:00:00 5 3/3/2010 12:00:00 3
Number of hits per Method (GET, POST, etc)	SELECT      cs-method As Method,      COUNT(*) As Hits  FROM c:\inetpub\logs\LogFiles\W3SVC1\*  GROUP BY Method	Method Hits GET 133566 POST 10901 HEAD 568 OPTIONS 11 PROPFIND 18
Number of requests made by user	SELECT TOP 25      cs-username As User,      COUNT(*) as Hits  FROM c:\inetpub\logs\LogFiles\W3SVC1\*  WHERE User Is Not Null  GROUP BY User	User Count Administrator 566 Guest 1
Extract Values from Query String (d and t) and use them for Aggregation	SELECT TOP 25      EXTRACT_VALUE(cs-uri-query,'d') as Query_D,      EXTRACT_VALUE(cs-uri-query,'t') as Query_T,      COUNT(*) As Hits  FROM c:\inetpub\logs\LogFiles\W3SVC1\*  WHERE Query_D IS NOT NULL  GROUP BY Query_D, Query_T  ORDER By Hits DESC	Query_D Query_T Hits Value in Query1 Value in T1 1556 Value in Query2 Value in T2 938 Value in Query3 Value in T3 877 Value in Query4 Value in T4 768
Find the Slowest 25 URLs (in average) in the site	SELECT TOP 25      cs-uri-stem as URL,      MAX(time-taken) As Max,      MIN(time-taken) As Min,      Avg(time-taken) As Average  FROM c:\inetpub\logs\LogFiles\W3SVC1\*  GROUP BY URL  ORDER By Average DESC	URL Max Min Average /Test/Default.aspx 23215 23215 23215 /WebSite/Default.aspx 5757 2752 4178 /Remote2008.jpg 3510 3510 3510 /wordpress/ 6541 2 3271 /RemoteVista.jpg 3314 2 1658
List the count of each Status and Substatus code	SELECT TOP 25      STRCAT(TO_STRING(sc-status),      STRCAT('.', TO_STRING(sc-substatus))) As Status,      COUNT(*) AS Hits  FROM c:\inetpub\logs\LogFiles\W3SVC1\*  GROUP BY Status  ORDER BY Status ASC	Status Hits 200 144 304 38 400 9 403.14 10 404 64 404.3 2 500.19 23
List all the requests by user agent	SELECT      cs(User-Agent) As UserAgent,      COUNT(*) as Hits  FROM c:\inetpub\logs\LogFiles\W3SVC1\*  GROUP BY UserAgent  ORDER BY Hits DESC	UserAgent Hits iisbot/1.0+(+http://www.iis.net/iisbot.html) 104 Mozilla/4.0+(compatible;+MSIE+8.0;… 77 Microsoft-WebDAV-MiniRedir/6.1.7600 23 DavClnt 1
List all the Win32 Error codes that have been logged	SELECT      sc-win32-status As Win32-Status,      WIN32_ERROR_DESCRIPTION(sc-win32-status) as Description,      COUNT(*) AS Hits  FROM c:\inetpub\logs\LogFiles\W3SVC1\*  WHERE Win32-Status<>0  GROUP BY Win32-Status  ORDER BY Win32-Status ASC	Win32-Status Description Hits 2 The system cannot find the file specified. 64 13 The data is invalid. 9 50 The request is not supported. 2

A final note: any time you deal with Date and Time, remember to use the TO_LOCALTIME function to convert the log times to your local time, otherwise you will find it very confusing when your entries seem to be reported incorrectly.

If you need any help you can always visit the Log Parser Forums to find more information or ask specific questions.

Any other useful queries I missed?

NOTE: IIS Admin Pack Technical Preview 2 has been released: http://blogs.msdn.com/carlosag/archive/2008/05/13/IISAdminPackTechnicalPreview2Released.aspx

I'm really exited to announce that today we released the Technical Preview of the IIS Admin Pack and it includes 7 new features for IIS Manager that will help you in a bunch of different scenarios.

Download

You can download the IIS 7.0 Admin Pack Technical Preview from (It requires less than 1MB):

(x86) http://www.iis.net/downloads/default.aspx?tabid=34&g=6&i=1646
(x64) http://www.iis.net/downloads/default.aspx?tabid=34&g=6&i=1647

Documentation

http://learn.iis.net/page.aspx/401/using-the-administration-pack/

These UI modules include the following features:

• Request Filtering UI - This UI exposes the configuration of the IIS runtime feature called Request Filtering.
• Configuration Editor UI - This UI provides an advanced generic configuration editor entirely driven by our configuration schema. It includes things like Script Generation, Search functionality, advanced information such as locking and much more.
• Database Manager UI - This UI allows you to manage SQL Server databases from within IIS Manager, including the ability to create tables, execute queries, add indexes, primary keys, query data, insert rows, delete rows, and much more.
• IIS Reports UI - This extensible platform exposes a set of reports including some log parser based reports, displaying things like Top URL's, Hits per User, Page Performance, and many more.
• FastCGI UI - This UI exposes the configuration for the FastCGI runtime feature.
• ASP.NET Authorization UI - This UI allows you to configure the ASP.NET authorization settings.
• ASP.NET Custom Errors UI - This UI allows you to configure the Custom errors functionality of ASP.NET

Please, help us, we want to ask for your help on trying them and give us feedback of all these modules, do they work for you? what would you change? what would you add? What features are we missing?

Some things to think about,

Database Manager, what other database features are critical for you to build applications?

IIS Reports set of reports, what reports would you find useful?, would you want to have Configuration based reports (such as summarizing the Sites and their configuration, which configuration)? More Security Reports (such as)?

Configuration Editor, is it easy to use?, what concepts from configuration would you like to see?, etc

Given that each individual feature above has a lot of interesting features that can easily be missed, or might be confusing, I will be blogging in the near feature talking about why we decided to build each feature, what makes them different from any other thing you've seen as well as how you can make the most out of each of them.

NOTE: IIS Admin Pack Technical Preview 2 has been released: http://blogs.msdn.com/carlosag/archive/2008/05/13/IISAdminPackTechnicalPreview2Released.aspx

Carlos

Last Wednesday we released the IIS Manager 7.0 client for Windows XP SP2, Windows Server 2003 and Windows Vista SP1.
This is basically the IIS 7.0 Manager GUI that provides the ability to connect remotely to a Windows Server 2008 running the Web Management Service (WMSVC) to manage IIS 7.0 remotely.

There are several key differences in this version of IIS Manager and its remote infrastructure:

1) It allows for the first time users without administrative privileges to connect and manage their web sites and applications remotely

2) It runs over SSL, no more DCOM, which makes this a firewall friendly feature easy to setup.

3) Runs as a smart client, which means if a new feature is installed on the server it will automatically download the updated versions to the client machines.

You can download it from:

IIS.NET Web Site
x86:
http://www.iis.net/downloads/default.aspx?tabid=34&i=1626&g=6

x64:
http://www.iis.net/downloads/default.aspx?tabid=34&i=1633&g=6

Microsoft.com/Downloads
http://www.microsoft.com/downloads/details.aspx?FamilyID=32c54c37-7530-4fc0-bd20-177a3e5330b7&displaylang=en

To learn more about remote management and how to install it:
http://learn.iis.net/page.aspx/159/configuring-remote-administration-and-feature-delegation-in-iis-7/

Now, to really show you what this is, I created a very simple demo that briefly shows the remote management capabilities over SSL. (Below there is a transcript in case my accent makes it difficult to understand my english :))

Transcript:

The purpose of this demonstration is to show you how easy it is to manage IIS 7.0 running in Windows Server 2008, from any machine that has Windows XP or Windows 2003 or Windows Vista by downloading the IIS Manager 7.0 that runs on all of those platforms.

Now, today I am not going to focus on the details of how to configure it and how to setup the server to support remote management, but mainly just focus on the client aspect.

On of the most interesting aspects of this remote management infrastructure is that it now uses an architecture that uses HTTPS to communicate to the server making this a nice firewall friendly remote management feature. Another key feature of this functionality is that it allows users without administrative privileges to connect and manage their Web Sites or their applications in a delegated way, where an administrator can restrict which options they can modify or not.

OK, so to show you this I have here a Windows Server 2008 installed with IIS 7.0, and as you would expect I can manage it locally quite easily using IIS Manager, whether its adding a Web Site or managing the configuration from both IIS or ASP.NET I can do it here.

This is all good, but now turns out I don’t want to connect locally but instead be able to remotely from my development machine connect to the server and still be able to do that and have the same experience as if I was locally logged on to the machine.

To show this, I have here a Virtual PC image running a clean install of Windows XP SP2, the only thing it has installed additionally is the .NET Framework 2.0 which is the only requirement for the installation of IIS Manager 7.

I have already downloaded the IIS Manager installer which takes only about 3MB of disk, that you can find at http://www. iis.net or http://Microsoft.com/downloads.

Installing it is really simple and fast, just double click the icon and click next…

Once installed I can now connect to any machine running Windows Server 2008 that has been configured to support remote management. To do that I just need to choose the option “Connect To Server/Site/Application” from the File Menu or the Start Page.

Today, I will not drill down on the multiple differences between these connections, so for now I will just show how you can connect and manage the entire server by using a Windows Administrator account.

Another interesting feature of the remote management platform is that if some new feature built on top of the UI Management extensibility API is installed on the server, when I connect again to the server, it will automatically prompt me if I want to get the new functionliaty and I can choose which features to install or not.

To summarize, the IIS Manager 7 for Windows XP SP2, 2003 SP1 and Vista SP1 is available now, it only depends on the .NET FX 2.0 and it will allow you to connect to a remote server to manage it and have the same rich experience as if you were locally but using its new SSL remoting architecture.

IIS 7.0 includes a very cool feature that is not so well known called Hostable WebCore (HWC). This feature basically allows you to host the entire IIS functionality within your own process. This gives you the power to implement scenarios where you can customize entirely the functionality that you want "your Web Server" to expose, as well as control the lifetime of it without impacting any other application running on the site. This provides a very nice model for automating tests that need to run inside IIS in a more controlled environment. 

This feature is implemented in a DLL called hwebcore.dll, that exports two simple methods:

1. WebCoreActivate. This method allows you to start the server. It receives three arguments, out of which the most important one is applicationHostConfigPath that allows you to point it to your very own copy of ApplicationHost.config where you can customize the list of modules, the list of handlers and any other settings that you want your "in-proccess-IIS" to use. Just as ApplicationHost.config you can also specify the "root web.config" that you want your server to use, giving you the ability to completely run an isolated copy of IIS.
2. WebCoreShutdown. This method basically stops the server listening.

The real trick for this feature is to know exactly what you want to support and "craft" the IIS Server configuration needed for different workloads and scenarios, for example:

1. Static Files Web Server - Supporting only static file downloads, good for HTML scenarios and other simple sites.
2. Dynamic Web Sites
   1. ASPX Pages
   2. WCF
   3. Custom set of Http Modules and Handlers
3. All of the above

An interesting thing to mention is that the file passed to ApplicationHostConfigPath parameter is live, in the sense that if you change the configuration settings your "in-process-IIS" will pick up the changes and apply them as you would expect to. In fact even web.config's in the site content or folder directories will be live and you'll get the same behavior.

Sample

To show how easy this can be done I wrote a small simple class to be able to run it easily from managed code. To consume this, you just have to do something like:

This will start your very own "copy" of IIS running in your own process, this means that you can control which features are available as well as the site and applications inside it without messing with the local state of the machine.

A very interesting thing is that it will even run without administrator privileges, meaning any user in the machine can start this program and have a "web server" of their own, that they can recycle, start and stop at their own will. (Note that this non-administrative feature requires Vista SP1 or Windows Server 2008, and it only works if the binding will be a local binding, meaning no request from outside the machine).

You can download the entire sample which includes two configurations: 1) one that runs only an anonymous static file web server that can only download HTML and other static files, and 2) one that is able to run ASP.NET pages as well.

 Download the entire sample source code (9 kb)

You might be asking why would I even care to have my own IIS in my executable and not just use the real one? Well there are several scenarios for this:

1. Probably one of the most useful, as mentioned above this actually allows non-administrators to be able to develop applications that they can debug, change configuration and pretty much do anything without interfering with the machine state.
2. Another scenario might include something like a "Demo/Trial CD" where you can package your application in a CD/DVD that users then can insert in their machine and suddenly get a running/live demo of your Web Application without requiring them to install anything or define new applications in their real Web Server.
3. Test Driven Development. Testing in the real Web Server tends to interfere with the machine state which is by definition something you don't want in your test environments, ideally you want your tests to be done in an isolated environment that is fully under control and that you will not have to do any manual configuration. This makes this feature an ideal candidate for such scenario where you own the configuration and can "hard-code" it as part of your automated tests. No more code for "preparing the server and site", everything starts pre-configured.
4. Build your own service. You can build your own service and use Hostable WebCore as a simple yet powerful alternative to things like HttpListener, where you will be able to execute Managed and Native code Http Modules and Handlers easily without you having to do any custom hosting for ASP.NET infrastructure.
5. Have your own Development Web Server where you can have advance interaction between both the client and the server and trace and provide live debugging information.
6. many, many more...

In future posts I intent to share more samples that showcase some of this cool stuff.

Summary

IIS 7.0 Hostable WebCore feature allows you to host a "copy" of IIS in your own process. This is not your average "HttpListener" kind of solution where you will need to implement all the functionality for File downloads, Basic/Windows/Anonymous Authentication, Caching, Cgi, ASP, ASP.NET, Web Services, or anything else you need; Hostable WebCore will allow you to configure and extend in almost any way the functionality of your own Web Server without having to build any code.

I was running out of disk space in C: and was unable to install a small software that I needed, so I decided to clean up a bit. For that I like using WinDirStat http://windirstat.info/ which very quickly allows you to find where the big files/folders are. In this case I found that my c:\Windows\winsxs folder was over 12 GB of size. One way to reclaim some of that disk space is to cleanup all files that have been backed up when a Service Pack has been installed. To do that in Windows 7 you can run the following DISM command:

dism /online /cleanup-image /spsuperseded /hidesp

That freed up 4 GB in my machine and now I can move on.

Disclaimer: I only ran this in my Windows 7 machine and it worked great, have not tried it in Server SKUs so run at your own risk.