June, 2008

Posts
  • CarlosAg Blog

    IIS 7.0 Site ID Computation

    • 1 Comments

    In my previous post I mentioned that IIS Manager (more specifically Microsoft.Web.Administration) has two algorithms for assigning a Site ID when no ID is specified.

    The two algorithms are:

    1. Incremental Site ID. When this algorithm is configured we will assign the first consecutive number available, for example, if you have Site ID's 1,2,3,5,6,8 the next time you add a Site in the UI (or using ServerManager.Sites.Add friendly overloads), you will get Site ID = 4, next will be 7, and next will be 9. The side effect of this is that you will get different Site ID's depending on the sites that are available which might complicate things in certain scenarios when provisioning the same site in multiple machines. The good news is that in IIS 7.0 the Side ID is not so important in terms of the configuration system since now all the configuration is stored using the Site Name instead. (Exceptions apply, for example when using the Metabase compatibility (customMetadata) or when you are using things like Session State in ASP.NET)
    2. Site Name Hash. In this case we will calculate a Hash of the Site Name and use that number for the Site ID. This will end up using some funky numbers that will be impossible to remember but that "can work" in different machines. Technically we use the .NET implementation of "String.GetHashCode()" which you can read the remarks "The behavior of GetHashCode is dependent on its implementation, which might change from one version of the common language runtime to another. A reason why this might happen is to improve the performance of GetHashCode."

    In either case we will ensure that whatever ID we generate it will not collide with any existing Site.

    These algorithms are configured using the following registry key:

    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\InetMgr\Parameters

    In there you will find a DWORD attribute called IncrementalSiteIDCreation when set to 1 we will use the Incremental algorithm (1), otherwise we will use the Site name hash.

    Bottom line if you really care about the Site ID and are provisioning the sites using Microsoft.Web.Administration you are better off having your own deterministsic algorithm for assigning the Site ID yourself and it will actually be considerably faster.

    If you are using the UI chances are you really don't care about the Site ID, cause you would never have to interact with it (again might need changes when using certain ASP.NET features).

  • CarlosAg Blog

    Faster IIS Web Sites Provisioning using Microsoft Web Administration

    • 6 Comments

    Yesterday I got an email about some performance numbers that one of our customers were running into when creating remotely Web Sites, Applications, Application Pools and other tasks in IIS using Microsoft.Web.Administration. In case you don't know Microsoft.Web.Administration is a .NET library that exposes the IIS Configuration System.

    The code was using Microsoft.Web.Administration from PowerShell to create a single Application Pool, a new Web Site, and finally assign the Root Application to the new Application Pool, and it was measuring each of the tasks. The application then was called continually till 10,000 sites were created. They then created a very nice Excel Spreadsheet that had a chart with how long it took when performing each operation that clearly show how fast (or slow) things were depending on the existing number of sites in IIS.

    Funny enough the code was doing almost exactly what we use to measure in IIS 7.0 performance while provisioning. And that we ran into very similar numbers back a couple of years ago but we probably never share them. The results back in the day were showing a trend like the one below, where the Creation of the AppPool was quite fast (less than 400ms once you had 10,000 sites), the Application Pool assignment was taking almost 10 times more (almost 4 seconds) and the creation of the Sites was taking almost 20 times more (up to 8 seconds).

    SampleData

    Our test code looked something like this (removing all the Timing code):

    using(ServerManager m = new ServerManager()) {
       
    ApplicationPool pool = m.ApplicationPools.Add("MyPool");

       
    Site site = m.Sites.Add("MySite", ... "d:\inetpub\wwwroot");
       
    site.Applications["/"].AppPoolName = "MyPool";
        
       
    m.CommitChanges();
    }

    Back in those days (2005) the numbers were even worst, you would have to wait literally minutes to get a site created but we profiled it and understood exactly what the performance behavior was and we made it better.

    Here is how to make it faster

    1. Most importantly batch as many changes you need to do to our configuration system so that you only CommitChanges once, for example, creating the 10,000 sites and only Committing once will probably average 1,000 sites per second. More importantly you will reduce the load on the server since every time the configuration is changed our server has to process its changes and that incurs in other CPU cycles in WAS and W3SVC.
    2. The reason we noticed Application Pools was being slower at the beginning when using ServerManager locally was just an artifact of the fact that it’s the first operation being done in our Test Script. The reason is that when you create a ServerManager in reality nothing interesting happens other than we activate a COM object that parses the schema, however whenever you request a section that is when the real XML parsing of files happens. This means that the first operation (in this case accessing the AppPool collection) pays the price of parsing machine.config, root web.config and ApplicationHost.config, which is the file that grows considerably as new sites/pools are added.
    3. Definitely using ServerManager remotely (OpenRemote) will be several times slower because every time property is set or anything is changed the code actually hits the network to set the values in the remote objects, since on the client there are only client proxies. This means that setting appPool.Name actually goes through the network to set the values on the dllhost.exe hosting DCOM.
    4. Finally the most important one, the reason the Site creation is extremely slow is because the way our test code was creating the site and the application, in particular we are using the method m.Sites.Add() which has several overloads, where most of them provide a set of "friendly-non-performant” combinations. This is due to the fact that none of them receives a Site ID which forces us to automatically calculate one every time a Site will be added. We have implemented two algorithms for this (depending on a registry key called IncrementalSiteIDCreation) and both have the issue that we need to read all the sites first to make sure we don’t generate an ID that is already in use. This means that if you have 4,000 sites we have to read all 4,000 sites and that of course over the network translates to literally thousands of round trips.
      So bottom line use the m.Sites.CreateElement() followed by a m.Sites.Add(newSite) which will not create an ID for the Site that will not incur in the costs mentioned. Something like the snippet below. Notice that there is a little bit more code since you also need to create the Root Application and set the bindings, but this will proof useful since setting the AppPoolName requires the App anyway and you save all the Lookups on the Sites list as well.
    5. using(ServerManager m = new ServerManager()) {
         
      ApplicationPool pool = m.ApplicationPools.Add("MyPool");

         
      Site site = m.Sites.CreateElement("site");
         
      site.Name = "MySite"
         
      site.ID = "MySite".GetHashCode();

         
      Application app = site.Applications.Add("/", "d:\inetp...");
         
      app.AppPoolName = "MyPool";
          
         
      m.CommitChanges();
      }

    After all that story, what is important to realize is that there are ways to make this way faster and that you should use them if you are going to do any sort of massive creation of sites. With the two changes outlined above (Specify the Site ID yourself and don't lookup the Site again to get the App) the results were incredible we were back to under 350ms for all the tasks.

  • CarlosAg Blog

    Extending the IIS Configuration System using COM

    • 3 Comments

    Today I was going to post about extending the IIS Configuration, in particular about a feature that not everybody knows that allows you to extend the IIS Configuration System using dynamic code. What this means is that instead of hard-coding the configuration using XML in a .config file, your configuration can be provided by a COM object that implements IAppHostPropertyExtension, IAppHostElementExtension and IAppHostMethodExtension.

    Then, just to make sure I was not repeating what somebody else already said I searched for this in live.com (Worth to say, excellent results, first hit is the documentation of the interface, second hit is an excellent article in iis.net).

    So instead of repeating what you can already find in those places in IIS.NET I decided to not blog about it in details, but instead mention some of the things that are not specified in these places.

    This dynamic configuration is great and offers lots of interesting features since it allows you to expose any random code that can immediately be accessed through all of our configuration API's, including Microsoft.Web.Administration, AHADMIN, etc, giving your end-user a common programming paradigm, in fact this also means that its immediately accessible to the UI API's and even to the new Configuration Editor in the Admin Pack.

    Another interesting benefit is that through these API's your code can be called remotely so that it can be scripted to manage the machines remotely without the need to write any serialization or complex remote infrastructure (restrictions might apply).

    However, one thing that is also important to mention is that these dynamic configuration extensions are only available for administration tools, meaning you cannot access this extensions from the worker process by default. To clarify, you cannot use the Worker Process configuration instance to invoke these extensions since the worker process specifically disables the ability to call them in its configuration instance. However, if you create your own instance of Microsoft.Web.Administration.ServerManager (which requires you to be running in Full Trust) you will be able. You can also create your own instance of Microsoft.ApplicationHost.AdminManager and you will be able to access them. However in both cases this will only work if your an Administrator in the machine or have read ACL's for ApplicationHost.config file (which by default is only readable by Administrators). This is why methods like Microsoft.Web.Administration.WebConfiigrationManager::GetSection (and CoGetObject for AHADMIN) are provided so you don't run into these issues when developing Web Applications and are still able to read configuration sections for your worker process without requiring administrative privileges (in MWA provided you are either are in Full Trust or the section definition marks it as requirePermission=false).

    To understand better some scenarios its worth to mention that In IIS 7.0 we actually use these API's to provide access to runtime information in an easy way and other tasks, for example, to query the state of a Site, to Recycle an Application Pool, to assign an SSL certificate to a binding, to stop a Site, are all provided through this mechanism. If you want to see all the things we do this way just open %windir%\System32\Inetsrv\config\schema\rscaext.xml where all of our Web Server extensions are declared. Our own FTP Server for IIS 7.0 uses the same mechanism for things like querying Sessions, and other cool stuff.

    Anyway, feel free to give the IIS.NET article a good read, its quite good.

    http://www.iis.net/articles/view.aspx/IIS7/Extending-IIS7/Extending-IIS7-Configuration/Configuration-Extensibility

  • CarlosAg Blog

    IIS Admin Pack: Database Manager

    • 2 Comments

    OK, this blog entry sat in my drafts for more than a month now, but finally today I will be talking about another cool feature of the IIS Admin Pack, Database Manager.

    Database Manager allows you to easily manage a SQL Server Database from within the IIS Manager User Interface. Now, why would you need another UI for managing a database? Here are some of the facts that make Database Manager interesting:

    1. No additional components are required in the machine, just by installing the IIS Admin Pack you get a UI with support for remote management, and many more features (with TP2 it will install SQL Server Management Objects).
    2. Allows you to manage both local and remote database from your machine.
    3. By using the Remote Management capabilities of IIS provides a clean firewall-friendly option for managing a remote SQL Server.
    4. Exposes a public extensibility platform with support for other databases.

    Features

    The version included in the Admin Pack Technical Preview 2 includes the following features:

    1. Add/Rename/Drop and Edit Tables
    2. Primary Key/Indexes and Foreign Keys management
    3. Query Analyzer like function
    4. Data Editing
    5. View/Drop Store Procedures

    The only permissions you need to get all the functionality in the database are "db_databreader, db_datawriter and db_ddladmin", where db_ddladmin is only required if the schema manipulation such as adding tables, deleting tables, etc, is needed.

    Database Manager automatically reads the list of Connection Strings stored in the configuration system of the selected object, for example a specific site's config, or a specific application, and they will be automatically pre-loaded in the Database Manager connections tree view where you can just expand any of them and you will get the tables and all the functionality. This option makes it really convenient for querying data and allows server administrators to maintain control on what databases can be managed using this feature making it a really nice option for developers building Data Driven applications. It additionally allows users to specify their own connections that are not stored in configuration files and still get all the functionality.

    This means that if you are like many of the web sites running in hosted environments (or are a hoster that provisions a single user/database in advance for users) and you only get a single database user for accessing the data as well as doing all the database maintenance, then you can live with the connection strings approach. This provides a convenient way to provide delegation in a secure way, by adding the connectionString in the root web.config with a locationPath for that particular web site and being able to lock them prevents them from uploading new connectionStrings that could be just trying to fish for users/passwords in your server. As a Server Administrator, you can enforce this functionality by specifying allowClientConnectionStrings="false" in administration.config.

    Remote Administration

    Arguably what makes IIS Database Manager really interesting is that it uses the IIS Manager Remote Administration infrastructure, allowing it to work remotely using nothing but HTTPS to communicate to the server. This way the server does not require to open any additional ports, all the communication happens over the IIS Remote Management HTTPS protocol. It also means that the server is the only one that requires connectivity with the Database Server, making this ideal for hosting environments because the Clients does not require any software installed on their machines, no SQL libraries or anything, just IIS Manager which can be run in Windows Server 2008, Windows Vista, Windows 2003 and Windows XP.

    Note that when using IIS Manager locally such as when using Windows Vista or Windows 2008 to manage "localhost" then IIS Manager will connect directly to SQL so no additional jump through HTTPS.

    To try to explain this using a picture:

     Database Manager Options

    Extensibility

    Another very interesting feature of Database Manager is that it was conceived as an easy way to expose different databases (not only SQL Server) to Web Developers, IT-Pro's and Hosters. This means that we created it following a Provider model where all you need to do is implement a single class in C# called DatabaseProvider, register it in the Server and now you can manage your own database. We have implemented the SQL Server functionality on top of this provider model and are working on other providers that you should soon hear from.

    Andrew Birck, who is a Developer in my team wrote a very nice blog entry on how to extend the functionality. To keep it simple he implemented a DataSet provider that allows you to manage an XML as if it was a real database, allowing you to use the same features (Create Table, Edit Data, etc), make sure to check it out at: http://blogs.msdn.com/andrewbi/archive/2008/05/21/writing-a-databaseprovider.aspx

Page 1 of 1 (4 items)