Robert McMurray's Blog [MSFT]

Using URL Rewrite with Web Site Testing

robert_mcmurray — Tue, 13 Oct 2009 22:49:00 GMT

Recently I ran into an interesting situation when I was rolling out a new web-based application. I had tested it on my development server, and because it was a rather complex application I wanted to test it on my production server before making it live. I had already set up a placeholder web site with a home page that announced the new site as "Coming Soon," and I had some of the supporting infrastructure configured for the placeholder site: database connections, permissions, FTP bindings, etc.

In order to test the new site, I could have set up a temporary web site by duplicating the placeholder web site, running my tests, and then deleting the temporary site after I was comfortable that everything was working in production. Or I could simply remove the placeholder web site and replace it with the temporary web site once the testing phase was over. I didn't like either of those ideas, so I came up with much easier solution using URL Rewrite. Here are the details:

First, I added an additional host header binding to the placeholder web site with a temporary name that only I knew about. Since I use a wildcard A record for my DNS, I can add any additional prefixes to my domain name without registering those as CNAME records in DNS. (This makes the temporary name more difficult to discover.) For example, if I created an A record for "*.example.com," then I could use "temp.example.com" without any changes to DNS.

Second, I added a URL Rewrite rule that checked all inbound requests by using the following logic:

If the request was for "temp.example.com", then URL Rewrite would allow regular access the web site.
If the request was for any other domain name and any web page other than "ComingSoon.htm", then URL Rewrite would redirect those requests to "/ComingSoon.htm".

Here's what the URL Rewrite rule looked like in my web.config file:

<system.webServer>
  <rewrite>
    <rules>
      <clear />
      <rule name="Temporary Redirect"
          patternSyntax="Wildcard"
          stopProcessing="true">
        <match url="ComingSoon.htm"
          negate="true" />
        <conditions>
          <add input="{HTTP_HOST}"
            negate="true"
            pattern="temp.example.com" />
        </conditions>
        <action type="Redirect"
          url="/ComingSoon.htm"
          appendQueryString="false"
          redirectType="Found" />
      </rule>
    </rules>
  </rewrite>
</system.webServer>

This allowed me to test the web site on my production server by using a temporary domain name that no one else knew about, and sending every other HTTP request to the "Coming Soon" announcement page. After a few days of putting the web site through some test passes, all that I needed to do was to remove the URL Rewrite rule and the site was officially live.

Migrating FPSE Roles Between Servers

robert_mcmurray — Mon, 12 Oct 2009 17:35:00 GMT

I ran into one of those strange situations the other day where you feel like you've been doing the same thing on your computer at some time in the distant past - kind of like déjà vu for geeks. In this specific case, I was moving some web sites that I am hosting for other people that still use FPSE from an older physical server to a new virtual server that is hosted through Hyper-V. (I'm also trying to convert them all to WebDAV, but that's another story.)

Anyway, I had dozens of custom FPSE roles set up for each of those sites that I didn't want to manually replicate on the new server. Unfortunately, FPSE doesn't have a way to migrate the roles from one server to another. All of those FPSE-related roles are kept in local groups with cryptic names like OWS_nnnnn_xxxxx, so I started thinking, "If only I could write a script that could migrate the OWS_nnnnn_xxxxx groups between the two servers..."

Then it dawned on me - I had written such a script several years ago! (Now if only I could find it...) Like many people that write code, I'm something of a code packrat - I tend to keep all of my old code around somewhere, just in case. Sparing you the details of my long search, I eventually found the script that I was looking for, and I thought that it would make a nice blog because I'm sure that someone else may need to migrate their FPSE roles.

Here's the script and a brief description of what script it will do:

Create ADSI objects for the source and destination servers.
Loop through the ADSI objects and only looks for groups.
- Note: You could also use an object.Filter statement for this.
- You will obviously need to update the server names for your source and destination servers.
Compare each group name with the group stub and only process those groups that match the stub.
- By way of explanation, FPSE role groups have names like OWS_nnnnn_xxxxx, where nnnnn is a simple numeric hash that identifies the site, and the xxxxx denotes the individual FPSE role like admin, browser, author, etc.
- The script uses the OWS_nnnnn stub and will copy all of the role groups that it finds. For example: OWS_12345_admin, OWS_12345_author, OWS_12345_browser, etc.
- In the code I used OWS_nnnnn for the stub, so you would have to replace nnnnn with the numbers that you see in the list of groups using Computer Management on the source computer.
Determine if the group exists on the destination server, and creates it if it doesn't already exist.
Loop through the list of users in the group on the source server and adds those same users to the group on the destination server.
- Note: This will fail for any local users that were used on the source server that do not exist on the destination server. I only used domain accounts, so I didn't run into that problem.

Some additional notes:

The person that runs this code must be an administrator on each server.
The two computers must be on the same network or the ADSI calls will fail.

With that in mind, here's the script code:

Option Explicit
On Error Resume Next

Dim objComputer1, objGroup1
Dim objComputer2, objGroup2

Dim objOBJECT
Dim objUSER
Dim strGroupName
Dim strGroupDesc

Const ERROR_SUCCESS = 0

' Note: update the following server names.
Const strComputer1 = "SERVER1"
Const strComputer2 = "SERVER2"

' Note: update the following group stub.
Const strGroupStub = "OWS_nnnnn"

' ----------------------------------------------------------------------

Set objComputer1 = GetObject("WinNT://" & strComputer1 & ",computer")
If CLng(Err.Number) <> ERROR_SUCCESS Then HandleError
Set objComputer2 = GetObject("WinNT://" & strComputer2 & ",computer")
If CLng(Err.Number) <> ERROR_SUCCESS Then HandleError

For Each objOBJECT in objComputer1
  If UCase(objOBJECT.Class) = "GROUP" Then
    strGroupName = objOBJECT.Name

    If UCase(Left(strGroupName,Len(strGroupStub))) = UCase(strGroupStub) Then

      Err.Clear : Set objGroup1 = GetObject("WinNT://" & strComputer1 & "/" & strGroupName & ",group")
      If CLng(Err.Number) <> ERROR_SUCCESS Then HandleError

      strGroupDesc = objGroup1.Description
      WScript.Echo "Copying " & strGroupName & "..."

      Err.Clear : Set objGroup2 = GetObject("WinNT://" & strComputer2 & "/" & strGroupName & ",group")
      
      If CLng(Err.Number) <> ERROR_SUCCESS Then
        If CLng(Err.Number) <> -2147022676 Then
          HandleError
        Else
          Err.Clear : Set objGroup2 = objComputer2.Create("group",strGroupName)
          If CLng(Err.Number) <> ERROR_SUCCESS Then HandleError

          Err.Clear : objGroup2.SetInfo
          If CLng(Err.Number) <> ERROR_SUCCESS Then HandleError

          Err.Clear : objGroup2.Description = strGroupDesc
          If CLng(Err.Number) <> ERROR_SUCCESS Then HandleError

          Err.Clear : objGroup2.SetInfo
          If CLng(Err.Number) <> ERROR_SUCCESS Then HandleError
        End If
      End If

      For Each objUSER in objGroup1.Members
        WScript.Echo vbTab & "Adding " & objUSER.Name
        Err.Clear : objGroup2.Add objUSER.ADsPath 
        If CLng(Err.Number) <> ERROR_SUCCESS And CLng(Err.Number) <> -2147023518 Then HandleError
        Err.Clear : objGroup2.SetInfo
        If CLng(Err.Number) <> ERROR_SUCCESS Then HandleError
      Next

      Set objGroup1 = Nothing
      Set objGroup2 = Nothing
      
    End If
  End If
Next

' ----------------------------------------------------------------------

Sub HandleError()
  WScript.Echo vbCrLf & "FATAL ERROR:"
  WScript.Echo vbTab & "Number: " & CLng(Err.Number) & " (0x" & Hex(CLng(Err.Number)) & ")"
  WScript.Echo vbTab & "Description: " & Err.Description
  WScript.Quit
End Sub

As usual, all of the normal caveats like "this code is totally unsupported" will apply, but I've used this code with great success on several severs over the years. The great thing about this code is that it's non-destructive because it doesn't delete anything on the source server - it only creates groups on the destination server.

You can also use Const strGroupStub = "OWS_" to migrate all FPSE role groups from one server to another, but that's a major operation. With that in mind, I'd try it out on a single FPSE site using "OWS_nnnnn" as a stub before trying out a full server by using "OWS_" as a stub; the latter is very time-consuming and CPU-intensive.

Creating a Microsoft Access Provider for IIS Database Manager

robert_mcmurray — Sat, 26 Sep 2009 01:22:00 GMT

Following up on my last blog post about the API set for the IIS Database Manager, I have something of a secret to let you in on - you can use the code samples in several of those API documents to create a fully functional provider for Microsoft Access databases. I would never use an Access database in a production environment, but having an Access provider has had some great benefits for me from a test perspective. I often use Access databases for test projects, and using the IIS Database Manager to manage the Access databases on my test systems means that I don't need to install Microsoft Access on any of my test servers.

That being said, as I was writing the API documentation I needed to create something of value to test my code samples. Since the Database Manager feature team was already creating database providers for SQL Server and MySQL, it seemed to me like Microsoft Access was the only other readily-accessible database that I could use for my samples. I mentioned in my last post that Saad Ladki was the Program Manager for Database Manager; Saad had started work on an Access provider at one point, but he abandoned the provider as his schedule grew tighter, so I took over that project so I could use it for the API samples.

With that in mind, here's what you need to do to create a Microsoft Access provider for the IIS Database Manager:

Create a new class project named "AccessDatabase" in Visual Studio
Add project references for:
- System.Configuration
- Microsoft.Web.Management.DatabaseManager
  Note: This may require adding a reference path for your project, such as "C:\Windows\assembly\GAC_MSIL\Microsoft.Web.Management.DatabaseManager\1.0.1.0__31bf3856ad364e35", where "C:" is your operating system drive.

Open the class in the editor, remove the existing code, and insert the following empty class:

using System;
using System.Collections;
using System.Collections.Generic;
using System.Configuration.Provider;
using System.Data;
using System.Data.Common;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Text;
using System.Reflection;
using System.Data.OleDb;
using Microsoft.Web.Management.DatabaseManager;

namespace AccessDatabase
{
    public class AccessProvider :
        DatabaseProvider,
        IDbTableManager,
        IDbTableDataManager,
        IDbViewManager,
        IDbBackupManager,
        IDbRestoreManager
    {

    }
}

Copy the code samples from the following MSDN topics into the class; note that some of the utility methods in the code samples will be duplicated so you'll have to remove the duplicate methods:
- DatabaseProvider
- IDbTableManager
- IDbTableDataManager
- IDbViewManager
- IDbBackupManager
  - CreateBackup Method
- IDbRestoreManager
  - RestoreBackup Method
Save, compile, sign, and GAC the provider. For more information about using the Gacutil.exe tool, see the following topic on Microsoft the MSDN Web site:
Global Assembly Cache Tool (Gacutil.exe)

Follow the instructions in the Microsoft.Web.Management.DatabaseManager Namespace topic to add the following entry to your administration.config file, which will register the provider for IIS Manager:

<provider name="Access Provider"
   providerName="System.Data.OleDb"
   type="AccessDatabase.AccessProvider,AccessDatabase,Version=1.0.0.0,Culture=neutral,PublicKeyToken=426f62526f636b73" >
   <databaseBackup
      backupEnabled="true"
      restoreEnabled="true"
      backupPath="C:\backups" />
</provider>

Note: You will need to update the PublicKeyToken with the public key token from your assembly.

Add an OLEDB connection string for your Access database; the following web.config sample shows what that might look like:

<configuration>
  <connectionStrings>
    <add name="Northwind"
      providerName="System.Data.OleDb"
      connectionString="Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\inetpub\wwwroot\App_Data\Northwind.mdb;" />
  </connectionStrings>
</configuration>

Note: Since the code samples were all written using OLEDB, not ODBC, you will need to make sure that you use an OLEDB connection string like my example.

Open the IIS Manager and open the Database Manager for the site where you added the connection string. You should now be able to manage the tables and views for your Access database, as well as backing up and restoring your database.

Additional Notes

Creating Database Backups

Even though I wrote my provider to implement backing up and restoring databases, this functionality is purely optional - you could easily remove the IDbBackupManager and IDbRestoreManager implementations from the class and remove the <databaseBackup> element from administration.config.

OLEDB Errors When Saving JOIN/ORDER BY Views (Queries)

Unfortunately, there is one unexpected OLEDB problem with Microsoft Access databases that I ran into that I could not work around. When you attempt to save a view (query) that contains both JOIN statements and ORDER BY statements, you will receive an error that states "Only simple SELECT queries are allowed in VIEWS."

As long as the SQL code is syntactically correct you can execute the query, but you cannot save it. If a query that contains both JOIN statements and ORDER BY statements has already been created in Microsoft Access, you can open it and make changes, but you will not be able to save those changes back to the database.

For example, the following SQL code will cause the error to occur:

SELECT Categories.CategoryName, Products.ProductName
FROM Products INNER JOIN Categories ON Products.CategoryID = Categories.CategoryID
ORDER BY Categories.CategoryName, Products.ProductName;

While the following code will not cause the error:

SELECT Categories.CategoryName, Products.ProductName
FROM Products INNER JOIN Categories ON Products.CategoryID = Categories.CategoryID;

Even though the SQL code is syntactically correct and will execute without any problems, the error occurs at the OLEDB layer when the provider executes the CREATE VIEW statement to save the view to the database. This appears to be an unfortunate and irresolvable OLEDB limitation when you are using an Access database. When you are faced with such a situation, you will need to open the database in Microsoft Access to save the query.

Database Manager API Topics on MSDN

robert_mcmurray — Thu, 24 Sep 2009 21:16:00 GMT

Today Microsoft released version 1.0 of the IIS Database Manager, which enables you to manage local and remote SQL Server or MySQL databases through the IIS Manager. I cannot stress enough how this module has rapidly become one of my favorite extensions for IIS Manager. There are many times when I need to access the data in one of my databases where opening the database management tool would be inconvenient or impossible. (For example, when I am working remotely, or when I don't have the database management tools installed.) In these situations, the Database Manager has been worth its weight in gold.

Another great feature of the IIS Database Manager is that it is extensible, like every other part of IIS these days, meaning that you can create your own database providers. Following up on that thought, in addition to all of the walkthroughs and blog posts that I've been writing about FTP and WebDAV over the past several months, I've also been working on documenting the API set for the IIS Database Manager, and all of those topics have been published on the MSDN Web site under the following URL:

Microsoft.Web.Management.DatabaseManager Namespace

I would be remiss if I didn't give credit where it's due, so my special thanks go to:

The Database Manager Feature Team - the entire team did some great work:
- Saad, Brian, Bob, Tim, Andrew, Nitasha, Richard, Carlos, Madhur, Nazim, Ivan, Crystal, Faith, Diana, Pete, Rich, Raul
- That being said, there are two people that I'd like to call our for their additional assistance:
  - Saad Ladki - Saad was the Program Manager for the Database Manager. I have to give credit where it's due and say that my job would have been a lot harder if Saad hadn't done the initial research for some of my code samples.
  - Brian Delahunty - Brian was my principal contact on the feature team whenever I needed help or backed myself into a corner. Brian was very accommodating when I suggested changes, and he always had great explanations when something needed clarification.
Microsoft Press - I know that it seems like a gratuitous plug, but I spent dozens of hours going through my "Microsoft® OLE DB 2.0 Programmer's Reference and Data Access SDK" book looking up the various parts of the OLEDB schema. That book was simply indispensable while I was writing my API samples.

eWeek Reviews for IIS 7.5 and FTP 7.5

robert_mcmurray — Thu, 24 Sep 2009 01:45:00 GMT

One of my coworkers, Vijay Sen, just forwarded the following eWeek review of IIS 7.5 to me:

http://www.eweek.com/c/a/Windows/REVIEW-Microsoft-IIS-75-Improves-Management-Deployment-Options-822018/

The review was written by Jim Rapoza, and he said some great things about IIS 7.5, which ships with both Windows Server 2008 R2 and Windows 7 client. But what really made my day was the following things that he said about FTP 7.5:

Another welcome change in IIS 7.5 is the elevation of FTP as a full-fledged part of the server. In previous versions, setup and management of an FTP server in IIS were done pretty much separately from Web server management. In IIS 7.5, FTP administration is fully integrated into the IIS Management Console.

I found this to be a very good implementation of FTP, making it possible to quickly set up secure FTP servers and tie them to my Websites. Especially nice was the ability to easily use virtual host names for the FTP sites. All in all, the FTP implementation in IIS 7.5 is one of the best I’ve seen, even when compared with dedicated FTP server products.

It’s great to see all of our hard work being recognized!

My thanks once again to everyone on the FTP and IIS feature teams that helped make this version of the FTP service: Jaroslav, Emily, Daniel, Umer, Suditi, Ciprian, Jeong, Dave, Andrew, Carlos, Brian, Wade, Ulad, Nazim, Reagan, Claudia, Rick, Tim, Tobin, Kern, Jenny, Nitasha, Venkat, Vijay. (I hope that I didn't leave anyone out!)

FTP 7.5 Extensibility and Visual Studio Express Editions

robert_mcmurray — Thu, 17 Sep 2009 21:19:00 GMT

In earlier blog posts I have mentioned that I written the several walkthroughs to help developers get started writing providers for the FTP 7.5 service, all of which available on Microsoft's learn.iis.net Web site under the "Developing for FTP 7.5" section. In each of these walkthroughs I wrote the steps as if you were using Visual Studio 2008.

Following up on that, I received a great question yesterday from a customer, Paul Dowdle, who wondered if it was possible to write an extensibility provider for the FTP 7.5 service using one of the Visual Studio Express Editions. By way of coincidence, I used to install Visual C# Express Edition on my laptop when I was traveling around the world to speak at events like TechEd. I usually did this because the Express Edition took up less hard drive space than a full installation of Visual Studio, and I was only writing code in C# on my laptop.

To answer Paul's question, the short answer is - yes, you can use Visual Studio Express Editions to develop custom providers for the FTP 7.5 service, with perhaps a few small changes from my walkthroughs.

For example, if you look at my "How to Use Managed Code (C#) to Create a Simple FTP Authentication Provider" walkthrough, in the section that is titled "Step 1: Set up the Project Environment", there is an optional step 6 for adding a custom build event to register the DLL automatically in the Global Assembly Cache (GAC) on your development computer.

When I installed Microsoft Visual C# 2008 Express Edition on a new computer, I didn't have the "%VS90COMNTOOLS%" environment variable or the "vsvars32.bat" file, so I had to update the custom build event to the following:

net stop ftpsvc
"%ProgramFiles%\Microsoft SDKs\Windows\v6.0A\bin\gacutil.exe" /if "$(TargetPath)"
net start ftpsvc

Once I made that change, the rest of the walkthrough worked as written.

So, to reiterate my earlier statement - you can use Visual Studio Express Editions to develop custom providers for the FTP 7.5 service. My thanks to Paul for the great question!

Hiding your FTP Server Type and Preventing Unauthorized Access

robert_mcmurray — Thu, 10 Sep 2009 23:05:00 GMT

As evidenced by my How to Use Managed Code (C#) to Create an FTP Authentication Provider with Dynamic IP Restrictions walkthrough and my other FTP authentication extensibility walkthroughs, I spend a lot of time trying to find ways to prevent unauthorized access to my FTP server while still allowing valid users to have easy access to their site content. Today's blog discusses several of the ideas that I like to use on my FTP servers.

Preventing Unauthorized Access

To start things off, I globally disable FTP Basic Authentication on my server and I only use custom authentication providers. Since my FTP users do not have actual accounts on my server or my domain, that helps prevent access to my physical server. It was for this reason that I wrote the authentication provider that I discuss in my How to Use Managed Code (C#) to Create an FTP Authentication Provider using an XML Database walkthrough; I can hand out accounts with FTP access for sites that are hosted on my servers, but those accounts are stored in an XML file and not in the security accounts database on my server or my domain.

But there are other ways to help secure your FTP server. For example, I wrote an earlier blog post about Creating a Global Listener FTP Site. On my servers I like to create a global listener site and disable all of the built-in authentication methods.

Once that has been completed I add the custom authentication provider from my How to Use Managed Code (C#) to Create an FTP Authentication Provider with Dynamic IP Restrictions walkthrough, but I don't create any accounts in the database - I just want to log and block the IP addresses of would-be attackers that are attempting to brute force their way into my server. Configuring the authentication options is accomplished through the FTP Authentication feature in IIS Manager.

Ever since I wrote and deployed my dynamic IP address restriction authentication provider, it has faithfully blocked dozens of would-be attackers.

Hiding your FTP Server Type

The next approach that I take is "Security through Obscurity", where I hide the easily recognizable signatures that identify my FTP server. The first step is to suppress the default banner and add a custom FTP banner. This is accomplished through the FTP Messages feature in IIS Manager.

Next, switch your directory listing style to Unix format. All FTP clients that I have tested understand this format, but it hides the fact that your server can do MS-DOS listings. Configuring your directory listing style is accomplished through the FTP Directory Browsing feature in IIS Manager.

After implementing these changes, it might now resemble the following example FTP session when you log in with FTP:

CMD>ftp localhost

Connected to DESKTOP.domain.local.
220 Welcome to my server!
User (DESKTOP.domain.local:(none)): foobar
331 Password required for foobar.
Password:
230 User logged in.
ftp> dir
200 EPRT command successful.
125 Data connection already open; Transfer starting.
drwxrwxrwx   1 owner    group               0 Jan 22 11:45 App_Code
drwxrwxrwx   1 owner    group               0 Sep 10 10:32 App_Data
drwxrwxrwx   1 owner    group               0 Aug 17 13:35 bin
-rwxrwxrwx   1 owner    group             689 May  8  2008 iisstart.htm
-rwxrwxrwx   1 owner    group            2714 Jan 22 11:54 web.config
-rwxrwxrwx   1 owner    group          184946 May  8  2008 welcome.png
226 Transfer complete.
ftp: 418 bytes received in 0.02Seconds 83.00Kbytes/sec.
ftp> bye
221 Goodbye.

CMD>

The last step that I take is to block the FTP infrastructure commands SYST and FEAT on the global listener FTP site. These commands respectively display your operating system and list of FTP features. Blocking these commands helps to further obfuscate your server type from attackers.

Note: It is important that you block these two commands only on your global listener FTP site and not on your other FTP sites, because some FTP clients may depend on these commands in order to interact properly with your other FTP sites. That said, if you are working with a small set of FTP clients and none of them seem to have a problem with suppressing the SYST command, you could probably suppress that globally. Most clients that I have tested seem to have no problem with that, especially if you are using Unix directory listings.

In any event, you block the SYST and FEAT commands through the FTP Request Filtering feature in IIS Manager.

Now when you log in with FTP and attempt to use the SYST and FEAT commands, it should look like the following example FTP session:

CMD>ftp localhost

Connected to DESKTOP.domain.local.
220 Welcome to my server!
User (DESKTOP.domain.local:(none)): foobar
331 Password required for foobar.
Password:
230 User logged in.
ftp> quote SYST
500 'SYST': command not allowed.
ftp> quote FEAT
500 'FEAT': command not allowed.
ftp> bye
221 Goodbye.

CMD>

Summary

The steps in this blog are certainly not all-inclusive, and I keep coming up with other ideas as time goes on, but following these steps should help to prevent unauthorized access to your FTP service. All of the normal warnings about using strong passwords, changing your passwords often, using SSL whenever you can, and keeping your hotfixes up-to-date will always apply.

But that being said, one of the ways that I keep finding new things to block is by simply watching my FTP log files to see what attackers are trying to do. By using some of the techniques that I discuss in my Using LogParser with FTP 7.x Sessions blog, you can monitor your FTP log files in order to analyze what would-be attackers are trying to do.

Creating Recursive Directory Listing Files for FTP Clients

robert_mcmurray — Sat, 05 Sep 2009 22:39:00 GMT

One of the changes that we made in FTP 7.0 and FTP 7.5 was to remove recursive directory listings, which are commonly retrieved by typing "ls -lR" from a command-line FTP client, which should send a command like "NLST -lR" over FTP to the server. There were several reasons why we decided to remove recursive directory listings, but the main reason was simply to reduce CPU usage on the server; recursive directory listing requests take a lot of resources to fulfill. With that in mind, both FTP 7.0 and FTP 7.5 will ignore the recursive switch on directory requests.

That being said - quite often it's pretty handy to have a full directory listing from an FTP server. From a client perspective you could probably write script to automate an FTP client to create a recursive listing, but that's a lot of work. Back in my younger days when I ran FTP sites on Unix servers, I would always create two types of list files on my FTP servers for FTP clients to retrieve:

"ls-lr.txt" - I would create only one file of this type for my entire FTP server, which would go in the root of my FTP site and it would contain a full recursive listing of all files in my FTP site.
"00index.txt" - I would create one file of this type in each folder of my FTP site, and each index file would contain a listing of files and their descriptions for that folder.

Of course, anyone that's been around the Internet since the days before we had HTTP and the world-wide-web should know that I didn't come up with this idea on my own - I learned it from other FTP site administrators. (And anyone who remembers those days should also recognize those two files with a strange sense of nostalgia. 00index.txt files of course led to index.htm files when WWW sites came along later, but that's another story.)

In any event, as I continued to host FTP sites over the years I have written various scripts to create recursive directory listings, and I thought that one of my scripts might make a good blog post. With that in mind, here is a Windows Script Host file that I created, which I named "ls-lr.vbs", and this script will create a recursive directory listing for an FTP site. I choose the Unix directory listing style for this script since that's the format that I have used for years and the broader number of FTP clients and users should recognize it.

Option Explicit
On Error Resume Next

' Declare all variables.
Dim objArguments
Dim strBaseFolder
Dim objFSO
Dim objFile
Dim objFolder
Dim objSubFolder
Dim objSubFile
Dim lngFolderCount
Dim lngBaseCount
Set objArguments = WScript.Arguments

' Determine the number of command-line arguments.
Select Case objArguments.Count
  Case 0:
    strBaseFolder = WScript.ScriptFullName
    strBaseFolder = Left(strBaseFolder,InStrRev(strBaseFolder,"\"))
  Case 1:
    strBaseFolder = objArguments(0)
  Case Else:
    MsgBox "This script takes a single argument for the" & vbCrLf & _
      "starting directory, or specify no arguments" & vbCrLf & _
      "to use the current directory.", vbInformation
End Select

' Create a file system object.
Set objFSO = WScript.CreateObject("Scripting.FileSystemObject")

' Test if the base folder exists.
If Right(strBaseFolder,1) <> "\" Then strBaseFolder = strBaseFolder & "\"
If objFSO.FolderExists(strBaseFolder) = False Then
    MsgBox "The specified folder does not exist.", vbCritical
    WScript.Quit
End If

' Open the output file for the directory listing.
Set objFile = objFSO.CreateTextFile(strBaseFolder & "ls-lr.txt")
     
' Define the initial values for the folder counters.
lngFolderCount = 1
lngBaseCount = 0
  
' Dimension an array to hold the folder names.
ReDim strFolders(1)
  
' Store the root folder in the array.
strFolders(lngFolderCount) = strBaseFolder
    
' Loop while we still have folders to process.
While lngFolderCount <> lngBaseCount
  ' Set up a folder object to a base folder.
  Set objFolder = objFSO.GetFolder(strFolders(lngBaseCount+1))
  
  ' Output the folder name to the listing file.
  objFile.WriteLine vbCrLf & _
    Replace(Mid(strFolders(lngBaseCount+1),Len(strBaseFolder)),"\","/") & _
    vbCrLf
  
  ' Loop through the collection of subfolders for the base folder.
  For Each objSubFolder In objFolder.SubFolders
    ' Increment the folder count.
    lngFolderCount = lngFolderCount + 1
    ' Increase the array size
    ReDim Preserve strFolders(lngFolderCount)
    ' Store the folder name in the array.
    strFolders(lngFolderCount) = objSubFolder.Path
    ' Output the folder to the listing file.
    Call WriteEntry(objSubFolder)
  Next
  
  ' Loop through the collection of subfolders for the base folder.
  For Each objSubFile In objFolder.Files
    ' Output the file to the listing file.
    Call WriteEntry(objSubFile)
  Next
  ' Increment the base folder counter.
  lngBaseCount = lngBaseCount + 1
Wend

Sub WriteEntry(tmpObject)
  Dim tmpAttributes
  Dim tmpSize
  
  ' Test for a symbolic link.
  If (tmpObject.Attributes And 1024) Then
    tmpAttributes = "lrwxrwxrwx"
    tmpSize = 0
  ' Test for a directory.
  ElseIf (tmpObject.Attributes And 16) Then
    tmpAttributes = "drwxrwxrwx"
    tmpSize = 0
  ' Otherwise - it's a file.
  Else
    tmpAttributes = "-rwxrwxrwx"
    tmpSize = tmpObject.Size
  End If
  
  ' Test for a read-only object.
  If (tmpObject.Attributes And 1) Then
    tmpAttributes = Replace(tmpAttributes,"w","-")
  End If
  
  ' Write the list entry to the output file.
  objFile.WriteLine tmpAttributes & _
    "   1 owner    group " & _
    Right(String(15,Chr(32)) & CStr(tmpSize),15) & _
    " " & FormatDate(tmpObject.DateLastModified) & _
    " " & tmpObject.Name
End Sub

Function FormatDate(tmpDate)
  FormatDate = CStr(Year(tmpDate)) & _
    "-" & Right("00" & CStr(Month(tmpDate)),2) & _
    "-" & Right("00" & CStr(Day(tmpDate)),2)
End Function

To use the script, copy the code into Windows Notepad and save it to your computer as "ls-lr.vbs." If you double-click the script it will use the current folder to create a recursive folder listing, and if you run this script from a command-line it can take a single argument of a folder path, or you can pass no arguments to the script in order to use the current folder. In either case it will create a file named "ls-lr.txt" in the root of the destination folder that contains the recursive directory listing in Unix format.

For example, the following listing was created from a folder in my music collection on my desktop computer:

/

drwxrwxrwx   1 owner    group               0 2009-07-30 Against the Silence
dr-xr-xr-x   1 owner    group               0 2009-07-30 Collective
drwxrwxrwx   1 owner    group               0 2009-07-30 Speakeasy
-rwxrwxrwx   1 owner    group            2741 2009-09-05 ls-lr.txt

/Against the Silence

-rwxrwxrwx   1 owner    group         9386309 2009-07-30 01-Against the Silence.wma
-rwxrwxrwx   1 owner    group         3974684 2009-07-30 02-Side-Stage Syndrome.wma
-rwxrwxrwx   1 owner    group         7539014 2009-07-30 03-The Dash on my Headstone.wma
-rwxrwxrwx   1 owner    group         7244819 2009-07-30 04-Teeth Like Knives.wma
-rwxrwxrwx   1 owner    group         9910687 2009-07-30 05-The Band Played on.wma

/Collective

-r-xr-xr-x   1 owner    group         2767821 2009-03-05 At the Moment.wma
-r-xr-xr-x   1 owner    group         5259473 2009-03-05 Colt . 45.wma
-r-xr-xr-x   1 owner    group         2572687 2009-03-05 El Mariachi.wma
-r-xr-xr-x   1 owner    group         2395577 2009-03-05 Gold and Silver.wma
-r-xr-xr-x   1 owner    group         2269487 2009-03-05 Keep Waiting.wma
-r-xr-xr-x   1 owner    group         2050335 2009-03-05 Nighttown.wma
-r-xr-xr-x   1 owner    group         1458931 2009-03-05 Rise.wma
-r-xr-xr-x   1 owner    group         3140077 2009-03-05 Rivers Underneath.wma
-r-xr-xr-x   1 owner    group         2278489 2009-03-05 Sad Parade.wma
-r-xr-xr-x   1 owner    group         1909249 2009-03-05 The Hungry Wolf.wma
-r-xr-xr-x   1 owner    group         2467613 2009-03-05 Threshold.wma
-r-xr-xr-x   1 owner    group          795501 2009-03-05 Tranewreck.wma
-r-xr-xr-x   1 owner    group          417239 2009-03-05 Zzyzx.wma

/Speakeasy

-rwxrwxrwx   1 owner    group         4004604 2009-03-05 01-Minuteman.wma
-rwxrwxrwx   1 owner    group         6309752 2009-03-05 02-Sundown Motel.wma
-rwxrwxrwx   1 owner    group         5504122 2009-03-05 03-Keep Waiting.wma
-rwxrwxrwx   1 owner    group         2766262 2009-03-05 04-You Know How It Is.wma
-rwxrwxrwx   1 owner    group         7495952 2009-03-05 05-Rivers Underneath.wma
-rwxrwxrwx   1 owner    group         6294888 2009-03-05 06-Gold and Silver.wma
-rwxrwxrwx   1 owner    group         8062882 2009-03-05 07-Freefall.wma
-rwxrwxrwx   1 owner    group         4437286 2009-03-05 08-[Untitled].wma
-rwxrwxrwx   1 owner    group         3355592 2009-03-05 09-St. Eriksplan.wma
-rwxrwxrwx   1 owner    group         4966942 2009-03-05 10-Disquiet.wma
-rwxrwxrwx   1 owner    group         4788302 2009-03-05 11-Fascination Street.wma
-rwxrwxrwx   1 owner    group         7950944 2009-03-05 12-This Love.wma

(Note/Disclaimer/etc.: It may or may not be obvious that this listing is for music from the band Stavesacre, but just to be clear and avoid any RIAA entanglements - these files aren't actually hosted on any of my FTP sites; I used the script on my desktop computer to create a listing as an example for this blog.)

Customizing the Script Output

There are several customizations that you can do with this script, each of which has it's own benefits and drawbacks.

Adding Directory Sizes

It is trivial from a coding perspective to have the code calculate directory sizes since the Folder object that I use has as Size property, but it slows down the script exponentially to calculate that. That being said, if you're willing to take the performance hit, you can modify the highlighted section of the WriteEntry() function as follows:

  ' Test for a symbolic link.
  If (tmpObject.Attributes And 1024) Then
    tmpAttributes = "lrwxrwxrwx"
    tmpSize = 0
  ' Test for a directory.
  ElseIf (tmpObject.Attributes And 16) Then
    tmpAttributes = "drwxrwxrwx"
    tmpSize = tmpObject.Size
  ' Otherwise - it's a file.
  Else
    tmpAttributes = "-rwxrwxrwx"
    tmpSize = tmpObject.Size
  End If

This will insert the folder size into the output, but once again it will make the script much slower and take up considerably more CPU time to compute.

Uppercase Folder Names and Lowercase File Names

Since Windows is a case-insensitive operating system, you can easily choose to display all of your folder names in all uppercase characters and your file names in all lowercase characters without causing any client confusion. This can be accomplished by adding the first highlighted section and modifying the second highlighted section of the WriteEntry() function as follows:

  Dim tmpName
  
  ' Test for a directory.
  If (tmpObject.Attributes And 16) Then
    tmpName = UCase(tmpObject.Name)
  Else
    tmpName = LCase(tmpObject.Name)
  End If

  ' Write the list entry to the output file.
  objFile.WriteLine tmpAttributes & _
    "   1 owner    group " & _
    Right(String(15,Chr(32)) & CStr(tmpSize),15) & _
    " " & FormatDate(tmpObject.DateLastModified) & _
    " " & tmpName

Parting Thoughts

There are other customizations that you can easily make, such as creating a string array to sort the files and folders for each folder listing as a single list rather than listing folders first and files second as the currently script does. But that slows down the script way too much, and I prefer to see folders listed before files anyway. (Which is why I always use SET DIRCMD=/OGN for my command prompt sessions as well.)

Another easy customization would be to change the FormatDate() function to change the date format for the output file, which is why I used a function to do my date formatting. For example, you could easily use the FormatDate() function as a wrapper for VBScript's built-in FormatDateTime() function, and then use any of the vbGeneralDate, vbLongDate, vbShortDate, etc. options to specify the format. You can also use your own customized logic to return the date string, so you don't need to feel limited by my examples.

Another useful customization would be to compute the actual size for the resulting "ls-lr.txt" file and modify the output file to contain the correct file size. Currently the script in this blog adds an entry to the listing for the "ls-lr.txt" file, but that contains the temporary size of the output file as the script is running so it will seldom be accurate. (I usually run my script and update the "ls-lr.txt" file manually, but in some versions of this script I have had it ignore the "ls-lr.txt" file and remove it from the output listings.)

In closing, this script may be doing more than it might actually need to do by way of checking for symbolic links and read-only attributes, which our FTP service doesn’t actually do, but it was very easy to add that code and it runs just as fast either way.

The End of Metabase Corruption (R.I.P.)

robert_mcmurray — Sat, 22 Aug 2009 10:28:00 GMT

Even though IIS 7 with its new XML-based configuration settings has been around for a while, I was going through some old directories on one of my computers the other day and I stumbled across an image that I had labeled "Metabase Corruption." I have kept that image around for the past decade or so because it's one of the few actual examples of metabase corruption that I have actually verified, although there have definitely been more cases than just the ones that I have seen. But I thought that it might make for a good blog entry to explain the origins of the term "Metabase Corruption" and what that means to you if you're using a version of IIS that is earlier than IIS 7.

A brief history of the IIS metabase and metabase editors

Many years ago I was working in the Microsoft Product Support Services (PSS) group, where I provided support for enterprise-level customers that were using IIS versions 1.0 through 6.0. The early versions of IIS, (versions 1.0 through 3.0), stored all of their settings in the registry. This was fine for that period of time, but it didn't scale well and for several other reasons it was cumbersome to deal with. Just to bring back some nightmares, here's an image of the IIS 3.0 manager:

When IIS 4.0 was released, it introduced the "metabase," which was a proprietary database that stored all of the metadata for IIS, and all of that information was physically stored in a file that was named metabase.bin. The metabase scaled considerably better than the registry, but it introduced several other problems. The biggest problem was that the metabase was something of a "black box" for end users - all of your settings were stored in the metabase, and the only way to get to those settings was through the IIS Manager or by using unsupported command-line tools. The next big problem was that the metabase wasn't portable - you couldn't copy the metabase from one computer to another. (If you tried you would quickly realize that you just killed IIS.) When the IIS 4.0 Resource Kit was released it introduced MetaEdit 1.0, which was an editor for the metabase that looked like RegEdit, but it crashed so often that it really wasn't useful.

During the Windows 2000 timeframe I took over the MetaEdit code base and I created MetaEdit 2.0, which was released with the Windows 2000 Resource Kit. I fixed all of the stability problems from MetaEdit 1.0 and I added export/import functionality so that users could copy sections of the metabase between servers. Actually, I wrote MetaEdit before Microsoft stopped adding Easter Eggs to software, so there's a small Easter Egg in the Help::About for MetaEdit that you can find by holding down Shift-Ctrl-Alt and double-clicking the icon - you first see a photo of Bill Gates and then you see a photo of me from the days when I had longer hair...

I released a couple of patches to MetaEdit over the years, and the last version that I released is version 2.2, which you can download through the following URL:

How To Install MetaEdit 2.2 on Windows NT 4.0 or Windows 2000

When Windows Server 2003 was in development, XML had risen within the development community as a great way to store your data, so the IIS team decided to switch from the metabase.bin format to metabase.xml. This allowed for editing the metabase using any XML editor, even Windows Notepad, and IIS added native export and import functionality, so you could migrate settings from one computer to another by using IIS Manager.

It was sometime during the Windows 2003 timeframe that I was working on MetaEdit 3.0 when I heard that someone else was working on Metabase Explorer, and the early versions of Metabase Explorer that I was able to test were really cool, so I ceased work on MetaEdit in favor of Metabase Explorer, which was eventually released with the IIS 6.0 Resource Kit Tools. You can download the IIS 6.0 Resource Kit Tools through the following URL:

Internet Information Services (IIS) 6.0 Resource Kit Tools

Note: MetaEdit should only be used on Windows NT 4.0 with IIS 4.0 or Windows 2000 with IIS 5.0. For IIS 6.0 you should only use Metabase Explorer. (And yes - you can blame me for the totally useless dialog that you get when you try to use MetaEdit with IIS 6.0. Sorry about that. )

What's all this about metabase corruption?

It was shortly after the release of IIS 4.0 that I first heard the term "Metabase Corruption" being used. People were using the term "Metabase Corruption" as a catch-all phrase for "I don't what's broken but reinstalling IIS fixed the problem so it must have been a problem with the metabase." Everyone's general opinion seemed to be that if you couldn't find the problem and you can't look at the metabase, then it must be corruption in the metabase if you can't resolve the issue without reinstalling IIS. To me, this seemed like a cop-out for good troubleshooting, and since I was a technical lead in PSS around the time that IIS 5.0 was being released I would generally refuse to accept anyone's claim when they said that they had a customer with "Metabase Corruption." Instead, I would require them to troubleshoot the problem further, and in nearly 100% of those situations the problem was not caused by the metabase but by some other problem that could be rectified without reinstalling IIS.

Here's a case in point - several years ago I worked on an issue that some of the folks that supported SharePoint Portal Server 2001 on IIS 5.0 were seeing. They first described the problem to me as "IIS was losing settings." What they were actually seeing was that they would make changes in IIS Manager and apply them, but when they would close and reopen IIS Manager they saw that all their settings were gone. They claimed that this was "Metabase Corruption," and because of this they would often make the customer reinstall IIS completely. Needless to say I made them quite angry by stating that "Metabase Corruption" is almost never actual corruption, and that they were wasting the customer's time by making them reinstall IIS. Even though reinstalling IIS made the problem go away, it didn't actually fix the problem and it only hid the actual cause at the customer's expense.

Since I knew a little more than they did about how the metabase worked, I could theorize reasons why that behavior might occur and I asked them to contact me when they had a customer with this problem again. The next time they saw this problem I asked them to follow these steps:

Stop IIS by using "iisreset /stop" from a command-line
Start just the IISADMIN service by using "net start IISADMIN" from a command-line
Open the IIS Manager, make their changes, then close the IIS Manager
Restart IIS by using "iisreset /restart" from a command-line
Open IIS Manager and verify their settings

In 100% of the cases this enabled them to work around the issue, but that was only part of the solution. I theorized that something was causing IIS to crash before the metabase changes were flushed to disk, but the Windows service recovery settings were instructing the server to restart IIS whenever it crashed so it seemed to the customer like nothing had crashed at all. With this in mind, I had them check the Windows System Log in the Event Viewer, and it listed several errors indicating that the IISADMIN service had terminated unexpectedly, which confirmed my suspicions. After some additional troubleshooting I discovered that SharePoint Portal Server 2001 was setting several illegal values in the metabase, and sooner or later that would cause IIS 5.0 to crash. (SharePoint has long since fixed that problem.)

Does metabase corruption actually happen? If so, what can I do about it?

Sadly - yes it does. It doesn't happen as often as some people seem to think that it does, but it is certainly possible. The metabase is simply a file, and file corruption can occur for any number of reasons. In the early days of IIS 5.0 on the Windows 2000 beta I discovered (quite by accident) how I could overwrite one specific value in the metabase and crash IIS so badly that IIS couldn't even be uninstalled. (Don't worry - that problem has long since been fixed.) I have also seen metabase corruption due to hardware failures, like a drive controller that is going bad and the problem is manifesting itself slowly over time.

That being said, there are several methods to check if you actually have metabase corruption:

The first and easiest thing to check is - does IIS start and all of your web sites seem to work?
Can you open up the IIS Manager and see all of your sites?
Can you enumerate the metabase without errors using MetaEdit, Metabase Explorer, or "adsutil.vbs enum_all w3svc" from a command line?

If the answer to any or all of these questions is yes - then chances are good that you do not have metabase corruption, and whatever problem you have is due to misconfiguration or some other issue.

The best way that I can think of to prevent metabase corruption is to avoid using a variety of tools to constantly read and write to the metabase outside of normal IIS operations. I have written scripts that perform literally thousands of metabase operations under a nominal load in a lab environment for testing purposes and never had a problem, but a server under heavy load with thousands of read/write operations per minute could theoretically run into problems if left to run like that for an indefinite length of time. But theoretically the same may be true for any file under heavy read/write load for an indefinite length of time.

Protecting against metabase corruption

The best thing that you can do to prevent against metabase corruption is backup the metabase regularly!!! I cannot count the number of times where having a good backup of the metabase saved me from a myriad of issues. Fortunately, there are several KB articles that describe backing up the metabase:

The last KB article in the above list is based on a script that I wrote for one of my IIS 5.0 on Windows 2000 servers many years ago that I continue to use on any servers that run IIS 6.0 on Windows Server 2003.

Does IIS 7 have metabase corruption?

The quick answer to this is "no", but that answer is a little misleading since IIS 7 doesn't have a metabase!

In IIS 7 we rolled out a brand new XML-based configuration store, and the bulk of your IIS settings are stored in a file named applicationHost.config, which greatly resembles the web.config configuration paradigm that was established with the .NET framework. That being said, applicationHost.config is just another file, and even though its syntax is much easier to understand and manually edit, any file can theoretically become corrupted, so you should always make sure that you backup your applicationHost.config file as well.

With this in mind I wrote the following blog post some time ago where I discuss creating a script to automate backing up your IIS 7 configuration settings:

Automating IIS 7 Backups

(Note: I also wrote another blog post recently about how the script in that blog post saved my day when something did get corrupted on one of my servers.)

Parting thoughts

I started out this blog post by describing an image that I found on one of my computers where actual metabase corruption had occurred, so I thought that a nice way to close out this blog post would be to show you the actual image:

It's been almost ten years now, so I don't remember the root cause anymore, but I remember what it looked like. I had a mixture of sites that were working and others that were not, and when I opened the metabase in MetaEdit I noticed that scattered throughout the metabase were hundreds of values that shouldn't be there or values that did not make sense. MetaEdit was trying its best to show all of the data, but you'll notice in the highlighted areas of the image that several of the values had bad data in them.

Merging FTP Extensibility Walkthroughs - Part 2

robert_mcmurray — Thu, 20 Aug 2009 06:36:00 GMT

I had not intended to do a series on this subject when I wrote my original Merging FTP Extensibility Walkthroughs blog post, but I came up with a scenario that I felt was worth sharing. I recently posted the following walkthrough on the learn.iis.net web site:

How to Use Managed Code (C#) to Create an FTP Authentication Provider with Dynamic IP Restrictions

We have had many customer requests for a dynamic IP restrictions provider for the FTP server, and I wanted to get that out to customers as soon as I could. That being said, like several of my extensibility walkthroughs in the past, I wrote and tested the provider in that walkthrough on one of the servers that I manage. To show how effective it was, within the first couple of hours the provider had caught and blocked its first script kiddie who was attempting a brute force attack on my FTP server. Over the next few days the provider caught its next hacker, and over the past few weeks it has continued to do so.

That being said, I thought that it might be nice to know when an IP address was blocked, and I had already written the following walkthrough:

How to Use Managed Code (C#) to Create an FTP Provider that Sends an Email when Files are Uploaded

With that in mind, merging the two walkthroughs seemed like a simple thing to do.

Before continuing I need to reiterate the notice that I added to the dynamic IP restrictions walkthrough:

IMPORTANT NOTE: The latest version of the FTP 7.5 service must be installed in order to use the provider in this walkthrough. A version FTP 7.5 was released on August 3, 2009 that addressed an issue where the local and remote IP addresses in the IFtpLogProvider.Log() method were incorrect. Because of this, using an earlier version of the FTP service will prevent this provider from working.

With that warning out of the way, here are the steps that you need to follow in order to merge the two walkthroughs:

Step 1 - Create the project

Create a new C# project following all of the steps in the How to Use Managed Code (C#) to Create an FTP Authentication Provider with Dynamic IP Restrictions walkthrough.

Step 2 - Merge global variables

In this step you need to merge the global variables from the two walkthroughs. In my provider this looked like the following:

// Define the default values - these are only
// used if the configuration settings are not set.
const int defaultLogonAttempts = 5;
const int defaultFloodSeconds = 30;
const int defaultSmtpPort = 25;

// Define a connection string with no default.
private static string _connectionString;

// Initialize the private variables with the default values.
private static int _logonAttempts = defaultLogonAttempts;
private static int _floodSeconds = defaultFloodSeconds;

// Flag the application as uninitialized.
private static bool _initialized = false;

// Define a list that will contain the list of flagged sessions.
private static List<string> _flaggedSessions;

private string _smtpServerName;
private string _smtpFromAddress;
private string _smtpToAddress;
private int _smtpServerPort;

Step 3 - Merge the Initialize() methods

In this step you need to merge the Initialize() methods from the two walkthroughs so that all of the settings are retrieved from the IIS configuration file when the provider is loaded by the FTP service. In my provider this looked like the following:

// Initialize the provider.
protected override void Initialize(StringDictionary config)
{
    // Test if the application has already been initialized.
    if (_initialized == false)
    {
        // Create the flagged sessions list.
        _flaggedSessions = new List<string>();
        
        // Retrieve the connection string for the database connection.
        _connectionString = config["connectionString"];
        if (string.IsNullOrEmpty(_connectionString))
        {
            // Raise an exception if the connection string is missing or empty.
            throw new ArgumentException(
                "Missing connectionString value in configuration.");
        }
        else
        {
            // Determine whether the database is a Microsoft Access database.
            if (_connectionString.Contains("Microsoft.Jet"))
            {
                // Throw an exception if the database is a Microsoft Access database.
                throw new ProviderException("Microsoft Access databases are not supported.");
            }
        }
        
        // Retrieve the number of failures before an IP
        // address is locked out - or use the default value.
        if (int.TryParse(config["logonAttempts"], out _logonAttempts) == false)
        {
            // Set to the default if the number of logon attempts is not valid.
            _logonAttempts = defaultLogonAttempts;
        }
        
        // Retrieve the number of seconds for flood
        // prevention - or use the default value.
        if (int.TryParse(config["floodSeconds"], out _floodSeconds) == false)
        {
            // Set to the default if the number of logon attempts is not valid.
            _floodSeconds = defaultFloodSeconds;
        }
        
        // Test if the number is a positive integer and less than 10 minutes.
        if ((_floodSeconds <= 0) || (_floodSeconds > 600))
        {
            // Set to the default if the number of logon attempts is not valid.
            _floodSeconds = defaultFloodSeconds;
        }
        
        // Retrieve the email settings from configuration.
        _smtpServerName = config["smtpServerName"];
        _smtpFromAddress = config["smtpFromAddress"];
        _smtpToAddress = config["smtpToAddress"];
        
        // Detect and handle any mis-configured settings.
        if (!int.TryParse(config["smtpServerPort"], out _smtpServerPort))
        {
            _smtpServerPort = defaultSmtpPort;
        }
        if (string.IsNullOrEmpty(_smtpServerName))
        {
            throw new ArgumentException(
                "Missing smtpServerName value in configuration.");
        }
        if (string.IsNullOrEmpty(_smtpFromAddress))
        {
            throw new ArgumentException(
                "Missing smtpFromAddress value in configuration.");
        }
        if (string.IsNullOrEmpty(_smtpToAddress))
        {
            throw new ArgumentException(
                "Missing smtpToAddress value in configuration.");
        }
        
        // Initial garbage collection.
        GarbageCollection(true);
        
        // Flag the provider as initialized.
        _initialized = true;
    }
}

Step 4 - Add a SendEmail() method

For this step I copied some of my code from the email walkthrough and used it as the foundation for a new SendEmail() method that I added to the provider. In my provider this looked like the following:

private void SendEmail(string emailSubject, string emailMessage)
{
    // Create an SMTP message.
    SmtpClient smtpClient = new SmtpClient(_smtpServerName, _smtpServerPort);
    MailAddress mailFromAddress = new MailAddress(_smtpFromAddress);
    MailAddress mailToAddress = new MailAddress(_smtpToAddress);
    
    using (MailMessage mailMessage = new MailMessage(mailFromAddress, mailToAddress))
    {
        try
        {
            // Format the SMTP message as UTF8.
            mailMessage.BodyEncoding = Encoding.UTF8;
            // Add the subject.
            mailMessage.Subject = emailSubject;
            // Add the body.
            mailMessage.Body = emailMessage;
            // Send the email message.
            smtpClient.Send(mailMessage);
        }
        catch (SmtpException ex)
        {
            // Send an exception message to the debug
            // channel if the email fails to send.
            Debug.WriteLine(ex.Message);
        }
    }
}

Note: This uses the settings that you store in your IIS applicationHost.config file and are loaded by the Initialize() method.

Step 5 - Add email functionality to the BanAddress() method

In this step you add the functionality to send an email whenever an IP address is added to the list of banned IP addresses. In my provider this looked like the following:

// Mark an IP address as banned.
private void BanAddress(string ipAddress)
{
    // Check if the IP address is already banned.
    if (IsAddressBanned(ipAddress) == false)
    {
        // Ban the IP address if it is not already banned.
        InsertDataIntoTable("[BannedAddresses]",
            "[IPAddress]", "'" + ipAddress + "'");
        // Send an email for the banned address.
        SendEmail("Banned IP Address",
            "The IP address " + ipAddress + " was banned.");
    }
}

Step 6 - Methods that are not changed

I need to point out that there are several methods that require no changes. These methods are listed here for reference:

Dispose()
AuthenticateUser()
Log()
IsValidUser()
IsAddressBanned()
IsSessionFlagged()
FlagSession()
GarbageCollection
GetRecordCountByCriteria()
InsertDataIntoTable()
DeleteRecordsByCriteria()
ExecuteQuery()

Note: You could easily add the email functionality to the FlagSession() method so you will see when a banned IP address is trying to access your server, but depending on the number of sessions that are flagged on your server you might receive more emails than you really need.

Step 7 - Register the provider and configure your settings

In this last step you add the provider to your IIS configuration settings using the AppCmd utility, and you specify the values for the various settings that the provider requires:

cd %SystemRoot%\System32\Inetsrv AppCmd.exe set config -section:system.ftpServer/providerDefinitions /+"[name='FtpAddressRestrictionAuthentication',type='FtpAddressRestrictionAuthentication,FtpAddressRestrictionAuthentication,version=1.0.0.0,Culture=neutral,PublicKeyToken=426f62526f636b73']" /commit:apphost AppCmd.exe set config -section:system.ftpServer/providerDefinitions /+"activation.[name='FtpAddressRestrictionAuthentication']" /commit:apphost AppCmd.exe set config -section:system.ftpServer/providerDefinitions /+"activation.[name='FtpAddressRestrictionAuthentication'].[key='smtpServerName',value='localhost']" /commit:apphost AppCmd.exe set config -section:system.ftpServer/providerDefinitions /+"activation.[name='FtpAddressRestrictionAuthentication'].[key='smtpServerPort',value='25']" /commit:apphost AppCmd.exe set config -section:system.ftpServer/providerDefinitions /+"activation.[name='FtpAddressRestrictionAuthentication'].[key='smtpFromAddress',value='someone@contoso.com']" /commit:apphost AppCmd.exe set config -section:system.ftpServer/providerDefinitions /+"activation.[name='FtpAddressRestrictionAuthentication'].[key='smtpToAddress',value='someone@contoso.com']" /commit:apphost AppCmd.exe set config -section:system.ftpServer/providerDefinitions /+"activation.[name='FtpAddressRestrictionAuthentication'].[key='connectionString',value='Server=localhost;Database=FtpAuthentication;User ID=FtpLogin;Password=P@ssw0rd']" /commit:apphost AppCmd.exe set config -section:system.ftpServer/providerDefinitions /+"activation.[name='FtpAddressRestrictionAuthentication'].[key='logonAttempts',value='5']" /commit:apphost AppCmd.exe set config -section:system.ftpServer/providerDefinitions /+"activation.[name='FtpAddressRestrictionAuthentication'].[key='floodSeconds',value='30']" /commit:apphost

Note: You need to update the above syntax using the managed type information for your provider and the configuration settings for your SMTP server, email addresses, and database connection string.

Step 8 - Add the provider to a site

In this last step you add the provider to a site. If you were adding the provider to your Default Web Site that would look like the following:

AppCmd.exe set config -section:system.applicationHost/sites /"[name='Default Web Site'].ftpServer.security.authentication.basicAuthentication.enabled:False" /commit:apphost AppCmd.exe set config -section:system.applicationHost/sites /+"[name='Default Web Site'].ftpServer.security.authentication.customAuthentication.providers.[name='FtpAddressRestrictionAuthentication',enabled='True']" /commit:apphost AppCmd set site "Default Web Site" /+ftpServer.customFeatures.providers.[name='FtpAddressRestrictionAuthentication',enabled='true'] /commit:apphost

Summary

That wraps it up for today's post, and I hope you find it useful.

Automatically Creating Checksum Files for FTP Uploads

robert_mcmurray — Fri, 31 Jul 2009 04:01:00 GMT

I had a great question in the publishing forums on forums.iis.net, where someone was asking if FTP 7 supported the XCRC command. The short answer is that the XCRC command is not supported, but I came up with a way to create an FTP provider that supports something like it. Since it was a rather fun code sample to write, I thought that I'd turn it into a blog.

The sample FTP provider code in this blog post will automatically calculate an MD5 checksum from a file that is uploaded and store it in a file with a "*.MD5.TXT" file name extension. You can then compare the uploaded checksum with a local checksum on the client to verify the uploaded file's integrity.

There are a few points that I need to discuss before I present the code sample:

I chose to use MD5 because it is built-in to the .NET System.Security.Cryptography namespace and I often like to use MD5 for file checksums. I could just have easily implemented SHA1, SHA256, or any of the other built-in hashing algorithms. Unfortunately, CRC32 is not a built-in algorithm for .NET, but a quick search around the Internet yielded several CRC32 samples in C# from various developers, so if you specifically need the CRC32 algorithm you can find it pretty quickly and substitute it for MD5 in my example. (You can click here to search for examples.) You could go one step further and have your provider support multiple checksum algorithms, but that's going way outside the scope of this blog.
There are a couple of security considerations for this provider:
- The provider needs to calculate the path of the uploaded file, and to do so requires calling into the IIS configuration APIs. As I mention in the code remarks:
  - The FTP service will host the compiled assembly in the "Microsoft FTP Service Extensibility Host" COM+ package (DLLHOST.EXE), which runs by default as NETWORK SERVICE.
  - Also by default, the NETWORK SERVICE account does not have sufficient privileges to read the IIS configuration settings. As such, you must either grant READ permissions to NETWORK SERVICE for the IIS configuration files, or configure the COM+ package to run as a user that has at least READ access to the files in the InetSrv\config folder.
  - By default, the NETWORK SERVICE account may not have WRITE permission to the folder where your files are uploaded, so the checksum files cannot be written. As such, you will need to grant READ/WRITE access to the destination where the checksum files will be written.
- The above steps are not generally recommended practices; but if you choose to grant NETWORK SERVICE permission to the configuration files, the remarks section in the code sample provides the details that you need.
- Alternatively, you could skip the path lookup and always store the checksum files in a known location. This allows you to remove the MapSiteRootPath() and FindElement() methods from the code sample, and you need only grant the NETWORK SERVICE account permission for the known location.
The MapSiteRootPath() method in the provider sample calculates the path of the site's root, then uses the relative path of the uploaded file to compute the full path to the checksum file. This does not take into account any paths that include virtual directories; as such, you would need to accommodate for any virtual paths in your site's hierarchy. (That's too much code for this blog post.)
The provider defines a 1 GB constant for the maximum file size for computing checksums. I specified this value so that large files would not tie up your system's resources. You can increase or decrease that value, you could make that a parameter that is stored in the provider's settings, or you can remove the functionality completely. This provider runs synchronously, so larger files will obviously take more time. While it's outside the scope of this blog, you could implement some form of asynchronous functionality. (When discussing this provider with Daniel Vasquez Lopez, he suggested using MSMQ - but that's really going way beyond the scope of what I wanted to accomplish with this blog.)

All of that being said, this provider follows the same development path as the provider in my How to Use Managed Code (C#) to Create a Simple FTP Logging Provider walkthrough, so if you follow the steps in that walkthrough and substitute "FtpUploadChecksumDemo" every place that you see "FtpLoggingDemo" and add a reference to Microsoft.Web.Administration, you should have all of the steps that you need in order to use this provider.

So without further discussion, here's the code for the provider:

using System;
using System.Configuration.Provider;
using System.IO;
using System.Security.Cryptography;
using System.Text;
using Microsoft.Web.Administration;
using Microsoft.Web.FtpServer;

// NOTE: This code is provided "as-is" and comes with the following security
// considerations. The FTP service will host the compiled assembly in the
// "Microsoft FTP Service Extensibility Host" COM+ package (DLLHOST.EXE),
// which runs by default as NETWORK SERVICE. By default, this account does not
// have sufficient privileges to read the IIS configuration settings. As such,
// you must either grant READ permissions to NETWORK SERVICE for the configuration
// files, or configure the COM+ package to run as a user that has at least READ
// access to the files in the InetSrv\config folder and READ/WRITE access to the
// destination where the checksum file will be written. However, these are not
// generally recommended practices.
//
// If you choose to grant NETWORK SERVICE permission to the configuration files,
// the following three commands should accomplish the requisite permissions:
//
//  cacls "%SystemRoot%\System32\inetsrv\config" /G "Network Service":R /E
//  cacls "%SystemRoot%\System32\inetsrv\config\redirection.config" /G "Network Service":R /E
//  cacls "%SystemRoot%\System32\inetsrv\config\applicationHost.config" /G "Network Service":R /E
//
// NOTE: You will need to do something similar for your content directory so that
// the checksum files can be created.

public sealed class FtpUploadChecksumDemo : BaseProvider, IFtpLogProvider
{
  // Implement the logging method.
  void IFtpLogProvider.Log(FtpLogEntry loggingParameters)
  {
    // Test for a successful file upload operation.
    if ((loggingParameters.Command == "STOR") && 
      (loggingParameters.FtpStatus == 226))
    {
      try
      {
        // Define a 1GB maximum length - to prevent system hogging.
        const long maxLength = 0x3fffffff;

        // Map the path to the site root.
        string fullPath = MapSiteRootPath(loggingParameters.SiteName);
        // Append the relative path of the uploaded file.
        fullPath += loggingParameters.FullPath;
        // Expand any environment variables.
        fullPath = Environment.ExpandEnvironmentVariables(fullPath);
        // Convert forward slashes to back slashes
        fullPath = fullPath.Replace(@"/", @"\");

        // Open the uploaded file to create a CRC.
        using (FileStream input = File.Open(
          fullPath,
          FileMode.Open,
          FileAccess.Read,
          FileShare.Read))
        {
          // Test the input file length.
          if (input.Length > maxLength)
          {
            // Throw an execption if the file is too big.
            throw new ProviderException(
              String.Format("Input file is too large: {0}",
              input.Length.ToString()));
          }
          else
          {
            // Open the hash file for output.
            using (StreamWriter output = new StreamWriter(
              fullPath + ".MD5.txt",
              false))
            {
              // Create an MD5 object.
              MD5 md5 = MD5.Create();
              // Retrieve the hash byte array.
              byte[] byteArray = md5.ComputeHash(input);
              // Create a new string builder for the ASCII hash string.
              StringBuilder stringBuilder =
                new StringBuilder(byteArray.Length * 2);
              // Loop through the hash.
              foreach (byte byteMember in byteArray)
              {
                // Append each ASCII hex byte to the hash string.
                stringBuilder.AppendFormat("{0:x2}", byteMember);
              }
              // Write the hash string to the output file.
              output.Write(stringBuilder);
            }
          }
        }
      }
      catch(Exception ex)
      {
        throw new ProviderException(ex.Message);
      }
    }
  }

  // This method is almost 100% from scripts that were created
  // by the IIS Manager Configuration Editor admin pack tool.
  private static string MapSiteRootPath(string siteName)
  {
    try
    {
      using (ServerManager serverManager = new ServerManager())
      {
        Configuration config =
          serverManager.GetApplicationHostConfiguration();
        ConfigurationSection sitesSection =
          config.GetSection("system.applicationHost/sites");
        ConfigurationElementCollection sitesCollection =
          sitesSection.GetCollection();
        ConfigurationElement siteElement =
          FindElement(sitesCollection, "site", "name", siteName);
        if (siteElement == null)
        {
          throw new InvalidOperationException("Element not found!");
        }
        else
        {
          ConfigurationElementCollection siteCollection =
            siteElement.GetCollection();
          ConfigurationElement applicationElement =
            FindElement(siteCollection,
            "application",
            "path", @"/");
          if (applicationElement == null)
          {
            throw new InvalidOperationException("Element not found!");
          }
          else
          {
            ConfigurationElementCollection applicationCollection =
              applicationElement.GetCollection();
            ConfigurationElement virtualDirectoryElement =
              FindElement(applicationCollection,
              "virtualDirectory",
              "path", @"/");
            if (virtualDirectoryElement == null)
            {
              throw new InvalidOperationException("Element not found!");
            }
            else
            {
              return virtualDirectoryElement["physicalPath"].ToString();
            }
          }
        }
      }
    }
    catch (Exception ex)
    {
      throw new ProviderException(ex.Message);
    }
  }

  // This method is almost 100% from scripts that were created
  // by the IIS Manager Configuration Editor admin pack tool.
  private static ConfigurationElement FindElement(
    ConfigurationElementCollection collection,
    string elementTagName,
    params string[] keyValues)
  {
    foreach (ConfigurationElement element in collection)
    {
      if (String.Equals(element.ElementTagName,
        elementTagName,
        StringComparison.OrdinalIgnoreCase))
      {
        bool matches = true;

        for (int i = 0; i < keyValues.Length; i += 2)
        {
          object o = element.GetAttributeValue(keyValues[i]);
          string value = null;
          if (o != null)
          {
            value = o.ToString();
          }

          if (!String.Equals(value,
            keyValues[i + 1],
            StringComparison.OrdinalIgnoreCase))
          {
            matches = false;
            break;
          }
        }
        if (matches)
        {
          return element;
        }
      }
    }
    return null;
  }
}

That wraps it up for today's post.

A Little Scripting Saved My Day (;-])

robert_mcmurray — Mon, 27 Jul 2009 21:55:00 GMT

I have mentioned in previous blog posts that I tend to write many of my blog posts and walkthroughs for IIS.NET based on code that I’ve written for myself, and today’s blog post is the story of how one of my samples saved my rear over this past weekend.

One of the servers that I manage is used to host web sites for several friends of mine. (It’s their hobby to have a web site and it’s my hobby to host it for them.) Anyway, sometime on Sunday someone let me know that one of my sites didn’t seem to be behaving correctly, so I browsed it with Internet Explorer and saw that I was getting an HTTP 503 error. I’ve seen this error when an application pool goes offline for some reason, so I didn’t panic – yet – because I knew that the web site was in a separate application pool. With that in mind, I browsed to a web site that is in a different application pool. Same thing – HTTP 503 error. This was beginning to concern me.

I logged into the web server and ran iisreset from a command-line – this threw the following error - and now I was really starting to become agitated:

CMD>iisreset Attempting stop... Internet services successfully stopped Attempting start... Restart attempt failed. The IIS Admin Service or the World Wide Web Publishing Service, or a service dependent on them failed to start. The service, or dependent services, may had an error during its startup or may be disabled. CMD>

I knew that the cause of the error should be in the Windows Event Viewer, so I opened the System log in Event Viewer and saw the following error:

Log Name: System

Source: Microsoft-Windows-WAS

Date: 7/26/2009 10:59:52 AM

Event ID: 5172

Task Category: None

Level: Error

Keywords: Classic

User: N/A

Computer: MYSERVER

Description: The Windows Process Activation Service encountered an error trying to read configuration data from file '\\?\C:\Windows\system32\inetsrv\config\applicationHost.config', line number '308'. The error message is: 'Configuration file is not well-formed XML'. The data field contains the error number.

Event Xml:

<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"> <System> <Provider Name="Microsoft-Windows-WAS" Guid="{4E616D65-6F6E-6D65-6973-526F62657274}" EventSourceName="WAS" /> <EventID Qualifiers="49152">5172</EventID> <Version>0</Version> <Level>2</Level> <Task>0</Task> <Opcode>0</Opcode> <Keywords>0x80000000000000</Keywords> <TimeCreated SystemTime="2009-07-26T17:59:52.000Z" /> <EventRecordID>32807</EventRecordID> <Correlation /> <Execution ProcessID="0" ThreadID="0" /> <Channel>System</Channel> <Computer>MYSERVER</Computer> <Security /> </System> <EventData> <Data Name="File">\\?\C:\Windows\system32\inetsrv\config\applicationHost.config</Data> <Data Name="LineNumber">308</Data> <Data Name="Error">Configuration file is not well-formed XML</Data> <Binary>0D000780</Binary> </EventData> </Event>

Log Name:	System
Source:	Microsoft-Windows-WAS
Date:	7/26/2009 10:59:52 AM
Event ID:	5172
Task Category:	None
Level:	Error
Keywords:	Classic
User:	N/A
Computer:	MYSERVER
Description:	The Windows Process Activation Service encountered an error trying to read configuration data from file '\\?\C:\Windows\system32\inetsrv\config\applicationHost.config', line number '308'. The error message is: 'Configuration file is not well-formed XML'. The data field contains the error number.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"> <System> <Provider Name="Microsoft-Windows-WAS" Guid="{4E616D65-6F6E-6D65-6973-526F62657274}" EventSourceName="WAS" /> <EventID Qualifiers="49152">5172</EventID> <Version>0</Version> <Level>2</Level> <Task>0</Task> <Opcode>0</Opcode> <Keywords>0x80000000000000</Keywords> <TimeCreated SystemTime="2009-07-26T17:59:52.000Z" /> <EventRecordID>32807</EventRecordID> <Correlation /> <Execution ProcessID="0" ThreadID="0" /> <Channel>System</Channel> <Computer>MYSERVER</Computer> <Security /> </System> <EventData> <Data Name="File">\\?\C:\Windows\system32\inetsrv\config\applicationHost.config</Data> <Data Name="LineNumber">308</Data> <Data Name="Error">Configuration file is not well-formed XML</Data> <Binary>0D000780</Binary> </EventData> </Event>

Now that I was armed with the file name and line number of the failure in my configuration settings, I was able to go straight to the source of the problem. (I love IIS 7's descriptive error messages - don't you?) Once I opened the file and jumped to the correct location, I saw several lines of unintelligible garbage. For reasons that are still unknown to me – my applicationHost.config file had become corrupted and IIS was dead in the water until I fixed the problem. I looked through the file and removed most of the garbage and saved the edited file to IIS – this got the web sites working, but only partially. Some necessary settings had obviously been removed while I was clearing all of out the unintelligible garbage, and it might take me a long time to discover what those settings were.

The next thing that I did was to take a look in my two readily-accessible backup drives; I have two external hard drives that keep a backup of the web server - one hard drive is directly plugged into the web server via a USB cable, and the other hard drive is plugged into a physically separate server that rotates drives with off-site storage on a monthly basis. The problem is, my weekly backups had just run, so the copy in each backup location had been overwritten with the corrupted version. (I’m going to have to rethink my backup strategy after this – but that’s another story.) The backup copy in my off-site storage location should be intact, but that copy would be a few weeks old so I would be missing some settings, and I would have to drive an hour or so round-trip in order to pick up the drive. This wasn’t an ideal solution – but it was definitely a feasible strategy.

It was at this point that I remembered that I had written following blog post some time ago:

Automating IIS 7 Backups
http://blogs.msdn.com/robert_mcmurray/archive/2008/03/08/automating-iis-7-backups.aspx

I wrote the script in that blog post for the server that I was currently managing, and because of this preventative measure I had dozens of backups going back several weeks to choose from. So I was able to quickly find a copy with no corruption and I restored that copy to my IIS config directory. At this point all of my web sites came online with all of their functionality. Having fixed the major issues, I used WinDiff to verify any settings that might have been changed between the restored copy and the corrupted copy.

So in conclusion, this story had a happy ending, and it left me with a few lessons learned:

You can never have too many backups
I need to rethink how I roll out my backup strategy with regard to using external hard drives
Writing cool scripts to automate your backups can save your rear end

That sums it up for today’s post. ;-]

Merging FTP Extensibility Walkthroughs

robert_mcmurray — Thu, 23 Jul 2009 19:43:00 GMT

Over the past several months I’ve been publishing a series of walkthroughs that use the extensibility in FTP 7.5 to create a several custom providers for a variety of scenarios, and today I posted my most recent entry in the series:

How to Use Managed Code to Create an FTP Authentication Provider using an XML Database

As a piece of behind-the-scenes trivia, some of these walkthroughs were based off custom providers that I had actually written for my FTP servers, and I used the samples that I wrote for some of the other walkthroughs as a starting point for custom providers that I currently use. With that in mind, I’d like to use today’s blog to talk about some of the ways that I combine what you see in a few of these walkthroughs into some useful scenarios.

One of the common providers that I use is a combination of the code that you see in these two walkthroughs:

Here's the way that I create the provider - I start with a single provider class that implements the IFtpHomeDirectoryProvider, IFtpAuthenticationProvider, and IFtpRoleProvider interfaces, and I create a few global variables that I'll use later.

public class FtpXmlAuthentication : BaseProvider,
    IFtpHomeDirectoryProvider,
    IFtpAuthenticationProvider,
    IFtpRoleProvider
{
    private string _XmlFileName;

    private string _HomeDirectory;

    private Dictionary<string, XmlUserData> _XmlUserData =
        new Dictionary<string, XmlUserData>(
            StringComparer.InvariantCultureIgnoreCase);
}

I add an Initialize() method to the class, where I load the values named xmlFileName and homeDirectory from the configuration settings.

protected override void Initialize(StringDictionary config)
{
    _XmlFileName = config["xmlFileName"];
    _HomeDirectory = config["homeDirectory"];
    if (string.IsNullOrEmpty(_XmlFileName))
    {
        throw new ArgumentException("Missing xmlFileName value in configuration.");
    }
}

I recycle the provider across a bunch of different FTP sites, and I don't always use the custom home directory feature, so my GetUserHomeDirectoryData() method has to accommodate for that. (Note: this means that your FTP site has to use a method of User Isolation other than "Custom". You can find more information about User Isolation on the FTP User Isolation Page.)

string IFtpHomeDirectoryProvider.GetUserHomeDirectoryData(
    string sessionId,
    string siteName,
    string userName)
{
    if (string.IsNullOrEmpty(_HomeDirectory))
    {
        throw new ArgumentException("Missing homeDirectory value in configuration.");
    }
    return _HomeDirectory;
}
(Note: While it may seem that I could throw the ArgumentException() in the Initialize() method, since I don't always need this value for providers that don't implement the home directory lookup it's best to throw the exception in the GetUserHomeDirectoryData() method.)

The last thing that I do for the provider is to copy the AuthenticateUser(), IsUserInRole(), ReadXmlDataStore(), GetInnerText() methods and XmlUserData class from the How to Use Managed Code to Create an FTP Authentication Provider using an XML Database walkthrough. This gives me a custom FTP authentication provider that provides user, role, and home directory lookups. This means the XML file for the provider registration has to vary a little from the walkthroughs in order to define settings for the xmlFileName and homeDirectory values. Here's an example of that that might look like:

<system.ftpServer>
    <providerDefinitions>
        <add name="ContosoXmlAuthentication" type="FtpXmlAuthentication,FtpXmlAuthentication,version=1.0.0.0,Culture=neutral,PublicKeyToken=426f62526f636b73" />
        <activation>
            <providerData name="ContosoXmlAuthentication">
                <add key="xmlFileName" value="C:\Inetpub\www.contoso.com\Users.xml" />
                <add key="homeDirectory" value="C:\Inetpub\www.contoso.com\ftproot" />
            </providerData>
        </activation>
    </providerDefinitions>

    <!-- Other XML goes here -->

</system.ftpServer>

The last thing that you need to do is to create the XML file that contains the usernames and passwords, which you can copy from the How to Use Managed Code to Create an FTP Authentication Provider using an XML Database walkthrough.

I use this provider on multiple FTP sites, so I simply re-register the provider under a different name and specify different values for the xmlFileName and homeDirectory values:

<system.ftpServer>
    <providerDefinitions>
        <add name="ContosoXmlAuthentication" type="FtpXmlAuthentication,FtpXmlAuthentication,version=1.0.0.0,Culture=neutral,PublicKeyToken=426f62526f636b73" />
        <add name="FabrikamXmlAuthentication" type="FtpXmlAuthentication,FtpXmlAuthentication,version=1.0.0.0,Culture=neutral,PublicKeyToken=426f62526f636b73" />
        <add name="WingTipToysXmlAuthentication" type="FtpXmlAuthentication,FtpXmlAuthentication,version=1.0.0.0,Culture=neutral,PublicKeyToken=426f62526f636b73" />
        <activation>
            <providerData name="ContosoXmlAuthentication">
                <add key="xmlFileName" value="C:\Inetpub\www.Contoso.com\Users.xml" />
                <add key="homeDirectory" value="C:\Inetpub\www.Contoso.com\ftproot" />
            </providerData>
            <providerData name="FabrikamXmlAuthentication">
                <add key="xmlFileName" value="C:\Inetpub\www.Fabrikam.com\Users.xml" />
                <add key="homeDirectory" value="C:\Inetpub\www.Fabrikam.com\ftproot" />
            </providerData>
            <providerData name="WingTipToysXmlAuthentication">
                <add key="xmlFileName" value="C:\Inetpub\www.WingTipToys.com\Users.xml" />
                <add key="homeDirectory" value="C:\Inetpub\www.WingTipToys.com\ftproot" />
            </providerData>
        </activation>
    </providerDefinitions>

    <!-- Other XML goes here -->

</system.ftpServer>

So in the end I have a provider that provides unique users, roles, and home directory for each FTP site. I point the FTP root to a path that is outside of the HTTP root, so my users can upload files for an application like a photo gallery that I provide them, but they can't access the actual ASP.NET files for the application. Since they're using accounts from the XML file, I don't have to hand out physical accounts on my servers or my domain. (The security-paranoid side of my personality really likes that.)

For some sites I use the XML file for ASP.NET membership by following the instructions in the How to use the Sample Read-Only XML Membership and Role Providers with IIS 7.0 walkthrough. In those cases, I move the XML file into the App_Data folder of the web site. Once again, since the FTP root is different than the HTTP root, this prevents any of my FTP users from accessing the XML file and making changes to it. (Although you could do that if you wanted to allow one of your users to update the list of FTP users for their site. But as you can imagine, the security-paranoid side of my personality really does not like that.)

All that being said, I hope that this helps you to get an idea for other ways that you can use some of the walkthroughs that I've been writing. I have several additional providers and walkthroughs that I’m working on for the IIS.NET web site, but I’ll keep those as a secret for now. ;-]

Using Apple's WebDavFS for Mac OS X with WebDAV on IIS 7

robert_mcmurray — Thu, 02 Jul 2009 05:19:00 GMT

Today's blog post needs to have a disclaimer right up front - I freely admit I'm not a Mac OS X expert, so I may not have everything 100% correct in this post. But I've seen a lot of questions on forums.iis.net that discuss using IIS WebDAV with Mac OS X, so I thought that I'd share a few of the things that I've noticed. Just the same, if I were writing a formal walkthrough I would have said something like, "Microsoft is not responsible for the behavior of Apple's Mac family of products. The information that is provided in this topic is provided to assist Mac OS X users that are connecting to IIS using WebDAV."

All that being said, here are the prerequisites for getting your environment together:

Your server needs to be running Windows Vista, Windows Server 2008, or Windows 7.
Your server needs to have Internet Information Services 7 and the WebDAV module installed. (Note: See the Installing and Configuring WebDAV on IIS 7.0 topic for more information.)
For best results, your Mac client needs to be running OS X version 10.4 or later.

Connecting to a WebDAV server using Mac OS X

In Mac OS X, open Finder.
Choose Go, then Connect To Server.
Enter the URL of the WebDAV server in Server Address. For example:
http://www.example.com/path/
Click Connect.

For more information, please see the following help topics that are available on Apple's Web site:

Troubleshooting WebDAV connections using Mac OS X

These are some of the issues that I've seen:

WebDavFS connections are read-only if WebDAV LOCKs are disabled on the server. Because of this:
- If you are using WebDAV 7.0 on IIS 7 you will not be able write files to the server; this is because WebDAV locks were not available in this release.
- If you are using WebDAV 7.5 on IIS 7 you will need to enable locks before you can write files to the server; this is because WebDAV locks are disabled by default. (Note: See the How to Use WebDAV Locks topic for more information.)
WebDavFS connections attempt to create files that may be blocked by IIS:
- "Desktop Services Store" (.DS_Store) files. (See http://en.wikipedia.org/wiki/.DS_Store for a description.)
- "Resource Fork" (e.g. "._filename.ext") files. (See http://en.wikipedia.org/wiki/Resource_fork for a description.)
Allowing unknown MIME types for WebDAV requests should allow these file types, and that setting is located under Web Settings action for the WebDAV Authoring Rules feature of IIS Manager.

For more information, see the following topics that are available on Apple's Web site:
- Mac OS X 10.4: How to prevent .DS_Store file creation over network connections
- Mac OS X 10.4: Windows Server 2003 IIS Web Server fails GET call on .DS_Store and ._ files

In Closing...

I have to reiterate that I'm not a Mac OS X expert, so this list is probably not all-inclusive, but it's helped to resolve some of the issues that I've seen.

Using LogParser with FTP 7.x Sessions

robert_mcmurray — Fri, 12 Jun 2009 05:15:00 GMT

One of the great features that we added to our W3C logging enhancements in FTP 7.0 and FTP 7.5 is the ability to track unique sessions, which are represented by GUIDs in a field that is named x-session. Because of this addition, you can do some interesting things with LogParser when analyzing your FTP logs.

The purpose of today's blog is to show a couple of the scripts that I use to analyze some of the session-based information that I'm interested in from time to time.

Using LogParser to Count FTP Sessions

Since the new FTP service tracks unique sessions, it is now possible to generate reports that show the number of unique FTP sessions you served by day. The following batch file accomplishes this in two parts: first it creates a temporary tab-separated-value file that contains the unique sessions by day, then it calculates the number of sessions by day and writes the totals to a tab-separated-value file that is named Sessions.tsv, which you can open using an application like Microsoft Excel.

@echo off set LOGPATTERN=u_ex*.log logparser.exe "SELECT DISTINCT date,x-session INTO '%~n0.tmp' from %LOGPATTERN%" -i:w3c -o:tsv -headers:ON if exist "%~n0.tmp" ( logparser.exe "SELECT date,COUNT(x-session) AS sessions INTO sessions.tsv FROM '%~n0.tmp' GROUP BY date" -i:tsv -o:tsv -headers:ON del "%~n0.tmp" ) set LOGPATTERN=

Using LogParser to Split FTP Log Files into Unique Session Activity Logs

I use the following script when I am testing various FTP scenarios that will split my FTP log files into individual log files that are named after the GUID for each session. (Note: Please bear in mind, this may generate a lot of log files, so use it sparingly!) You can then analyze the resulting log files to see the list of client activity that was unique to each session.

This script accomplishes its objective in two parts: first it creates a temporary tab-separated-value file with the list of unique session IDs, then it loops through each session ID and creates a W3C log file for each session's activity.

@echo off set LOGPATTERN=u_ex*.log logparser.exe "select distinct x-session into '%~n0.tmp' from '%LOGPATTERN%'" -i:w3c -o:tsv -headers:off if exist "%~n0.tmp" ( for /f "delims=|" %%a in (%~n0.tmp) do ( logparser.exe "select date,time,c-ip,cs-username,s-ip,s-port,cs-method,cs-uri-stem,sc-status,sc-win32-status,sc-substatus,x-session,x-fullpath into '%%a.log' from '%LOGPATTERN%' where x-session='%%a' order by date,time" -i:w3c -o:w3c ) del "%~n0.tmp" ) set LOGPATTERN=

That about does it for today - I hope this helps!