Robert McMurray's Blog [MSFT] : FTP

IIS Videos on Dr. Dobb's Web Site

robert_mcmurray — Sun, 27 Dec 2009 06:43:00 GMT

Like many people I know, I get a lot of email from various industry-related publications; some of the emails are spam, and the others are from computer magazines that I subscribe to. (And there are several of those. ;-] )

Anyway, I received an email the other day from the folks at Dr. Dobb’s, and it contained a link for a video on IIS Bitrate Throttling. It always piques my interest to see what others have to say about IIS, so I followed the link and discovered that the following page has several IIS-related videos listed:

http://www.ddj.com/ddjtvmsondemand.jhtml

Within the list of videos I found videos for FTP and WebDAV, so I couldn't resist watching those:

IIS FTP Publishing Service
http://ddj.com/ddjtvmsondemand.jhtml?bcpid=10177845001&bclid=9347349001&bctid=40439588001
IIS WebDAV
http://ddj.com/ddjtvmsondemand.jhtml?bcpid=10177845001&bclid=9347349001&bctid=40444327001

As I was watching those videos, the thought occurred to me that those videos were really familiar - but not in a déjà vu kind of way. As it turns out, those are the same videos that are on the IIS.NET web site:

IIS FTP Publishing Service
http://www.iis.net/expand/ftp
IIS WebDAV
http://www.iis.net/expand/WebDAV

Oh well - I was hoping for something new. But the fact remains that the folks at Dr. Dobb’s have collected a good number of IIS and Windows-related videos and put them all in one place, so it's a nice reference to have around.

IP Address Lookups for Addresses that were Blocked by FTP Dynamic IP Restrictions

robert_mcmurray — Mon, 30 Nov 2009 00:51:00 GMT

A few months ago I wrote a blog post that was titled "Merging FTP Extensibility Walkthroughs - Part 2", where I described how to merge my FTP Dynamic IP Restrictions Authentication Provider walkthrough with my FTP Provider that Sends an Email when Files are Uploaded walkthrough. The result of this code combination was a custom FTP authentication provider that provides support for dynamic IP restrictions that sends me an email every time a new IP address is blocked. I deployed this custom FTP authentication provider on one of my public-facing FTP sites when I wrote the blog post, and in the three months that I have been using that provider it has blocked 88 IP addresses.

Recently it seems that every day I receive a new email that another IP address has been blocked. If you've seen my LogParser and other log-related blog posts you'll realize that I'm kind of a log analysis geek, so I thought that it might be interesting to at least show something of a breakdown for the information that I've been seeing. For privacy reasons I can't distribute the list of IP addresses, so I'll just show a list of countries that I've seen in my dynamic IP restriction provider's SQL database since I deployed my provider.

The method that I employed for obtaining the location information was to write a couple of small scripts that dumped the list of IP addresses from my dynamic IP restriction provider's SQL database to a text file and then perform an IP address lookup to count the various countries from where the various hacking attempts have been originating. This gave me the following country-by-country information:

Country Total

UNITED STATES 23

CHINA 20

[unknown] 15

GERMANY 5

REPUBLIC OF KOREA 4

FRANCE 3

CZECH REPUBLIC 3

SPAIN 3

POLAND 2

BRAZIL 1

CYPRUS 1

SAUDI ARABIA 1

ARGENTINA 1

NETHERLANDS 1

JAPAN 1

THAILAND 1

SWITZERLAND 1

UKRAINE 1

INDIA 1

Total 88

Country	Total
UNITED STATES	23
CHINA	20
[unknown]	15
GERMANY	5
REPUBLIC OF KOREA	4
FRANCE	3
CZECH REPUBLIC	3
SPAIN	3
POLAND	2
BRAZIL	1
CYPRUS	1
SAUDI ARABIA	1
ARGENTINA	1
NETHERLANDS	1
JAPAN	1
THAILAND	1
SWITZERLAND	1
UKRAINE	1
INDIA	1
Total	88

I can't give out the name of the site that I used for the IP lookups, but there are several IP locator, geotargeting, or IP mapping sites available; a simple Internet search should provide you with a list of those sites. Once I choose one of those sites, writing a script to pull the IP addresses from my dynamic IP restriction provider's SQL database and perform the lookup was simple. The code that I wrote for the IP lookups was in VBScript, which I have included below. To use this script, you would need to populate a file named "IP_Addresses.txt" with the IP addresses to query, change the value of IP_LOOKUP_STUB in the example below to one of the available IP lookup URLs, then run the script, which will create a file named "IP_Addresses.log" with the resulting IP lookup information:

' ****************************************
Option Explicit

' ****************************************
Const MAX_ATTEMPTS   = 10    ' the number of times to keep retrying to get a file
Const FAILURE_SLEEP  = 1000  ' the number of milliseconds to sleep between retries
Const SUCCESS_SLEEP  = 1000  ' the number of milliseconds to sleep between successes
Const IP_LOOKUP_STUB = "http://www.example.com/page?ip="

' ****************************************
Dim objFSO
Dim objFile
Dim strLookupURL

' ****************************************
Set objFSO = WScript.CreateObject("Scripting.FileSystemObject")
Set objFile = objFSO.OpenTextFile("IP_Addresses.txt")
Do While Not objFile.AtEndOfStream
  strLookupURL = IP_LOOKUP_STUB & Trim(objFile.ReadLine)
  Call IpLookup(strLookupURL)
Loop
objFile.Close
WScript.Quit

' ****************************************
Function IpLookup(tmpURL)
  On Error Resume Next
  Dim intTempAttempt
  Dim blnTempStatus
  Dim objTempFSO
  Dim objTempFile
  Set objTempFSO = CreateObject("Scripting.FileSystemObject")
  blnTempStatus = False
  Dim objHTTP
  Set objHTTP = CreateObject("MSXML2.ServerXMLHTTP")
  For intTempAttempt = 1 To MAX_ATTEMPTS
    objHTTP.Open "GET", tmpURL, False
    objHTTP.Send
    If (objHTTP.Status = 200) Then
      If Err.Number = 0 Then
        blnTempStatus = True
        Set objTempFile = objTempFSO.OpenTextFile("IP_Addresses.log",8,True)
        objTempFile.WriteLine String(80,"=")
        objTempFile.WriteLine tmpURL
        objTempFile.WriteLine objHTTP.responseText
        objTempFile.Close
        Exit For
      End If
    End If
    WScript.Sleep FAILURE_SLEEP * intTempAttempt
  Next
  Set objHTTP = Nothing
  IpLookup = blnTempStatus
End Function

If you would rather use managed-code, a great example that you could use as a starting point is the KBSoft IP Locator example by Alexandr Golovanov, which also includes the URLs for a couple of IP locator services. Sometime in the future I think that I might write an ASP.NET application that replaces the VBScript code with a web-based interface, or I might add the IP locator logic to my dynamic IP restriction provider through an asynchronous function so that I have the information readily available whenever I want, but for the moment I'm content with my low-tech solution. ;-]

eWeek Reviews for IIS 7.5 and FTP 7.5

robert_mcmurray — Thu, 24 Sep 2009 01:45:00 GMT

One of my coworkers, Vijay Sen, just forwarded the following eWeek review of IIS 7.5 to me:

http://www.eweek.com/c/a/Windows/REVIEW-Microsoft-IIS-75-Improves-Management-Deployment-Options-822018/

The review was written by Jim Rapoza, and he said some great things about IIS 7.5, which ships with both Windows Server 2008 R2 and Windows 7 client. But what really made my day was the following things that he said about FTP 7.5:

Another welcome change in IIS 7.5 is the elevation of FTP as a full-fledged part of the server. In previous versions, setup and management of an FTP server in IIS were done pretty much separately from Web server management. In IIS 7.5, FTP administration is fully integrated into the IIS Management Console.

I found this to be a very good implementation of FTP, making it possible to quickly set up secure FTP servers and tie them to my Websites. Especially nice was the ability to easily use virtual host names for the FTP sites. All in all, the FTP implementation in IIS 7.5 is one of the best I’ve seen, even when compared with dedicated FTP server products.

It’s great to see all of our hard work being recognized!

My thanks once again to everyone on the FTP and IIS feature teams that helped make this version of the FTP service: Jaroslav, Emily, Daniel, Umer, Suditi, Ciprian, Jeong, Dave, Andrew, Carlos, Brian, Wade, Ulad, Nazim, Reagan, Claudia, Rick, Tim, Tobin, Kern, Jenny, Nitasha, Venkat, Vijay. (I hope that I didn't leave anyone out!)

FTP 7.5 Extensibility and Visual Studio Express Editions

robert_mcmurray — Thu, 17 Sep 2009 21:19:00 GMT

In earlier blog posts I have mentioned that I written the several walkthroughs to help developers get started writing providers for the FTP 7.5 service, all of which available on Microsoft's learn.iis.net Web site under the "Developing for FTP 7.5" section. In each of these walkthroughs I wrote the steps as if you were using Visual Studio 2008.

Following up on that, I received a great question yesterday from a customer, Paul Dowdle, who wondered if it was possible to write an extensibility provider for the FTP 7.5 service using one of the Visual Studio Express Editions. By way of coincidence, I used to install Visual C# Express Edition on my laptop when I was traveling around the world to speak at events like TechEd. I usually did this because the Express Edition took up less hard drive space than a full installation of Visual Studio, and I was only writing code in C# on my laptop.

To answer Paul's question, the short answer is - yes, you can use Visual Studio Express Editions to develop custom providers for the FTP 7.5 service, with perhaps a few small changes from my walkthroughs.

For example, if you look at my "How to Use Managed Code (C#) to Create a Simple FTP Authentication Provider" walkthrough, in the section that is titled "Step 1: Set up the Project Environment", there is an optional step 6 for adding a custom build event to register the DLL automatically in the Global Assembly Cache (GAC) on your development computer.

When I installed Microsoft Visual C# 2008 Express Edition on a new computer, I didn't have the "%VS90COMNTOOLS%" environment variable or the "vsvars32.bat" file, so I had to update the custom build event to the following:

net stop ftpsvc
"%ProgramFiles%\Microsoft SDKs\Windows\v6.0A\bin\gacutil.exe" /if "$(TargetPath)"
net start ftpsvc

Once I made that change, the rest of the walkthrough worked as written.

So, to reiterate my earlier statement - you can use Visual Studio Express Editions to develop custom providers for the FTP 7.5 service. My thanks to Paul for the great question!

Hiding your FTP Server Type and Preventing Unauthorized Access

robert_mcmurray — Thu, 10 Sep 2009 23:05:00 GMT

As evidenced by my How to Use Managed Code (C#) to Create an FTP Authentication Provider with Dynamic IP Restrictions walkthrough and my other FTP authentication extensibility walkthroughs, I spend a lot of time trying to find ways to prevent unauthorized access to my FTP server while still allowing valid users to have easy access to their site content. Today's blog discusses several of the ideas that I like to use on my FTP servers.

Preventing Unauthorized Access

To start things off, I globally disable FTP Basic Authentication on my server and I only use custom authentication providers. Since my FTP users do not have actual accounts on my server or my domain, that helps prevent access to my physical server. It was for this reason that I wrote the authentication provider that I discuss in my How to Use Managed Code (C#) to Create an FTP Authentication Provider using an XML Database walkthrough; I can hand out accounts with FTP access for sites that are hosted on my servers, but those accounts are stored in an XML file and not in the security accounts database on my server or my domain.

But there are other ways to help secure your FTP server. For example, I wrote an earlier blog post about Creating a Global Listener FTP Site. On my servers I like to create a global listener site and disable all of the built-in authentication methods.

Once that has been completed I add the custom authentication provider from my How to Use Managed Code (C#) to Create an FTP Authentication Provider with Dynamic IP Restrictions walkthrough, but I don't create any accounts in the database - I just want to log and block the IP addresses of would-be attackers that are attempting to brute force their way into my server. Configuring the authentication options is accomplished through the FTP Authentication feature in IIS Manager.

Ever since I wrote and deployed my dynamic IP address restriction authentication provider, it has faithfully blocked dozens of would-be attackers.

Hiding your FTP Server Type

The next approach that I take is "Security through Obscurity", where I hide the easily recognizable signatures that identify my FTP server. The first step is to suppress the default banner and add a custom FTP banner. This is accomplished through the FTP Messages feature in IIS Manager.

Next, switch your directory listing style to Unix format. All FTP clients that I have tested understand this format, but it hides the fact that your server can do MS-DOS listings. Configuring your directory listing style is accomplished through the FTP Directory Browsing feature in IIS Manager.

After implementing these changes, it might now resemble the following example FTP session when you log in with FTP:

CMD>ftp localhost

Connected to DESKTOP.domain.local.
220 Welcome to my server!
User (DESKTOP.domain.local:(none)): foobar
331 Password required for foobar.
Password:
230 User logged in.
ftp> dir
200 EPRT command successful.
125 Data connection already open; Transfer starting.
drwxrwxrwx   1 owner    group               0 Jan 22 11:45 App_Code
drwxrwxrwx   1 owner    group               0 Sep 10 10:32 App_Data
drwxrwxrwx   1 owner    group               0 Aug 17 13:35 bin
-rwxrwxrwx   1 owner    group             689 May  8  2008 iisstart.htm
-rwxrwxrwx   1 owner    group            2714 Jan 22 11:54 web.config
-rwxrwxrwx   1 owner    group          184946 May  8  2008 welcome.png
226 Transfer complete.
ftp: 418 bytes received in 0.02Seconds 83.00Kbytes/sec.
ftp> bye
221 Goodbye.

CMD>

The last step that I take is to block the FTP infrastructure commands SYST and FEAT on the global listener FTP site. These commands respectively display your operating system and list of FTP features. Blocking these commands helps to further obfuscate your server type from attackers.

Note: It is important that you block these two commands only on your global listener FTP site and not on your other FTP sites, because some FTP clients may depend on these commands in order to interact properly with your other FTP sites. That said, if you are working with a small set of FTP clients and none of them seem to have a problem with suppressing the SYST command, you could probably suppress that globally. Most clients that I have tested seem to have no problem with that, especially if you are using Unix directory listings.

In any event, you block the SYST and FEAT commands through the FTP Request Filtering feature in IIS Manager.

Now when you log in with FTP and attempt to use the SYST and FEAT commands, it should look like the following example FTP session:

CMD>ftp localhost

Connected to DESKTOP.domain.local.
220 Welcome to my server!
User (DESKTOP.domain.local:(none)): foobar
331 Password required for foobar.
Password:
230 User logged in.
ftp> quote SYST
500 'SYST': command not allowed.
ftp> quote FEAT
500 'FEAT': command not allowed.
ftp> bye
221 Goodbye.

CMD>

Summary

The steps in this blog are certainly not all-inclusive, and I keep coming up with other ideas as time goes on, but following these steps should help to prevent unauthorized access to your FTP service. All of the normal warnings about using strong passwords, changing your passwords often, using SSL whenever you can, and keeping your hotfixes up-to-date will always apply.

But that being said, one of the ways that I keep finding new things to block is by simply watching my FTP log files to see what attackers are trying to do. By using some of the techniques that I discuss in my Using LogParser with FTP 7.x Sessions blog, you can monitor your FTP log files in order to analyze what would-be attackers are trying to do.

Creating Recursive Directory Listing Files for FTP Clients

robert_mcmurray — Sat, 05 Sep 2009 22:39:00 GMT

One of the changes that we made in FTP 7.0 and FTP 7.5 was to remove recursive directory listings, which are commonly retrieved by typing "ls -lR" from a command-line FTP client, which should send a command like "NLST -lR" over FTP to the server. There were several reasons why we decided to remove recursive directory listings, but the main reason was simply to reduce CPU usage on the server; recursive directory listing requests take a lot of resources to fulfill. With that in mind, both FTP 7.0 and FTP 7.5 will ignore the recursive switch on directory requests.

That being said - quite often it's pretty handy to have a full directory listing from an FTP server. From a client perspective you could probably write script to automate an FTP client to create a recursive listing, but that's a lot of work. Back in my younger days when I ran FTP sites on Unix servers, I would always create two types of list files on my FTP servers for FTP clients to retrieve:

"ls-lr.txt" - I would create only one file of this type for my entire FTP server, which would go in the root of my FTP site and it would contain a full recursive listing of all files in my FTP site.
"00index.txt" - I would create one file of this type in each folder of my FTP site, and each index file would contain a listing of files and their descriptions for that folder.

Of course, anyone that's been around the Internet since the days before we had HTTP and the world-wide-web should know that I didn't come up with this idea on my own - I learned it from other FTP site administrators. (And anyone who remembers those days should also recognize those two files with a strange sense of nostalgia. 00index.txt files of course led to index.htm files when WWW sites came along later, but that's another story.)

In any event, as I continued to host FTP sites over the years I have written various scripts to create recursive directory listings, and I thought that one of my scripts might make a good blog post. With that in mind, here is a Windows Script Host file that I created, which I named "ls-lr.vbs", and this script will create a recursive directory listing for an FTP site. I choose the Unix directory listing style for this script since that's the format that I have used for years and the broader number of FTP clients and users should recognize it.

Option Explicit
On Error Resume Next

' Declare all variables.
Dim objArguments
Dim strBaseFolder
Dim objFSO
Dim objFile
Dim objFolder
Dim objSubFolder
Dim objSubFile
Dim lngFolderCount
Dim lngBaseCount
Set objArguments = WScript.Arguments

' Determine the number of command-line arguments.
Select Case objArguments.Count
  Case 0:
    strBaseFolder = WScript.ScriptFullName
    strBaseFolder = Left(strBaseFolder,InStrRev(strBaseFolder,"\"))
  Case 1:
    strBaseFolder = objArguments(0)
  Case Else:
    MsgBox "This script takes a single argument for the" & vbCrLf & _
      "starting directory, or specify no arguments" & vbCrLf & _
      "to use the current directory.", vbInformation
End Select

' Create a file system object.
Set objFSO = WScript.CreateObject("Scripting.FileSystemObject")

' Test if the base folder exists.
If Right(strBaseFolder,1) <> "\" Then strBaseFolder = strBaseFolder & "\"
If objFSO.FolderExists(strBaseFolder) = False Then
    MsgBox "The specified folder does not exist.", vbCritical
    WScript.Quit
End If

' Open the output file for the directory listing.
Set objFile = objFSO.CreateTextFile(strBaseFolder & "ls-lr.txt")
     
' Define the initial values for the folder counters.
lngFolderCount = 1
lngBaseCount = 0
  
' Dimension an array to hold the folder names.
ReDim strFolders(1)
  
' Store the root folder in the array.
strFolders(lngFolderCount) = strBaseFolder
    
' Loop while we still have folders to process.
While lngFolderCount <> lngBaseCount
  ' Set up a folder object to a base folder.
  Set objFolder = objFSO.GetFolder(strFolders(lngBaseCount+1))
  
  ' Output the folder name to the listing file.
  objFile.WriteLine vbCrLf & _
    Replace(Mid(strFolders(lngBaseCount+1),Len(strBaseFolder)),"\","/") & _
    vbCrLf
  
  ' Loop through the collection of subfolders for the base folder.
  For Each objSubFolder In objFolder.SubFolders
    ' Increment the folder count.
    lngFolderCount = lngFolderCount + 1
    ' Increase the array size
    ReDim Preserve strFolders(lngFolderCount)
    ' Store the folder name in the array.
    strFolders(lngFolderCount) = objSubFolder.Path
    ' Output the folder to the listing file.
    Call WriteEntry(objSubFolder)
  Next
  
  ' Loop through the collection of subfolders for the base folder.
  For Each objSubFile In objFolder.Files
    ' Output the file to the listing file.
    Call WriteEntry(objSubFile)
  Next
  ' Increment the base folder counter.
  lngBaseCount = lngBaseCount + 1
Wend

Sub WriteEntry(tmpObject)
  Dim tmpAttributes
  Dim tmpSize
  
  ' Test for a symbolic link.
  If (tmpObject.Attributes And 1024) Then
    tmpAttributes = "lrwxrwxrwx"
    tmpSize = 0
  ' Test for a directory.
  ElseIf (tmpObject.Attributes And 16) Then
    tmpAttributes = "drwxrwxrwx"
    tmpSize = 0
  ' Otherwise - it's a file.
  Else
    tmpAttributes = "-rwxrwxrwx"
    tmpSize = tmpObject.Size
  End If
  
  ' Test for a read-only object.
  If (tmpObject.Attributes And 1) Then
    tmpAttributes = Replace(tmpAttributes,"w","-")
  End If
  
  ' Write the list entry to the output file.
  objFile.WriteLine tmpAttributes & _
    "   1 owner    group " & _
    Right(String(15,Chr(32)) & CStr(tmpSize),15) & _
    " " & FormatDate(tmpObject.DateLastModified) & _
    " " & tmpObject.Name
End Sub

Function FormatDate(tmpDate)
  FormatDate = CStr(Year(tmpDate)) & _
    "-" & Right("00" & CStr(Month(tmpDate)),2) & _
    "-" & Right("00" & CStr(Day(tmpDate)),2)
End Function

To use the script, copy the code into Windows Notepad and save it to your computer as "ls-lr.vbs." If you double-click the script it will use the current folder to create a recursive folder listing, and if you run this script from a command-line it can take a single argument of a folder path, or you can pass no arguments to the script in order to use the current folder. In either case it will create a file named "ls-lr.txt" in the root of the destination folder that contains the recursive directory listing in Unix format.

For example, the following listing was created from a folder in my music collection on my desktop computer:

/

drwxrwxrwx   1 owner    group               0 2009-07-30 Against the Silence
dr-xr-xr-x   1 owner    group               0 2009-07-30 Collective
drwxrwxrwx   1 owner    group               0 2009-07-30 Speakeasy
-rwxrwxrwx   1 owner    group            2741 2009-09-05 ls-lr.txt

/Against the Silence

-rwxrwxrwx   1 owner    group         9386309 2009-07-30 01-Against the Silence.wma
-rwxrwxrwx   1 owner    group         3974684 2009-07-30 02-Side-Stage Syndrome.wma
-rwxrwxrwx   1 owner    group         7539014 2009-07-30 03-The Dash on my Headstone.wma
-rwxrwxrwx   1 owner    group         7244819 2009-07-30 04-Teeth Like Knives.wma
-rwxrwxrwx   1 owner    group         9910687 2009-07-30 05-The Band Played on.wma

/Collective

-r-xr-xr-x   1 owner    group         2767821 2009-03-05 At the Moment.wma
-r-xr-xr-x   1 owner    group         5259473 2009-03-05 Colt . 45.wma
-r-xr-xr-x   1 owner    group         2572687 2009-03-05 El Mariachi.wma
-r-xr-xr-x   1 owner    group         2395577 2009-03-05 Gold and Silver.wma
-r-xr-xr-x   1 owner    group         2269487 2009-03-05 Keep Waiting.wma
-r-xr-xr-x   1 owner    group         2050335 2009-03-05 Nighttown.wma
-r-xr-xr-x   1 owner    group         1458931 2009-03-05 Rise.wma
-r-xr-xr-x   1 owner    group         3140077 2009-03-05 Rivers Underneath.wma
-r-xr-xr-x   1 owner    group         2278489 2009-03-05 Sad Parade.wma
-r-xr-xr-x   1 owner    group         1909249 2009-03-05 The Hungry Wolf.wma
-r-xr-xr-x   1 owner    group         2467613 2009-03-05 Threshold.wma
-r-xr-xr-x   1 owner    group          795501 2009-03-05 Tranewreck.wma
-r-xr-xr-x   1 owner    group          417239 2009-03-05 Zzyzx.wma

/Speakeasy

-rwxrwxrwx   1 owner    group         4004604 2009-03-05 01-Minuteman.wma
-rwxrwxrwx   1 owner    group         6309752 2009-03-05 02-Sundown Motel.wma
-rwxrwxrwx   1 owner    group         5504122 2009-03-05 03-Keep Waiting.wma
-rwxrwxrwx   1 owner    group         2766262 2009-03-05 04-You Know How It Is.wma
-rwxrwxrwx   1 owner    group         7495952 2009-03-05 05-Rivers Underneath.wma
-rwxrwxrwx   1 owner    group         6294888 2009-03-05 06-Gold and Silver.wma
-rwxrwxrwx   1 owner    group         8062882 2009-03-05 07-Freefall.wma
-rwxrwxrwx   1 owner    group         4437286 2009-03-05 08-[Untitled].wma
-rwxrwxrwx   1 owner    group         3355592 2009-03-05 09-St. Eriksplan.wma
-rwxrwxrwx   1 owner    group         4966942 2009-03-05 10-Disquiet.wma
-rwxrwxrwx   1 owner    group         4788302 2009-03-05 11-Fascination Street.wma
-rwxrwxrwx   1 owner    group         7950944 2009-03-05 12-This Love.wma

(Note/Disclaimer/etc.: It may or may not be obvious that this listing is for music from the band Stavesacre, but just to be clear and avoid any RIAA entanglements - these files aren't actually hosted on any of my FTP sites; I used the script on my desktop computer to create a listing as an example for this blog.)

Customizing the Script Output

There are several customizations that you can do with this script, each of which has it's own benefits and drawbacks.

Adding Directory Sizes

It is trivial from a coding perspective to have the code calculate directory sizes since the Folder object that I use has as Size property, but it slows down the script exponentially to calculate that. That being said, if you're willing to take the performance hit, you can modify the highlighted section of the WriteEntry() function as follows:

  ' Test for a symbolic link.
  If (tmpObject.Attributes And 1024) Then
    tmpAttributes = "lrwxrwxrwx"
    tmpSize = 0
  ' Test for a directory.
  ElseIf (tmpObject.Attributes And 16) Then
    tmpAttributes = "drwxrwxrwx"
    tmpSize = tmpObject.Size
  ' Otherwise - it's a file.
  Else
    tmpAttributes = "-rwxrwxrwx"
    tmpSize = tmpObject.Size
  End If

This will insert the folder size into the output, but once again it will make the script much slower and take up considerably more CPU time to compute.

Uppercase Folder Names and Lowercase File Names

Since Windows is a case-insensitive operating system, you can easily choose to display all of your folder names in all uppercase characters and your file names in all lowercase characters without causing any client confusion. This can be accomplished by adding the first highlighted section and modifying the second highlighted section of the WriteEntry() function as follows:

  Dim tmpName
  
  ' Test for a directory.
  If (tmpObject.Attributes And 16) Then
    tmpName = UCase(tmpObject.Name)
  Else
    tmpName = LCase(tmpObject.Name)
  End If

  ' Write the list entry to the output file.
  objFile.WriteLine tmpAttributes & _
    "   1 owner    group " & _
    Right(String(15,Chr(32)) & CStr(tmpSize),15) & _
    " " & FormatDate(tmpObject.DateLastModified) & _
    " " & tmpName

Parting Thoughts

There are other customizations that you can easily make, such as creating a string array to sort the files and folders for each folder listing as a single list rather than listing folders first and files second as the currently script does. But that slows down the script way too much, and I prefer to see folders listed before files anyway. (Which is why I always use SET DIRCMD=/OGN for my command prompt sessions as well.)

Another easy customization would be to change the FormatDate() function to change the date format for the output file, which is why I used a function to do my date formatting. For example, you could easily use the FormatDate() function as a wrapper for VBScript's built-in FormatDateTime() function, and then use any of the vbGeneralDate, vbLongDate, vbShortDate, etc. options to specify the format. You can also use your own customized logic to return the date string, so you don't need to feel limited by my examples.

Another useful customization would be to compute the actual size for the resulting "ls-lr.txt" file and modify the output file to contain the correct file size. Currently the script in this blog adds an entry to the listing for the "ls-lr.txt" file, but that contains the temporary size of the output file as the script is running so it will seldom be accurate. (I usually run my script and update the "ls-lr.txt" file manually, but in some versions of this script I have had it ignore the "ls-lr.txt" file and remove it from the output listings.)

In closing, this script may be doing more than it might actually need to do by way of checking for symbolic links and read-only attributes, which our FTP service doesn’t actually do, but it was very easy to add that code and it runs just as fast either way.

Merging FTP Extensibility Walkthroughs - Part 2

robert_mcmurray — Thu, 20 Aug 2009 06:36:00 GMT

I had not intended to do a series on this subject when I wrote my original Merging FTP Extensibility Walkthroughs blog post, but I came up with a scenario that I felt was worth sharing. I recently posted the following walkthrough on the learn.iis.net web site:

How to Use Managed Code (C#) to Create an FTP Authentication Provider with Dynamic IP Restrictions

We have had many customer requests for a dynamic IP restrictions provider for the FTP server, and I wanted to get that out to customers as soon as I could. That being said, like several of my extensibility walkthroughs in the past, I wrote and tested the provider in that walkthrough on one of the servers that I manage. To show how effective it was, within the first couple of hours the provider had caught and blocked its first script kiddie who was attempting a brute force attack on my FTP server. Over the next few days the provider caught its next hacker, and over the past few weeks it has continued to do so.

That being said, I thought that it might be nice to know when an IP address was blocked, and I had already written the following walkthrough:

How to Use Managed Code (C#) to Create an FTP Provider that Sends an Email when Files are Uploaded

With that in mind, merging the two walkthroughs seemed like a simple thing to do.

Before continuing I need to reiterate the notice that I added to the dynamic IP restrictions walkthrough:

IMPORTANT NOTE: The latest version of the FTP 7.5 service must be installed in order to use the provider in this walkthrough. A version FTP 7.5 was released on August 3, 2009 that addressed an issue where the local and remote IP addresses in the IFtpLogProvider.Log() method were incorrect. Because of this, using an earlier version of the FTP service will prevent this provider from working.

With that warning out of the way, here are the steps that you need to follow in order to merge the two walkthroughs:

Step 1 - Create the project

Create a new C# project following all of the steps in the How to Use Managed Code (C#) to Create an FTP Authentication Provider with Dynamic IP Restrictions walkthrough.

Step 2 - Merge global variables

In this step you need to merge the global variables from the two walkthroughs. In my provider this looked like the following:

// Define the default values - these are only
// used if the configuration settings are not set.
const int defaultLogonAttempts = 5;
const int defaultFloodSeconds = 30;
const int defaultSmtpPort = 25;

// Define a connection string with no default.
private static string _connectionString;

// Initialize the private variables with the default values.
private static int _logonAttempts = defaultLogonAttempts;
private static int _floodSeconds = defaultFloodSeconds;

// Flag the application as uninitialized.
private static bool _initialized = false;

// Define a list that will contain the list of flagged sessions.
private static List<string> _flaggedSessions;

private string _smtpServerName;
private string _smtpFromAddress;
private string _smtpToAddress;
private int _smtpServerPort;

Step 3 - Merge the Initialize() methods

In this step you need to merge the Initialize() methods from the two walkthroughs so that all of the settings are retrieved from the IIS configuration file when the provider is loaded by the FTP service. In my provider this looked like the following:

// Initialize the provider.
protected override void Initialize(StringDictionary config)
{
    // Test if the application has already been initialized.
    if (_initialized == false)
    {
        // Create the flagged sessions list.
        _flaggedSessions = new List<string>();
        
        // Retrieve the connection string for the database connection.
        _connectionString = config["connectionString"];
        if (string.IsNullOrEmpty(_connectionString))
        {
            // Raise an exception if the connection string is missing or empty.
            throw new ArgumentException(
                "Missing connectionString value in configuration.");
        }
        else
        {
            // Determine whether the database is a Microsoft Access database.
            if (_connectionString.Contains("Microsoft.Jet"))
            {
                // Throw an exception if the database is a Microsoft Access database.
                throw new ProviderException("Microsoft Access databases are not supported.");
            }
        }
        
        // Retrieve the number of failures before an IP
        // address is locked out - or use the default value.
        if (int.TryParse(config["logonAttempts"], out _logonAttempts) == false)
        {
            // Set to the default if the number of logon attempts is not valid.
            _logonAttempts = defaultLogonAttempts;
        }
        
        // Retrieve the number of seconds for flood
        // prevention - or use the default value.
        if (int.TryParse(config["floodSeconds"], out _floodSeconds) == false)
        {
            // Set to the default if the number of logon attempts is not valid.
            _floodSeconds = defaultFloodSeconds;
        }
        
        // Test if the number is a positive integer and less than 10 minutes.
        if ((_floodSeconds <= 0) || (_floodSeconds > 600))
        {
            // Set to the default if the number of logon attempts is not valid.
            _floodSeconds = defaultFloodSeconds;
        }
        
        // Retrieve the email settings from configuration.
        _smtpServerName = config["smtpServerName"];
        _smtpFromAddress = config["smtpFromAddress"];
        _smtpToAddress = config["smtpToAddress"];
        
        // Detect and handle any mis-configured settings.
        if (!int.TryParse(config["smtpServerPort"], out _smtpServerPort))
        {
            _smtpServerPort = defaultSmtpPort;
        }
        if (string.IsNullOrEmpty(_smtpServerName))
        {
            throw new ArgumentException(
                "Missing smtpServerName value in configuration.");
        }
        if (string.IsNullOrEmpty(_smtpFromAddress))
        {
            throw new ArgumentException(
                "Missing smtpFromAddress value in configuration.");
        }
        if (string.IsNullOrEmpty(_smtpToAddress))
        {
            throw new ArgumentException(
                "Missing smtpToAddress value in configuration.");
        }
        
        // Initial garbage collection.
        GarbageCollection(true);
        
        // Flag the provider as initialized.
        _initialized = true;
    }
}

Step 4 - Add a SendEmail() method

For this step I copied some of my code from the email walkthrough and used it as the foundation for a new SendEmail() method that I added to the provider. In my provider this looked like the following:

private void SendEmail(string emailSubject, string emailMessage)
{
    // Create an SMTP message.
    SmtpClient smtpClient = new SmtpClient(_smtpServerName, _smtpServerPort);
    MailAddress mailFromAddress = new MailAddress(_smtpFromAddress);
    MailAddress mailToAddress = new MailAddress(_smtpToAddress);
    
    using (MailMessage mailMessage = new MailMessage(mailFromAddress, mailToAddress))
    {
        try
        {
            // Format the SMTP message as UTF8.
            mailMessage.BodyEncoding = Encoding.UTF8;
            // Add the subject.
            mailMessage.Subject = emailSubject;
            // Add the body.
            mailMessage.Body = emailMessage;
            // Send the email message.
            smtpClient.Send(mailMessage);
        }
        catch (SmtpException ex)
        {
            // Send an exception message to the debug
            // channel if the email fails to send.
            Debug.WriteLine(ex.Message);
        }
    }
}

Note: This uses the settings that you store in your IIS applicationHost.config file and are loaded by the Initialize() method.

Step 5 - Add email functionality to the BanAddress() method

In this step you add the functionality to send an email whenever an IP address is added to the list of banned IP addresses. In my provider this looked like the following:

// Mark an IP address as banned.
private void BanAddress(string ipAddress)
{
    // Check if the IP address is already banned.
    if (IsAddressBanned(ipAddress) == false)
    {
        // Ban the IP address if it is not already banned.
        InsertDataIntoTable("[BannedAddresses]",
            "[IPAddress]", "'" + ipAddress + "'");
        // Send an email for the banned address.
        SendEmail("Banned IP Address",
            "The IP address " + ipAddress + " was banned.");
    }
}

Step 6 - Methods that are not changed

I need to point out that there are several methods that require no changes. These methods are listed here for reference:

Dispose()
AuthenticateUser()
Log()
IsValidUser()
IsAddressBanned()
IsSessionFlagged()
FlagSession()
GarbageCollection
GetRecordCountByCriteria()
InsertDataIntoTable()
DeleteRecordsByCriteria()
ExecuteQuery()

Note: You could easily add the email functionality to the FlagSession() method so you will see when a banned IP address is trying to access your server, but depending on the number of sessions that are flagged on your server you might receive more emails than you really need.

Step 7 - Register the provider and configure your settings

In this last step you add the provider to your IIS configuration settings using the AppCmd utility, and you specify the values for the various settings that the provider requires:

cd %SystemRoot%\System32\Inetsrv AppCmd.exe set config -section:system.ftpServer/providerDefinitions /+"[name='FtpAddressRestrictionAuthentication',type='FtpAddressRestrictionAuthentication,FtpAddressRestrictionAuthentication,version=1.0.0.0,Culture=neutral,PublicKeyToken=426f62526f636b73']" /commit:apphost AppCmd.exe set config -section:system.ftpServer/providerDefinitions /+"activation.[name='FtpAddressRestrictionAuthentication']" /commit:apphost AppCmd.exe set config -section:system.ftpServer/providerDefinitions /+"activation.[name='FtpAddressRestrictionAuthentication'].[key='smtpServerName',value='localhost']" /commit:apphost AppCmd.exe set config -section:system.ftpServer/providerDefinitions /+"activation.[name='FtpAddressRestrictionAuthentication'].[key='smtpServerPort',value='25']" /commit:apphost AppCmd.exe set config -section:system.ftpServer/providerDefinitions /+"activation.[name='FtpAddressRestrictionAuthentication'].[key='smtpFromAddress',value='someone@contoso.com']" /commit:apphost AppCmd.exe set config -section:system.ftpServer/providerDefinitions /+"activation.[name='FtpAddressRestrictionAuthentication'].[key='smtpToAddress',value='someone@contoso.com']" /commit:apphost AppCmd.exe set config -section:system.ftpServer/providerDefinitions /+"activation.[name='FtpAddressRestrictionAuthentication'].[key='connectionString',value='Server=localhost;Database=FtpAuthentication;User ID=FtpLogin;Password=P@ssw0rd']" /commit:apphost AppCmd.exe set config -section:system.ftpServer/providerDefinitions /+"activation.[name='FtpAddressRestrictionAuthentication'].[key='logonAttempts',value='5']" /commit:apphost AppCmd.exe set config -section:system.ftpServer/providerDefinitions /+"activation.[name='FtpAddressRestrictionAuthentication'].[key='floodSeconds',value='30']" /commit:apphost

Note: You need to update the above syntax using the managed type information for your provider and the configuration settings for your SMTP server, email addresses, and database connection string.

Step 8 - Add the provider to a site

In this last step you add the provider to a site. If you were adding the provider to your Default Web Site that would look like the following:

AppCmd.exe set config -section:system.applicationHost/sites /"[name='Default Web Site'].ftpServer.security.authentication.basicAuthentication.enabled:False" /commit:apphost AppCmd.exe set config -section:system.applicationHost/sites /+"[name='Default Web Site'].ftpServer.security.authentication.customAuthentication.providers.[name='FtpAddressRestrictionAuthentication',enabled='True']" /commit:apphost AppCmd set site "Default Web Site" /+ftpServer.customFeatures.providers.[name='FtpAddressRestrictionAuthentication',enabled='true'] /commit:apphost

Summary

That wraps it up for today's post, and I hope you find it useful.

Automatically Creating Checksum Files for FTP Uploads

robert_mcmurray — Fri, 31 Jul 2009 04:01:00 GMT

I had a great question in the publishing forums on forums.iis.net, where someone was asking if FTP 7 supported the XCRC command. The short answer is that the XCRC command is not supported, but I came up with a way to create an FTP provider that supports something like it. Since it was a rather fun code sample to write, I thought that I'd turn it into a blog.

The sample FTP provider code in this blog post will automatically calculate an MD5 checksum from a file that is uploaded and store it in a file with a "*.MD5.TXT" file name extension. You can then compare the uploaded checksum with a local checksum on the client to verify the uploaded file's integrity.

There are a few points that I need to discuss before I present the code sample:

I chose to use MD5 because it is built-in to the .NET System.Security.Cryptography namespace and I often like to use MD5 for file checksums. I could just have easily implemented SHA1, SHA256, or any of the other built-in hashing algorithms. Unfortunately, CRC32 is not a built-in algorithm for .NET, but a quick search around the Internet yielded several CRC32 samples in C# from various developers, so if you specifically need the CRC32 algorithm you can find it pretty quickly and substitute it for MD5 in my example. (You can click here to search for examples.) You could go one step further and have your provider support multiple checksum algorithms, but that's going way outside the scope of this blog.
There are a couple of security considerations for this provider:
- The provider needs to calculate the path of the uploaded file, and to do so requires calling into the IIS configuration APIs. As I mention in the code remarks:
  - The FTP service will host the compiled assembly in the "Microsoft FTP Service Extensibility Host" COM+ package (DLLHOST.EXE), which runs by default as NETWORK SERVICE.
  - Also by default, the NETWORK SERVICE account does not have sufficient privileges to read the IIS configuration settings. As such, you must either grant READ permissions to NETWORK SERVICE for the IIS configuration files, or configure the COM+ package to run as a user that has at least READ access to the files in the InetSrv\config folder.
  - By default, the NETWORK SERVICE account may not have WRITE permission to the folder where your files are uploaded, so the checksum files cannot be written. As such, you will need to grant READ/WRITE access to the destination where the checksum files will be written.
- The above steps are not generally recommended practices; but if you choose to grant NETWORK SERVICE permission to the configuration files, the remarks section in the code sample provides the details that you need.
- Alternatively, you could skip the path lookup and always store the checksum files in a known location. This allows you to remove the MapSiteRootPath() and FindElement() methods from the code sample, and you need only grant the NETWORK SERVICE account permission for the known location.
The MapSiteRootPath() method in the provider sample calculates the path of the site's root, then uses the relative path of the uploaded file to compute the full path to the checksum file. This does not take into account any paths that include virtual directories; as such, you would need to accommodate for any virtual paths in your site's hierarchy. (That's too much code for this blog post.)
The provider defines a 1 GB constant for the maximum file size for computing checksums. I specified this value so that large files would not tie up your system's resources. You can increase or decrease that value, you could make that a parameter that is stored in the provider's settings, or you can remove the functionality completely. This provider runs synchronously, so larger files will obviously take more time. While it's outside the scope of this blog, you could implement some form of asynchronous functionality. (When discussing this provider with Daniel Vasquez Lopez, he suggested using MSMQ - but that's really going way beyond the scope of what I wanted to accomplish with this blog.)

All of that being said, this provider follows the same development path as the provider in my How to Use Managed Code (C#) to Create a Simple FTP Logging Provider walkthrough, so if you follow the steps in that walkthrough and substitute "FtpUploadChecksumDemo" every place that you see "FtpLoggingDemo" and add a reference to Microsoft.Web.Administration, you should have all of the steps that you need in order to use this provider.

So without further discussion, here's the code for the provider:

using System;
using System.Configuration.Provider;
using System.IO;
using System.Security.Cryptography;
using System.Text;
using Microsoft.Web.Administration;
using Microsoft.Web.FtpServer;

// NOTE: This code is provided "as-is" and comes with the following security
// considerations. The FTP service will host the compiled assembly in the
// "Microsoft FTP Service Extensibility Host" COM+ package (DLLHOST.EXE),
// which runs by default as NETWORK SERVICE. By default, this account does not
// have sufficient privileges to read the IIS configuration settings. As such,
// you must either grant READ permissions to NETWORK SERVICE for the configuration
// files, or configure the COM+ package to run as a user that has at least READ
// access to the files in the InetSrv\config folder and READ/WRITE access to the
// destination where the checksum file will be written. However, these are not
// generally recommended practices.
//
// If you choose to grant NETWORK SERVICE permission to the configuration files,
// the following three commands should accomplish the requisite permissions:
//
//  cacls "%SystemRoot%\System32\inetsrv\config" /G "Network Service":R /E
//  cacls "%SystemRoot%\System32\inetsrv\config\redirection.config" /G "Network Service":R /E
//  cacls "%SystemRoot%\System32\inetsrv\config\applicationHost.config" /G "Network Service":R /E
//
// NOTE: You will need to do something similar for your content directory so that
// the checksum files can be created.

public sealed class FtpUploadChecksumDemo : BaseProvider, IFtpLogProvider
{
  // Implement the logging method.
  void IFtpLogProvider.Log(FtpLogEntry loggingParameters)
  {
    // Test for a successful file upload operation.
    if ((loggingParameters.Command == "STOR") && 
      (loggingParameters.FtpStatus == 226))
    {
      try
      {
        // Define a 1GB maximum length - to prevent system hogging.
        const long maxLength = 0x3fffffff;

        // Map the path to the site root.
        string fullPath = MapSiteRootPath(loggingParameters.SiteName);
        // Append the relative path of the uploaded file.
        fullPath += loggingParameters.FullPath;
        // Expand any environment variables.
        fullPath = Environment.ExpandEnvironmentVariables(fullPath);
        // Convert forward slashes to back slashes
        fullPath = fullPath.Replace(@"/", @"\");

        // Open the uploaded file to create a CRC.
        using (FileStream input = File.Open(
          fullPath,
          FileMode.Open,
          FileAccess.Read,
          FileShare.Read))
        {
          // Test the input file length.
          if (input.Length > maxLength)
          {
            // Throw an execption if the file is too big.
            throw new ProviderException(
              String.Format("Input file is too large: {0}",
              input.Length.ToString()));
          }
          else
          {
            // Open the hash file for output.
            using (StreamWriter output = new StreamWriter(
              fullPath + ".MD5.txt",
              false))
            {
              // Create an MD5 object.
              MD5 md5 = MD5.Create();
              // Retrieve the hash byte array.
              byte[] byteArray = md5.ComputeHash(input);
              // Create a new string builder for the ASCII hash string.
              StringBuilder stringBuilder =
                new StringBuilder(byteArray.Length * 2);
              // Loop through the hash.
              foreach (byte byteMember in byteArray)
              {
                // Append each ASCII hex byte to the hash string.
                stringBuilder.AppendFormat("{0:x2}", byteMember);
              }
              // Write the hash string to the output file.
              output.Write(stringBuilder);
            }
          }
        }
      }
      catch(Exception ex)
      {
        throw new ProviderException(ex.Message);
      }
    }
  }

  // This method is almost 100% from scripts that were created
  // by the IIS Manager Configuration Editor admin pack tool.
  private static string MapSiteRootPath(string siteName)
  {
    try
    {
      using (ServerManager serverManager = new ServerManager())
      {
        Configuration config =
          serverManager.GetApplicationHostConfiguration();
        ConfigurationSection sitesSection =
          config.GetSection("system.applicationHost/sites");
        ConfigurationElementCollection sitesCollection =
          sitesSection.GetCollection();
        ConfigurationElement siteElement =
          FindElement(sitesCollection, "site", "name", siteName);
        if (siteElement == null)
        {
          throw new InvalidOperationException("Element not found!");
        }
        else
        {
          ConfigurationElementCollection siteCollection =
            siteElement.GetCollection();
          ConfigurationElement applicationElement =
            FindElement(siteCollection,
            "application",
            "path", @"/");
          if (applicationElement == null)
          {
            throw new InvalidOperationException("Element not found!");
          }
          else
          {
            ConfigurationElementCollection applicationCollection =
              applicationElement.GetCollection();
            ConfigurationElement virtualDirectoryElement =
              FindElement(applicationCollection,
              "virtualDirectory",
              "path", @"/");
            if (virtualDirectoryElement == null)
            {
              throw new InvalidOperationException("Element not found!");
            }
            else
            {
              return virtualDirectoryElement["physicalPath"].ToString();
            }
          }
        }
      }
    }
    catch (Exception ex)
    {
      throw new ProviderException(ex.Message);
    }
  }

  // This method is almost 100% from scripts that were created
  // by the IIS Manager Configuration Editor admin pack tool.
  private static ConfigurationElement FindElement(
    ConfigurationElementCollection collection,
    string elementTagName,
    params string[] keyValues)
  {
    foreach (ConfigurationElement element in collection)
    {
      if (String.Equals(element.ElementTagName,
        elementTagName,
        StringComparison.OrdinalIgnoreCase))
      {
        bool matches = true;

        for (int i = 0; i < keyValues.Length; i += 2)
        {
          object o = element.GetAttributeValue(keyValues[i]);
          string value = null;
          if (o != null)
          {
            value = o.ToString();
          }

          if (!String.Equals(value,
            keyValues[i + 1],
            StringComparison.OrdinalIgnoreCase))
          {
            matches = false;
            break;
          }
        }
        if (matches)
        {
          return element;
        }
      }
    }
    return null;
  }
}

That wraps it up for today's post.

Merging FTP Extensibility Walkthroughs

robert_mcmurray — Thu, 23 Jul 2009 19:43:00 GMT

Over the past several months I’ve been publishing a series of walkthroughs that use the extensibility in FTP 7.5 to create a several custom providers for a variety of scenarios, and today I posted my most recent entry in the series:

How to Use Managed Code to Create an FTP Authentication Provider using an XML Database

As a piece of behind-the-scenes trivia, some of these walkthroughs were based off custom providers that I had actually written for my FTP servers, and I used the samples that I wrote for some of the other walkthroughs as a starting point for custom providers that I currently use. With that in mind, I’d like to use today’s blog to talk about some of the ways that I combine what you see in a few of these walkthroughs into some useful scenarios.

One of the common providers that I use is a combination of the code that you see in these two walkthroughs:

Here's the way that I create the provider - I start with a single provider class that implements the IFtpHomeDirectoryProvider, IFtpAuthenticationProvider, and IFtpRoleProvider interfaces, and I create a few global variables that I'll use later.

public class FtpXmlAuthentication : BaseProvider,
    IFtpHomeDirectoryProvider,
    IFtpAuthenticationProvider,
    IFtpRoleProvider
{
    private string _XmlFileName;

    private string _HomeDirectory;

    private Dictionary<string, XmlUserData> _XmlUserData =
        new Dictionary<string, XmlUserData>(
            StringComparer.InvariantCultureIgnoreCase);
}

I add an Initialize() method to the class, where I load the values named xmlFileName and homeDirectory from the configuration settings.

protected override void Initialize(StringDictionary config)
{
    _XmlFileName = config["xmlFileName"];
    _HomeDirectory = config["homeDirectory"];
    if (string.IsNullOrEmpty(_XmlFileName))
    {
        throw new ArgumentException("Missing xmlFileName value in configuration.");
    }
}

I recycle the provider across a bunch of different FTP sites, and I don't always use the custom home directory feature, so my GetUserHomeDirectoryData() method has to accommodate for that. (Note: this means that your FTP site has to use a method of User Isolation other than "Custom". You can find more information about User Isolation on the FTP User Isolation Page.)

string IFtpHomeDirectoryProvider.GetUserHomeDirectoryData(
    string sessionId,
    string siteName,
    string userName)
{
    if (string.IsNullOrEmpty(_HomeDirectory))
    {
        throw new ArgumentException("Missing homeDirectory value in configuration.");
    }
    return _HomeDirectory;
}
(Note: While it may seem that I could throw the ArgumentException() in the Initialize() method, since I don't always need this value for providers that don't implement the home directory lookup it's best to throw the exception in the GetUserHomeDirectoryData() method.)

The last thing that I do for the provider is to copy the AuthenticateUser(), IsUserInRole(), ReadXmlDataStore(), GetInnerText() methods and XmlUserData class from the How to Use Managed Code to Create an FTP Authentication Provider using an XML Database walkthrough. This gives me a custom FTP authentication provider that provides user, role, and home directory lookups. This means the XML file for the provider registration has to vary a little from the walkthroughs in order to define settings for the xmlFileName and homeDirectory values. Here's an example of that that might look like:

<system.ftpServer>
    <providerDefinitions>
        <add name="ContosoXmlAuthentication" type="FtpXmlAuthentication,FtpXmlAuthentication,version=1.0.0.0,Culture=neutral,PublicKeyToken=426f62526f636b73" />
        <activation>
            <providerData name="ContosoXmlAuthentication">
                <add key="xmlFileName" value="C:\Inetpub\www.contoso.com\Users.xml" />
                <add key="homeDirectory" value="C:\Inetpub\www.contoso.com\ftproot" />
            </providerData>
        </activation>
    </providerDefinitions>

    <!-- Other XML goes here -->

</system.ftpServer>

The last thing that you need to do is to create the XML file that contains the usernames and passwords, which you can copy from the How to Use Managed Code to Create an FTP Authentication Provider using an XML Database walkthrough.

I use this provider on multiple FTP sites, so I simply re-register the provider under a different name and specify different values for the xmlFileName and homeDirectory values:

<system.ftpServer>
    <providerDefinitions>
        <add name="ContosoXmlAuthentication" type="FtpXmlAuthentication,FtpXmlAuthentication,version=1.0.0.0,Culture=neutral,PublicKeyToken=426f62526f636b73" />
        <add name="FabrikamXmlAuthentication" type="FtpXmlAuthentication,FtpXmlAuthentication,version=1.0.0.0,Culture=neutral,PublicKeyToken=426f62526f636b73" />
        <add name="WingTipToysXmlAuthentication" type="FtpXmlAuthentication,FtpXmlAuthentication,version=1.0.0.0,Culture=neutral,PublicKeyToken=426f62526f636b73" />
        <activation>
            <providerData name="ContosoXmlAuthentication">
                <add key="xmlFileName" value="C:\Inetpub\www.Contoso.com\Users.xml" />
                <add key="homeDirectory" value="C:\Inetpub\www.Contoso.com\ftproot" />
            </providerData>
            <providerData name="FabrikamXmlAuthentication">
                <add key="xmlFileName" value="C:\Inetpub\www.Fabrikam.com\Users.xml" />
                <add key="homeDirectory" value="C:\Inetpub\www.Fabrikam.com\ftproot" />
            </providerData>
            <providerData name="WingTipToysXmlAuthentication">
                <add key="xmlFileName" value="C:\Inetpub\www.WingTipToys.com\Users.xml" />
                <add key="homeDirectory" value="C:\Inetpub\www.WingTipToys.com\ftproot" />
            </providerData>
        </activation>
    </providerDefinitions>

    <!-- Other XML goes here -->

</system.ftpServer>

So in the end I have a provider that provides unique users, roles, and home directory for each FTP site. I point the FTP root to a path that is outside of the HTTP root, so my users can upload files for an application like a photo gallery that I provide them, but they can't access the actual ASP.NET files for the application. Since they're using accounts from the XML file, I don't have to hand out physical accounts on my servers or my domain. (The security-paranoid side of my personality really likes that.)

For some sites I use the XML file for ASP.NET membership by following the instructions in the How to use the Sample Read-Only XML Membership and Role Providers with IIS 7.0 walkthrough. In those cases, I move the XML file into the App_Data folder of the web site. Once again, since the FTP root is different than the HTTP root, this prevents any of my FTP users from accessing the XML file and making changes to it. (Although you could do that if you wanted to allow one of your users to update the list of FTP users for their site. But as you can imagine, the security-paranoid side of my personality really does not like that.)

All that being said, I hope that this helps you to get an idea for other ways that you can use some of the walkthroughs that I've been writing. I have several additional providers and walkthroughs that I’m working on for the IIS.NET web site, but I’ll keep those as a secret for now. ;-]

Using LogParser with FTP 7.x Sessions

robert_mcmurray — Fri, 12 Jun 2009 05:15:00 GMT

One of the great features that we added to our W3C logging enhancements in FTP 7.0 and FTP 7.5 is the ability to track unique sessions, which are represented by GUIDs in a field that is named x-session. Because of this addition, you can do some interesting things with LogParser when analyzing your FTP logs.

The purpose of today's blog is to show a couple of the scripts that I use to analyze some of the session-based information that I'm interested in from time to time.

Using LogParser to Count FTP Sessions

Since the new FTP service tracks unique sessions, it is now possible to generate reports that show the number of unique FTP sessions you served by day. The following batch file accomplishes this in two parts: first it creates a temporary tab-separated-value file that contains the unique sessions by day, then it calculates the number of sessions by day and writes the totals to a tab-separated-value file that is named Sessions.tsv, which you can open using an application like Microsoft Excel.

@echo off set LOGPATTERN=u_ex*.log logparser.exe "SELECT DISTINCT date,x-session INTO '%~n0.tmp' from %LOGPATTERN%" -i:w3c -o:tsv -headers:ON if exist "%~n0.tmp" ( logparser.exe "SELECT date,COUNT(x-session) AS sessions INTO sessions.tsv FROM '%~n0.tmp' GROUP BY date" -i:tsv -o:tsv -headers:ON del "%~n0.tmp" ) set LOGPATTERN=

Using LogParser to Split FTP Log Files into Unique Session Activity Logs

I use the following script when I am testing various FTP scenarios that will split my FTP log files into individual log files that are named after the GUID for each session. (Note: Please bear in mind, this may generate a lot of log files, so use it sparingly!) You can then analyze the resulting log files to see the list of client activity that was unique to each session.

This script accomplishes its objective in two parts: first it creates a temporary tab-separated-value file with the list of unique session IDs, then it loops through each session ID and creates a W3C log file for each session's activity.

@echo off set LOGPATTERN=u_ex*.log logparser.exe "select distinct x-session into '%~n0.tmp' from '%LOGPATTERN%'" -i:w3c -o:tsv -headers:off if exist "%~n0.tmp" ( for /f "delims=|" %%a in (%~n0.tmp) do ( logparser.exe "select date,time,c-ip,cs-username,s-ip,s-port,cs-method,cs-uri-stem,sc-status,sc-win32-status,sc-substatus,x-session,x-fullpath into '%%a.log' from '%LOGPATTERN%' where x-session='%%a' order by date,time" -i:w3c -o:w3c ) del "%~n0.tmp" ) set LOGPATTERN=

That about does it for today - I hope this helps!

FTP Clients - Part 6: Core FTP LE

robert_mcmurray — Sat, 02 May 2009 02:10:00 GMT

For this installment in my series about FTP Clients, I'd like to take a look at the Core FTP client. For this blog post I used Core FTP Lite Edition (LE) version 1.3c (build 1447) and version 2.1 (build 1603), although all of my screen shots are from version 2.1. Core FTP is available from the following URL:

http://www.coreftp.com/

At the time of this blog post, Core FTP provides the LE for free and charges a small fee for a professional version.

Like most graphical FTP clients, the Core FTP LE user interface is pretty easy to use and rather straight-forward - you have separate windows for your local and remote files/folders, as well as a logging window that lists the FTP commands that are sent and the FTP server's responses:

Core FTP LE has a great Site Manager feature, which allows you to store commonly-used connections to FTP sites:

Clicking on the Advanced button gives you a great deal of additional configuration settings, and I'll say more about that later:

Command-Line Support

This is one of my favorite Core FTP LE features: command-line support. Yes - I'm a geek - and I like being able to script things and run batch jobs to automate whatever I can, so command-line support is always a plus for me. That said, the interface for the Core FTP LE command-line client is not an interactive experience like you get with the built-in Windows FTP.EXE or MOVEit Freely command-line clients. The Core FTP LE command-line client is provided as via the Corecmd.exe file that is installed in the main the Core FTP LE application directory, and is used for a single FTP operation like GET or PUT - although you can pass the name of a script file to execute several commands before/after logging in or before/after a file transfer.

So my final judgment is that the Core FTP LE client doesn't have great command-line support, but it's still really nice to have.

Using FTP over SSL (FTPS)

The Core FTP LE client supports both Implicit and Explicit FTPS, so the choice is up to you which method to use. When creating a connection to a server, Core FTP LE has three FTP options that you can use with FTP7:

AUTH SSL
AUTH TLS
FTPS (SSL DIRECT)

It's important to choose this option correctly, otherwise you will run into problems when trying access a site using FTPS. If you'll recall from my "FTP Clients - Part 2: Explicit FTPS versus Implicit FTPS" and my other FTP client blog posts, Explicit FTPS allows the client to initiate SSL/TLS whenever it wants, but for most FTP clients that will be when logging in to your FTP site, and in that regard it may almost seem like Implicit FTPS, but behind the scenes the FTP client and server are communicating differently.

In the case of FTP7, the following rules apply:

If you enable FTPS in FTP7 and you assign the FTP site to port 990, you are using Implicit FTPS - Core FTP LE refers to this as FTPS (SSL DIRECT). (Note: make sure that you configure your FTP client to connect on port 990.)
If you enable FTPS in FTP7 and you assign the FTP site to any port other than port 990, you are using Explicit FTPS - Core FTP LE allows you to configure your connection to use AUTH SSL or AUTH TLS for the explicit connection.

The type of FTPS is specified on the Connection drop-down menu:

Once you have chosen an FTPS connection, the Core FTP LE client offers you additional options where you can customize which parts of the session will be encrypted:

You can combine the Core FTP SSL options with the advanced SSL policies for your FTP7 sites to customize your security level:

Using FTP Virtual Hosts

Because Core FTP LE's site manager allows you to specify the virtual host name as part of the user credentials, Core FTP LE works great with FTP7's virtual host names. All that you need to do is use the "ftp.example.com|username" syntax when specifying your username, and when you connect to the FTP7 server it will route your requests to the correct FTP virtual host site.

Using True FTP Hosts

A really great feature of Core FTP LE is the ability to send pre-login commands, and since this feature allows you to enter custom commands you can specify the actual FTP HOST command as part of your login:

This is a tremendous feature if you're hosting multiple FTP sites on the same IP address, and gives Core FTP LE some of the best support for true FTP HOSTs.

Scorecard for Core FTP LE

That wraps it up for our quick round-trip for some of Core FTP LE's features, and here's the scorecard results:

Client Name	Directory Browsing	Explicit FTPS	Implicit FTPS	Virtual Hosts	True HOSTs
Core FTP LE 1.3	Rich	Y	Y	Y	Y ¹
Core FTP LE 2.1	Rich	Y	Y	Y	Y ¹
¹ As noted earlier, true FTP HOSTs are available in Site Manager using pre-login commands.

Note: Keeping with my standard disclaimer, there are a great number of additional features that Core FTP LE provides - and I just focused on the topic areas that apply to FTP7.

BLOG - FTP 7.5 Service Extensibility References

robert_mcmurray — Fri, 24 Apr 2009 03:23:00 GMT

As I pointed out in my recent blog post that was titled "FTP 7.5 and WebDAV 7.5 have been released", one of the great new features of the FTP 7.5 service is extensibility. In that blog post I mentioned that I wrote the following walkthroughs to help developers get started writing providers for the FTP 7.5 service, and these walkthroughs are all available on Microsoft's learn.iis.net Web site:

We have also recently published the FTP Service Extensibility Reference on Microsoft's MSDN Web site, and here is a list of all the reference topics that we have written for FTP 7.5 service extensibility:

I hope this helps!

FTP 7.5 and WebDAV 7.5 have been released!

robert_mcmurray — Wed, 18 Mar 2009 10:29:00 GMT

Over the next few days you're going to hear a lot of details about many of the great new IIS extensions that the IIS feature team is releasing for the 2009 MIX Conference in Las Vegas later today. I don't want to spoil any surprises by talking about anyone else’s feature areas, but I’m about to board a plane to head out on vacation (to Peru!) and I'm not taking a computer with me (believe it or not!) so I thought that I’d take a moment to highlight just a few of the features that are in the FTP 7.5 and WebDAV 7.5 releases.

FTP 7.5

One of the great new features in FTP 7.5 is extensibility. We had some extensibility features that were partially implemented in FTP 7.0, and we used those for the ASP.NET and IIS Manager authentication providers, but FTP extensibility was not officially supported in the 7.0 release. With FTP 7.5 extensibility is fully supported, so you can now write providers that implement custom functionality for authentication/authorization, home directory lookups, and logging.

To help developers get started writing providers for FTP 7.5, I wrote the following walkthroughs that are available on Microsoft's learn.iis.net web site:

Another highlight in FTP 7.5 is the addition of a user interface for the FTP Request Filtering features. We shipped request filtering with FTP 7.0 while the request filtering user interface was still in development, and the 7.5 version seemed like a great time to release it.

For those of you that are unfamiliar with FTP request filtering, it allows you to add rules that allow or deny specific file extensions, hidden segments, URL sequences, and even FTP commands.

For more information about the features in the new FTP service, see the following page on the http://www.iis.net/ web site:

http://www.iis.net/extensions/FTP

WebDAV 7.5

One of the big changes in WebDAV 7.5 is the inclusion of WebDAV locks, which are implemented through a simple locking mechanism. Our lock implementation was still in development when we shipped the WebDAV 7.0 release, and this release should help publishing scenarios where WebDAV locks are required.

To help you get started using locks with WebDAV 7.5, I wrote the following walkthrough that is available on Microsoft's learn.iis.net web site:

How to Use WebDAV Locks

For more information about the features in the new WebDAV module, see the following page on the http://www.iis.net/ web site:

http://www.iis.net/extensions/WebDAV

In Closing...

So that about wraps it up for some of the major highlights for FTP and WebDAV; for news about everything else that's coming out for IIS, watch the news items on the http://www.iis.net/ home page!

FTP Clients - Part 5: MOVEit Freely Command-Line Secure FTP Client

robert_mcmurray — Wed, 07 Jan 2009 09:10:00 GMT

For this installment in my series about FTP Clients, I'd like to take a look at the MOVEit Freely Command-Line Secure FTP Client ("FTPS.EXE") from Ipswitch. For this blog post I used MOVEit Freely FTP Client version 5.0.0.0, and it is available from the following URL:

http://www.ipswitchft.com/products/moveit/client/freely/

At the time of this blog post, Ipswitch is providing the MOVEit Freely FTP client for free, although you are required to fill out a registration page with a short questionnaire. For more information on the license for the MOVEit Freely command-line FTP client, please see Ipswitch's web site.

If you're like me and you like to script a lot of batch jobs on your servers, the MOVEit Freely command-line FTP client can be quite handy. The command set for the MOVEit Freely FTP client is a greatly-enhanced superset of the commands that are available with the command-line FTP.EXE client that is built-in to Windows, with added features that make additional functionality possible, such as SSL, passive FTP, resumable downloads, etc. There is a manual available with the MOVEit Freely FTP client, and I highly recommend using the manual as a reference when writing automation scripts because there are a lot of options that are available to you.

Active and Passive FTP

One of the great things about the MOVEit Freely command-line FTP client is the ability to use either Passive or Active connections, and you can switch between the two connection types using the "passive" command in the FTP session. This helps immensely when working with firewalls and such. The following example shows what that might look like:

CMD>ftps.exe ftp.example.com

220 Microsoft FTP Service
Connected to ftp.example.com.
User: administrator
331 Password required for administrator.
Password: ********
230 User logged in.
ftp> passive
Passive mode  On .
ftp> put foobar.txt
227 Entering Passive Mode (192,168,0,1,224,39).
150 Opening ASCII mode data connection.
226 Transfer complete.
ftp: 8 bytes sent in 0.06Seconds 0.13Kbytes/sec.
ftp> passive
Passive mode  Off .
ftp> put foobar.txt
200 PORT command successful.
125 Data connection already open; Transfer starting.
226 Transfer complete.
ftp: 8 bytes sent in 0.01Seconds 0.78Kbytes/sec.
ftp> bye
221 Goodbye.

CMD>

Using FTP over SSL (FTPS)

The MOVEit Freely command-line FTP client supports both Implicit and Explicit FTPS, so the choice is up to you which one to use, but I generally use Explicit FTPS since Implicit FTPS should be considered obsolete. The SSL mode is specified using the "-e:" parameter on the command-line, and the 5.0.0.0 version of the MOVEit Freely command-line FTP supports the following values for that parameter:

Parameter FTPS Mode Description

off n/a Specifies that no encryption will be used on either the control channel or data channel.
Note: This is the default behavior.

on Explicit Specifies that both the control channel and data channel will use encryption over an explicit FTPS connection.
Notes:

This uses the AUTH TLS, PBSZ 0, and PROT P commands when establishing a connection.

You can use "prot on" and "prot off" to specify whether encryption will be used. (See Note 1 below.)

on-ccc Explicit Specifies the control channel will use encryption over an explicit FTPS connection during login, but the control channel will switch to unencrypted after a login has been established. Data channel connections will still be encrypted.
Notes:

This uses the AUTH TLS, PBSZ 0, and PROT P commands to enable encryption when establishing a connection, then uses the CCC command after the username and password are successfully negotiated. (See Note 2 below.)

The USER and PASS commands are the only commands that will be encrypted; all other FTP commands are unencrypted.

tls-p Explicit Specifies that both control and data channel will use encryption over an explicit FTPS connection.
Notes:

This uses the AUTH TLS and PROT P commands when establishing a connection.

You can use "prot on" and "prot off" to specify whether encryption will be used. (See Note 3 below.)

tls-c Explicit Specifies that only the control connection will use encryption over an explicit FTPS connection. Data channel connections will be unencrypted.
Notes:

This uses only the AUTH TLS to enable encryption when establishing a connection.

You must manually send a PBSZ command before you can use the "prot on" and "prot off" to specify whether encryption will be used. (See Note 1 and Note 3 below.)

tls-c-ccc Explicit Specifies that only the control connection will use encryption over an explicit FTPS connection during login, but the control channel will switch to unencrypted after a login has been established. Data connections will be unencrypted.
Notes:

This uses only the AUTH TLS to enable encryption when establishing a connection, then uses the CCC command after the username and password are successfully negotiated.

The USER and PASS commands are the only commands that will be encrypted; all other FTP commands are unencrypted.

implicit Implicit Specifies that both the control channel and data channel will use encryption over an implicit FTPS connection, which can only be on port 990 for the FTP7 service.
Notes:

The implicit FTPS connection will encrypt both the control channel and data channel without the use of an AUTH command.

You can use "prot on" and "prot off" to specify whether encryption will be used. (See Note 3 below.)

implicit-ccc Implicit Specifies that the control channel would use encryption over an implicit FTPS connection during login and switch the control channel to unencrypted after login, but this is not supported in FTP7. The implicit FTPS connection will succeed, but the FTP7 service will return an error when the CCC command is sent. You can ignore the error and continue to use the session.
Notes:

Implicit FTPS connections require encryption for the command channel. (See Note 4 below.)

You can use "prot on" and "prot off" to specify whether encryption will be used. (See Note 3 below.)

Parameter	FTPS Mode	Description
off	n/a	Specifies that no encryption will be used on either the control channel or data channel. Note: This is the default behavior.
on	Explicit	Specifies that both the control channel and data channel will use encryption over an explicit FTPS connection. Notes: This uses the AUTH TLS, PBSZ 0, and PROT P commands when establishing a connection. You can use "prot on" and "prot off" to specify whether encryption will be used. (See Note 1 below.)
on-ccc	Explicit	Specifies the control channel will use encryption over an explicit FTPS connection during login, but the control channel will switch to unencrypted after a login has been established. Data channel connections will still be encrypted. Notes: This uses the AUTH TLS, PBSZ 0, and PROT P commands to enable encryption when establishing a connection, then uses the CCC command after the username and password are successfully negotiated. (See Note 2 below.) The USER and PASS commands are the only commands that will be encrypted; all other FTP commands are unencrypted.
tls-p	Explicit	Specifies that both control and data channel will use encryption over an explicit FTPS connection. Notes: This uses the AUTH TLS and PROT P commands when establishing a connection. You can use "prot on" and "prot off" to specify whether encryption will be used. (See Note 3 below.)
tls-c	Explicit	Specifies that only the control connection will use encryption over an explicit FTPS connection. Data channel connections will be unencrypted. Notes: This uses only the AUTH TLS to enable encryption when establishing a connection. You must manually send a PBSZ command before you can use the "prot on" and "prot off" to specify whether encryption will be used. (See Note 1 and Note 3 below.)
tls-c-ccc	Explicit	Specifies that only the control connection will use encryption over an explicit FTPS connection during login, but the control channel will switch to unencrypted after a login has been established. Data connections will be unencrypted. Notes: This uses only the AUTH TLS to enable encryption when establishing a connection, then uses the CCC command after the username and password are successfully negotiated. The USER and PASS commands are the only commands that will be encrypted; all other FTP commands are unencrypted.
implicit	Implicit	Specifies that both the control channel and data channel will use encryption over an implicit FTPS connection, which can only be on port 990 for the FTP7 service. Notes: The implicit FTPS connection will encrypt both the control channel and data channel without the use of an AUTH command. You can use "prot on" and "prot off" to specify whether encryption will be used. (See Note 3 below.)
implicit-ccc	Implicit	Specifies that the control channel would use encryption over an implicit FTPS connection during login and switch the control channel to unencrypted after login, but this is not supported in FTP7. The implicit FTPS connection will succeed, but the FTP7 service will return an error when the CCC command is sent. You can ignore the error and continue to use the session. Notes: Implicit FTPS connections require encryption for the command channel. (See Note 4 below.) You can use "prot on" and "prot off" to specify whether encryption will be used. (See Note 3 below.)

The following notes should be considered:

Encryption of the data connection can be changed during the session with MOVEit Freely's "prot" command. You can use the "prot on" to specify that data connections will be encrypted, and "prot off" to specify that data connections will be unencrypted; these commands will respectively send the PROT P and PROT C commands over FTP.
The "on-ccc", "tls-c-ccc", and "implicit-ccc" parameters are useful with firewalls that inspect FTP traffic. Switching the control channel back to unencrypted allows the firewall to inspect and possibly modify the FTP commands. For example, firewalls that are performing Network Address Translation (NAT) may need to modify the PORT and PASV commands.

"RFC 2228 - FTP Security Extensions" states that FTP clients are required to send an FTP PBSZ command before sending an FTP PROT command, and unfortunately the MOVEit Freely 5.0.0.0 FTP client does not send this command, so you get a "503 Bad sequence of commands" error. You can work around this by issuing a literal command to the server using the FTP client's "quote" command, which appears to work. See the following example for more information. The following example shows what that might look like:

Note: I turned on debugging for this example with the "-d" option so you can see the sequence of commands.

CMD>ftps.exe -e:tls-c -d ftp.example.com

220 Microsoft FTP Service
---> AUTH TLS
234 AUTH command ok. Expecting TLS Negotiation.
Connected to ftp.example.com.
User: administrator
---> USER administrator
331 Password required for administrator.
Password: ********
---> PASS (hidden)
230 User logged in.
---> SYST
215 Windows_NT
ftp> prot on
---> PROT P
503 Bad sequence of commands.
Data connections will still NOT be encrypted
ftp> quot PBSZ 0
---> PBSZ 0
200 PBSZ command successful.
ftp> prot on
---> PROT P
200 PROT command successful.
Data connections will be encrypted
ftp> bye
---> QUIT
221 Goodbye.

CMD>

The FTP7 service treats implicit FTPS connections as though the SSL policy for the control is set to "Require".

One last note about FTPS, if you are using a certificate with trust issues, you will see the following prompt displayed:

You can get around this certificate prompt when writing scripts by using the "-z" switch. The following example shows what that might look like:

Note: For this example I bypassed a certificate prompt with the "-z" switch, and I specified passive FTP with the "passive" command.

CMD>ftps.exe -z -e:on ftp.example.com

220 Microsoft FTP Service
234 AUTH command ok. Expecting TLS Negotiation.
Connected to ftp.example.com.
User: administrator
331 Password required for administrator.
Password: ********
230 User logged in.
200 PBSZ command successful.
200 PROT command successful.
215 Windows_NT
ftp> passive
Passive mode On .
ftp> ls -l
227 Entering Passive Mode (192,168,0,1,224,97).
150 Opening ASCII mode data connection.
03-10-08 10:41AM <DIR> App_Data
09-04-08 11:41AM <DIR> aspnet_client
09-04-08 11:41AM <DIR> bin
12-17-02 11:47AM 2360 default.aspx
ftp: 128 bytes received in 0.03Seconds 83.25Kbytes/sec.
226 Transfer complete.
ftp> bye
221 Goodbye.

CMD>

Using FTP Virtual Hosts

Since everything is happening from a command-line, you can use both FTP7's Virtual Hosts and the actual FTP HOST command. Once again, see my Virtual Hosts and Host Names in FTP7 blog post for more information about FTP Virtual Host Names and FTP True Host Names, and see https://datatracker.ietf.org/drafts/draft-hethmon-mcmurray-ftp-hosts/ for more information about status of the FTP HOST command.

In any event, FTP7 virtual hosts are supported by using the "ftp.example.com|username" syntax when specifying your username, and when you connect to the FTP7 server it will route your requests to the correct FTP virtual host site. The following example shows what that might look like:

CMD>ftps.exe ftp.example.com

220 Microsoft FTP Service
Connected to ftp.example.com.
User: ftp.contoso.com|administrator
331 Password required for ftp.contoso.com|administrator.
Password: ********
230-Directory has 104,857,600 bytes of disk space available.
230 User logged in.
ftp> bye
221 Goodbye.

CMD>

True FTP hosts can be used by specifying the FTP HOST command before the client sends the USER and PASS credentials. This is accomplished in two parts:

You need to suppress the automatic username prompt MOVEit client by using the "-n" switch on the command-line.
You need to specify the host name using the MOVEit client's "quote" command, which allows you to send custom FTP commands. The syntax for this would be "quote HOST ftp.example.com".

The following example shows what that might look like:

CMD>ftps.exe -n ftp.example.com

220 Microsoft FTP Service
Connected to ftp.example.com.
ftp> quote HOST ftp.contoso.com
220 Host accepted.
ftp> USER administrator
331 Password required for administrator.
Password: ********
230-Directory has 104,857,600 bytes of disk space available.
230 User logged in.
ftp> bye
221 Goodbye.

CMD>

Scorecard for the MOVEit Freely command-line FTP client

This concludes our quick look at some of the features that are available with the MOVEit Freely command-line FTP client, and here's the scorecard results:

Client Name	Directory Browsing	Explicit FTPS	Implicit FTPS	Virtual Hosts	True HOSTs
MOVEit Freely 5.0.0.0	n/a	Y	Y	Y	Y ¹
¹ As noted earlier, true FTP HOSTs are available when using the "quote HOST ftp.example.com" syntax.

Note: Keeping with my standard disclaimer, there are a great number of additional features that the MOVEit Freely command-line FTP client provides - I'm just keeping the focus on those topic areas that apply to FTP7.

FTP Clients - Part 4: FileZilla

robert_mcmurray — Thu, 18 Dec 2008 02:25:00 GMT

For this next installment in my FTP Clients series, I'd like to take a look at the FileZilla FTP client. For this blog post I was using FileZilla version 3.1.6.

There are a lot of places where you can find FileZilla, but the best location is the official FileZilla web site at http://filezilla-project.org/. The FileZilla FTP client is free, so you can't beat the price. ;-]

The user interface is pretty straight-forward: you have separate windows for your local and remote files/folders, as well as a logging window that lists the FTP commands that are sent and the FTP server's responses.

FileZilla has a great Site Manager feature, which allows you to store commonly-used connections to FTP sites.

Using FTP over SSL (FTPS)

When creating a connection to an FTPS server, FileZilla has two options: FTPS and FTPES. It's important to have this option configured correctly, otherwise you will run into problems when trying access a site using FTPS. If you'll recall from my "FTP Clients - Part 2: Explicit FTPS versus Implicit FTPS" blog post, Explicit FTPS allows the client to initiate SSL/TLS whenever it wants, but for most FTP clients that will be when logging in to your FTP site, and in that regard it may almost seem like Implicit FTPS, but behind the scenes the FTP client and server are communicating differently.

In the case of FTP7, the following rules apply:

If you enable FTPS and you assign the FTP site to port 990, you are using Implicit SSL - FileZilla refers to this as FTPS.
If you enable FTPS and you assign the FTP site to any port other than port 990, you are using Explicit SSL - FileZilla refers to this as FTPES.

Using FTP Virtual Hosts

Because FileZilla's site manager allows you to specify the virtual host name as part of the user credentials, FileZilla works great with FTP7's virtual host names. All that you need to do is use the "ftp.example.com|username" syntax when specifying your username, and when you connect to the FTP7 server it will route your requests to the correct FTP virtual host site.

Unfortunately, even though FileZilla allows you to send custom commands, you cannot send custom commands outside of an established FTP session, so you can't send the FTP HOST command as part of your login, therefore true FTP hosts are not supported.

Directory Browsing

Using MS-DOS or UNIX directory listings in FTP7 didn't have any impact on whether FileZilla could render directory listings, nor did configuring any of the other options such as four-digit years, etc. When I create FTP connections in FileZilla's site manager it defaults to auto-detecting the FTP server type, which makes the directory browsing behavior transparent to the client. (Behind the scenes FileZilla is sending an FTP SYST command, which allows FileZilla to detect the operating system.)

You can customize the server type in the advanced settings for your FTP connection, so you can match up your FTP7 directory listing options and the server type that FileZilla expects, but personally I have had no problems with auto-detection so I prefer to use that option.

On a side note, if you intentionally misconfigure FileZilla's server type settings, you can cause FileZilla to behave strangely. For example, choosing a VMS server type and configuring FTP7 to use MS-DOS directory listings will not work, but then again - I wouldn't expect that to work. ;-]

Scorecard for FileZilla

So - that concludes our quick round-trip for some of FileZilla's features, and here's the scorecard results:

Client Name	Directory Browsing	Explicit FTPS	Implicit FTPS	Virtual Hosts	True HOSTs
FileZilla 3.1.6	Rich	Y	Y	Y	N

Note: As with all of the FTP clients in this blog series, there are a great number of additional features that FileZilla provides - I'm just keeping the focus on a few specific topic areas that apply to FTP7.