Robert McMurray's Blog [MSFT]

Discussing IIS, FTP, WebDAV, FPSE, WMI, ADSI, ISAPI, ASP, FastCGI, etc. ;-)

Virtual Hosts and Host Names in FTP7

Virtual Hosts and Host Names in FTP7

  • Comments 3

I received the following question from Mickey Binder in response to my blog entries about the new FTP service for IIS7:

"When using virtual hosts will it always be necessary to provide DOMAIN|Username or is it me doing something wrong. I can see in your documentation how it should be setup, I just wonder why it still needs the DOMAIN in the user login as this should already be specified by the host I connect to (Like with the http virtual hosts)."

I thought that was a great question, and I think that it deserves a detailed response. The short answer is that you need to provide the host name somehow when connecting to an FTP site, and FTP simply does not work the way that HTTP does. I'll explain why in detail with this blog post.

FTP and HTTP Host Names

HTTP provides a way to pass the host name in the headers that are passed between the client and server, but FTP currently does not provide this same functionality. Looking at one of my older blog posts, I pointed out that there are three ways that you can create unique bindings for a Web/HTTP site: IP address, port, or host header. Based on its current design, FTP can create unique bindings by IP address or port, but the FTP protocol currently does not allow for something like host headers.

Here's why - I'm sure most everyone is aware that HTTP packets consist of a set of headers and possibly a block of data. Here's an example of a simple GET request:

GET /default.aspx HTTP/1.0 [crlf]
Accept: */* [crlf]
[crlf]

When HTTP 1.1 was published in RFC 2068 and RFC 2616 it defined a header for specifying a "host" name in a separate name/value pair:

GET /default.aspx HTTP/1.1 [crlf]
Host: example.com [crlf]
Accept: */* [crlf]
[crlf]

This allows multiple virtual servers ("hosts") on the same IP address and port that are differentiated by host name. While this works great for the HTTP protocol, the FTP protocol currently has no comparable functionality. As such, the FTP protocol would have to be updated to allow multiple hosts on the same IP address and port, then IIS and all FTP clients would need to be updated to accommodate the changes to FTP.

I realize that most everyone is aware that when you attempt to connect to an HTTP or FTP server from a client, the client looks up the IP address using a name server and then creates a connection to the server by IP address. What many people may not realize is that the server is basically unaware of the host name that the client used - at connection time the server is really only concerned with inbound data on the IP address.

That being said, the connection protocol could provide a mechanism for specifying the host name. For example, as I mentioned above the HTTP/1.1 protocol provides a mechanism for specifying host names using a host header and FTP does not. In fact, if you were to open Network Monitor or similar tool and capture an FTP connection between a client and a server, you would be able to see that the host name is not passed as part of the FTP conversation between the client and server.

Host Names in FTP for IIS7

For the new FTP service for IIS7 we wanted to find a way to have host names in FTP, so we approached the situation with two possible ideas:

  1. Find a backwards-compatible method for specifying host names for FTP
  2. See if we could get the FTP protocol updated for host names

In the end, we chose to do both.

FTP Virtual Host Name Support

First of all we have "Virtual Host" names; this is the "ftp.example.com|username" syntax that led to the original question. Here's the way that this can be used in a hosting environment. Let's say that a web hoster has a customer with several Web/FTP sites that he needs to update. The web hoster gives all of his customers the same ftp address, like "ftp.example.com". When setting up the bindings for the Web/FTP sites the web hoster configures each site for the appropriate HTTP/FTP bindings, using host names like "www.contoso.com" & "www.fabrikam.com" for the HTTP bindings and "ftp.contoso.com" & "ftp.fabrikam.com" for FTP bindings. (All of the Web names would be set up in DNS, of course, but the FTP names are somewhat optional as I'll explain later.)

Since HTTP provides host headers to pass the host name, the Web client & Web server will both know which site they're working with. Since FTP doesn't have a built-in way to pass a host name, the customer will connect to the "ftp.example.com" name that his web hoster gave him and log in using the appropriate "ftp.contoso.com|customer" or "ftp.fabrikam.com|customer" syntax. The FTP service for IIS7 will then internally route the FTP activity to the correct site based on the FTP bindings. If the FTP names "ftp.contoso.com" & "ftp.fabrikam.com" were registered in DNS, the client would still need to specify the appropriate "ftp.contoso.com|customer" or "ftp.fabrikam.com|customer" syntax when logging in because the FTP activity did not actually pass the host name in any way.

The great thing about the "Virtual Host" names solution is that it is backwards-compatible because any client should be able to send the "ftp.example.com|username" syntax.

FTP True Host Name Support

We also wanted to see if the FTP protocol could be updated to allow sending a host name as part of the FTP session like HTTP does. After some research I discovered that Robert Elz and Paul Hethmon had provided a detailed discussion of a "HOST" command for FTP in their Internet draft titled "Extensions to FTP" during their work with the FTPEXT Working Group at the IETF. I contacted the two of them and with their approval Paul and I submitted a new IETF draft detailing a HOST command for FTP, which is posted at the following URL:

http://www.ietf.org/internet-drafts/draft-hethmon-mcmurray-ftp-hosts-00.txt

Here's the way that the HOST command works: the FTP server and FTP client both need to know that the HOST command is supported. The FTP client connects to an FTP server using either a DNS name or IP address and sends a FEAT command and sees that HOST is supported, so the client sends "HOST ftp.example.com" before sending USER and PASS, which allows the FTP server to route the request to the correct FTP site based on the bindings. (An FTP client could skip the FEAT command completely and simply attempt a HOST command and process the FTP reply, but that's not the best approach.)

The FTP HOST command solution is not backwards-compatible, however, because an FTP client needs to be able to send the "HOST ftp.example.com" syntax. Some FTP clients allow sending custom commands, which enables this functionality. For example, if you are using the FTP.EXE command-line tool that comes in Windows, you can type "quote HOST ftp.example.com" when connecting to the new FTP server for IIS7 and it will route the request to the appropriate site. In an ideal world, FTP clients will start negotiating the HOST feature behind the scenes and you should never know that this is occurring, which is how almost all Web browsers currently work. (e.g. When you enter "www.example.com" in the address bar of a Web browser it will automatically add the host header to the HTTP request.)

The Final Word

So the long answer to the original question is that you have two ways of specifying a host with the new FTP service for IIS7:

  • FTP "Virtual Host" names are supported using "ftp.example.com|username" syntax
  • True FTP host names are supported using the "HOST ftp.example.com" syntax

So my thanks to Mickey Binder for his great question, and I hope this helps other people understand this concept a little better.

Comments
  • Here are a couple of items that I thought needed mentioning as well:

    • Having an FTP HOST command does lead to strange protocol quirks that HTTP is also subject to - an HTTP or FTP client can connect to an IP address using a specific DNS name and then pass a different host name after connecting to the IP address. This isn't a security vulnerability - it's just the way that HTTP and FTP work; logically-speaking the client should be passing the same name for the connection and host name, but that's not necessarily required.
    • Someone responded to my earlier blog post and asked if reverse DNS could have been used as another option, but reverse DNS is out of the question because as I mentioned earlier in this blog post the server is not aware of the host name and therefore cannot perform a reverse lookup by host name. Likewise, if you have several host names that are registered to the same IP address, the server can perform a lookup on the IP address, but when the name server contains multiple registrations for the IP address, which host name should it map to? Since FTP packets from a client cannot pass a host name by default, the FTP server would have no way of knowing which host name to use from the list provided by a name server. But then again, if FTP provided a means to specify the host name the whole issue would be moot and a reverse lookup would not be needed, which is why we researched adding a HOST commend for FTP.

    I hope this helps!

  • How does FTPS (SSL) impact the virtual host scenario?

  • Robert,

    Here is my scenario:

    I am running FTP 7.5 on Windows Server 2008 R2 and I need to create two FTP sites, one hosting standard FTP over port 21 and 2nd SFTP over port 22 I believe (port 22?). Though I am not asked to change the port when I install the SSL cert so don’t know if I am doing something wrong here (help)

    I have created two host names thinking that it will be the same as hosting multiple website using the same IP with different host headers.

    ftp1.corp.domain.com

    ftp2.corp.domain.com

    After reading through your post I am now wondering if this solution is going to work for me.

    I have created my FTP site using the following blog (www.dotnetscraps.com/.../Did-you-know-Configure-Client-Certificate-Mapping-in-FTP-7-FTP-75-Part-3.aspx)

    It talks about adding SSL cert and assigning the host name, but this is not working for me.

    My standard FTP over port 21 works fine as long as I am not using the host name (i.e. ftp1.corp.domain.com) and just using the server name works fine and I can connect over port 21 using my domain ID.

    My questions:

    1. Can Microsoft FTP7.5 truly provide SFTP? If yes, I think I know how to create the site, but how can I test/verify that I can connect using SFTP because when creating the site I don’t get the option to bind to “SFTP”.

    2. I am using domain Ids to allow access to FTP sites (corp/userID format) if what you are saying using (host header/username) how will this work when I am using AD accounts/groups?

    3. How can I host multiple FTP sites where one is running standard FTP and 2nd is running SFTP?

    Please advice

Page 1 of 1 (3 items)
Leave a Comment
  • Please add 6 and 1 and type the answer here:
  • Post