Please read my blog's comment policy here.
Note: The “brain dump” series is akin to what the support.microsoft.com team calls “Fast Publish” articles—namely, things that are published quickly, without the usual level of polish, triple-checking, etc. I expect that these posts will contain errors, but I also expect them to be mostly correct. I’m writing these up this way now because they’ve been in my “Important things to write about” queue for ~5 years. Alas, these topics are so broad and intricate that a proper treatment would take far more time than I have available at the moment.
When you navigate to a web address like www.bing.com/help in your browser, in most cases, the browser (or more specifically, WinINET, the network stack below IE) will immediately convert the hostname component of that URL (in this case, www.bing.com) into a numeric IP address like 18.104.22.168. It performs this magic by sending a query to a Domain Name System (DNS) server that then returns one or more address records for the target hostname. The browser then establishes a TCP/IP connection to the target IP and then uses HTTP (and/or SSL/TLS) to transfer web traffic over that connection.
Many people assume that a hostname will only return one target address, but in practice, it will often return several. For instance, on my PC, a DNS query for www.bing.com returns the following:
2600:1409:1::6010:6119 2600:1409:1::6010:6123 22.214.171.124 126.96.36.199
2600:1409:1::6010:6119 2600:1409:1::6010:6123 188.8.131.52 184.108.40.206
The first two addresses are IPv6 addresses, while the latter two are IPv4 addresses. Prior to Windows 2000, Windows would randomly select one of the addresses and attempt to use it; after Windows 2000, the first returned address is always returned first. The change was made to support the idea that the DNS resolver will attempt to order the address list by placing “preferred” addresses first. For instance, the list might be sorted based on the resolver’s knowledge of locality or connection throughput to a given server.
Windows Vista and later sort the IPv6 addresses to the front of the list by default (In the comments below, Lionel provides a netsh command to control the default).
While WinINET will attempt to connect to the first address, if that connection fails, that entry in the address list will be marked as “bad” and a connection will be attempted to the next returned address. This process repeats until a connection is made, the address list runs out of candidate addresses, or the retry count limit is reached (INTERNET_OPTION_CONNECT_RETRIES defaults to 5). If a connection is not made, the browser will show a “Page could not be displayed” error message.
The linear process of failover can lead to performance problems, if a DNS returns many records which must be tried before a working address is reached. Each connection attempt could take as long as 21 seconds. The most common case where this problem occurs is when a server returns IPv6 addresses to a client that cannot successfully make IPv6 connections due to network configuration problems. A proposal (oddly named “Happy Eyeballs”) was made to address such problems by more quickly attempting to contact IPv4 addresses if IPv6 addresses are slow to resolve. As far as I understand it, Firefox and Chrome utilize a variant of this notion. You can read about how Windows 8 improved IPv6 resiliency and performance over on the B8 blog.
For performance reasons, WinINET caches address lists using an in-process memory cache; this cache allows reuse of recently used addresses without resolving them again. DNS resolutions are also cached by Windows' DNS Resolver itself, but retrieving them from the Windows cache requires an RPC call, which, while much faster than issuing a DNS request on the wire, still takes some time.
In Internet Explorer 9 and earlier, the per-process cache could hold up to 32 entries; in IE10 and later it holds up to 256 entries. The cache ignores any TTL (time-to-live) specified by the DNS server’s response. Instead each list is stored for 30 minutes by default; the DnsCacheTimeout registry key can be used to adjust this value. (WinHTTP, another Windows HTTP stack used primarily by services, has different defaults; it caches up to 32 records for up to 5 minutes).
It has been speculated that the WinINET DNS cache was created to attempt to prevent DNS Rebinding attacks, but this is a fallacy—the cache exists for performance reasons only, and is trivial to circumvent as a part of a rebinding attack.
If the client computer is behind a traditional (“CERN-style”) proxy server, WinINET skips the DNS lookup for the target site and instead sends the request to the proxy server, allowing the proxy server to perform the DNS lookup on the client’s behalf. This is useful in some cases, for instance, the case where the client computer doesn’t itself have access to a DNS server that is willing to return records for non-Intranet sites. Of course, the browser may still need to use DNS even in this case—if your browser is configured to send traffic to CORPPROXY:8080, it will need to use DNS to look up the address of CORPPROXY. It’s possible that the DNS lookup for the proxy will return more than one address; if it does, the same pattern described above is used to decide which address to use.
However, in the unlikely event that you’re using a SOCKS-style proxy, WinINET must first perform the DNS lookup locally, sending that address information to the proxy server. That’s because IE only supports SOCKSv4, and the ability to rely upon the remote proxy to do DNS lookups wasn’t introduced until the SOCKSv4a protocol.
Discussion of Firefox's DNS Cache and defaults: superuser.com/.../how-does-firefox-cache-dns-requests-replies
> Windows Vista and later always sort the IPv6 addresses to the front of the list.
Nitpick: I believe this is configurable using "netsh interface ipv6 set prefixpolicy".
> The cache ignores any TTL (time-to-live) specified by the DNS server’s response.
So, a server that uses dynamic DNS can become unreachable for 30min unless IE is restarted? Are other name resolution protocols (esp. Microsoft's Peer Name Resolution Protocol) cached in the same way?
It looks like WinInet's interaction with DNS is only done through getaddrinfo() or some similar API (which would explain e.g. why TTLs are ignored). With DNSSEC becoming slowly more widespread, there are other relevant information than addresses for browsers. I'm thinking esp. of TLSA records (RFC 6698) that enhance the old PKI model for certificates, which I would really like to see supported in Windows' TLS stack.
@Lionel: Thanks for the netsh suggestion. In general I only talk about defaults, since there are a billion settings for everything.
With regard to Dynamic DNS, no, it's the server is not likely to become unreachable. If socket connections can't be made to the old server, all of the addresses in its list get marked bad, and the next time a connection is needed, a new resolution occurs and the new addresses are used.
By the way, is there anyone on the IE team with an active blog beside the IE blog? Eric is the last that I know of, and he's leaving.
(Not that the IE blog is always bad. Posts have gotten a bit defensive and too marketing-like for my taste, but there is still a lot of information posted there. The comments, however, are very bad, full of trolls and angry people, and not enough answers. Personal blogs are more in my comfort zone.)
Thanks for the clarification on Dynamic DNS!
@Lionel: I'm not aware of any other active IE bloggers, but it's possible that someone will pick up the IEInternals torch after I go. Other blogs I like (some defunct) are listed at blogs.msdn.com/.../my-favorite-mostly-browser-related-blogs-worth-reading.aspx
Does anyone here know how IE8 and 9 handle sort lists and dns names with multiple IP addresses?
All the other browsers seem to pick the first IP from the list. But IE seems to try to round robin the results (which is bad in our case).
Thanks Eric, that's helpful.
Are this blog's archives going to persist, or will you recreate them somewhere else?
@iain: I answered this question directly in the section "DNS Failover". No, IE doesn't round-robin (and hasn't since unpatched IE5.0). It only goes to the second address if the first fails, and so on.
@MNot: As far as I know, this blog archive will remain up indefinitely.
Eric do you know if these settings have changed for IE11? Also how does IE10 mobile behave?
[EricLaw] Unfortunately, I no longer have source access, but I would be surprised if these settings changed in IE11. Mobile is a different story-- it's very likely that it has different defaults due to a significantly different networking implementation and greater concern about memory constraints.