IEInternals

A look at Internet Explorer from the inside out. @EricLaw left Microsoft in 2012, but was named an IE MVP in '13 & an IE userAgent (http://useragents.ie) in '14

Bugs in IE8's Lookahead Downloader

Bugs in IE8's Lookahead Downloader

All bugs mentioned in this post are now fixed. 

Internet Explorer has a number of features designed to render pages more quickly. One of these features is called the "Lookahead Downloader" and it's used to quickly scan the page as it comes in, looking for the URLs of resources which will be needed later in the rendering of the page (specifically, JavaScript files). The lookahead downloader runs ahead of the main parser and is much simpler-- its sole job is to hunt for those resource urls and get requests into the network request queue as quickly as possible. These download requests are called "speculative downloads" because it is not known whether the resources will actually be needed by the time that the main parser reaches the tags containing the URLs. For instance, inline JavaScript runs during the main rendering phase, and such script could (in theory) actually remove the tags which triggered the speculative downloads in the first place. However, this "speculative miss" corner case isn't often encountered, and even if it happens, it's basically harmless, as the speculative request will result in downloading a file which is never used.

IE8 Bugs and their impact
Unfortunately, since shipping IE8, we've discovered two problems in the lookahead downloader code that cause Internet Explorer to make speculative requests for incorrect URLs. Generally this has no direct impact on the visitor's experience, because when the parser actually reaches a tag that requires a subdownload, if the speculative downloader has not already requested the proper resource, the main parser will at that time request download of the proper resource. If your page encounters one of these two problems, typically:

  • The visitor will not notice any problems like script errors, etc
  • The visitor will have a slightly slower experience when rendering the page because the speculative requests all "miss"
  • Your IIS/Apache logs will note requests for non-existent or incorrect resources

If your server is configured to respond in some unusual way (e.g. logging the user out) upon request of a non-existent URL, the impact on your user-experience may be more severe.

The BASE Bug

Update: The BASE bug is now
 
fixed.

The first problem is that the speculative downloader "loses" the <BASE> element after its first use. This means that if your page at URL A contains a tag sequence as follows:

<html><head><base href=B><script src=relC><script src=relD><script src=relE><body>

which requests 3 JavaScript files from the path specified in "B", IE8's speculative downloader will incorrectly request download of URLs "B+relC", and "A+relD" and "A+relE". Correct behavior is to request download of URLs "B+relC", "B+relD", and "B+relE". Hence, in this case, two incorrect requests are sent, usually resulting in 404s from the server. Of course, when the main parser gets to these script tags, it will determine that "B+relC" is already available, but "B+relD", and "B+relE" have not yet been requested, and it will request those correct two URLs and complete rendering of the page.

At present, there is no simple workaround for this issue. Technically, the following syntax will result in proper behavior:

 <html><head><base href=B><script src=relC><base href=B><script src=relD><base href=B><script src=relE><body>

...but this is not standards-compliant and is not recommended. If the page removes its reliance upon the BASE tag, the problem will no longer occur.

Remember: The BASE bug is now fixed.

The Missing 4k Bug

Update: The 4k bug is now fixed. 

The second problem is significantly more obscure, although a number of web developers have noticed it and filed a bug on Connect. Basically, the problem here is that there are a number of tags which will cause the parser and lookahead downloader to restart scanning of the page from the beginning. One such tag is the META HTTP-EQUIV Content-Type tag which contains a CHARSET directive. Since the CHARSET specified in this tag defines what encoding is used for the page, the parser must restart to ensure that is parsing the bytes of the page in the encoding intended by the author. Unfortunately, IE8 has a bug where the restart of the parser may cause incorrect behavior in the Lookahead downloader, depending on certain timing and network conditions.

The incorrect behavior occurs if your page contains a JavaScript URL which spans exactly the 4096th byte of the HTTP response. If such a URL is present, under certain timing conditions the lookahead downloader will attempt to download a malformed URL consisting of the part of the URL preceding the 4096th byte combined with whatever text follows the 8192nd byte, up to the next quotation mark. Web developers encountering this problem will find that their logs contain requests for bogus URLs with long strings of URLEncoded HTML at the end.

As with the previous bug, end users will not typically notice this problem, but examination of the IIS logs will show the issue.

For many instances of this bug, a workaround is available-- the problem only appears to occur when the parser restarts, so by avoiding parser restarts, you can avoid the bug.  By declaring the CHARSET of the page using the HTTP Content-Type header rather than specifying it within the page, you can remove one cause of parser restarts.

So, rather than putting

<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=utf-8">

In your HEAD tag, instead, send the following HTTP response header:

Content-Type: text/html; charset=utf-8

Note that specification of the charset in the HTTP header results in improved performance in all browsers, because the browser's parsers need not restart parsing from the beginning upon encountering the character set declaration. Furthermore, using the HTTP header helps mitigate certain XSS attack vectors.

Unfortunately, however, suspension of the parser (e.g. when it encounters an XML Namespace declaration) can also result in this problem, and it's not feasible for a web developer to avoid suspension of the parser.

But, remember: The 4k bug is now fixed. 

Summary
While these problems are significant, they are not so dire as some readers will conclude at first glance. The second bug, in particular, is quite rarely encountered due to its timing-related nature and the requirement that page have a JavaScript URL spanning a particular byte in the response. Encountering the second issue is not nearly as prevalent as some web developers believe-- for instance, we've heard claims that IE6, 7, and Firefox all have this problem, which is entirely untrue. Readers can easily determine if a page is hitting either bug by examining server logs, or watching network requests with Fiddler.

The IE team will continue our investigation into these bugs and, as with any reported issues, may choose to make available an IE8 update to resolve the issues.

Remember: All bugs mentioned in this post are now fixed. 

Apologies for the inconvenience, and thanks for reading!

-Eric

  • Thank you for tracking this issue.  We are experiencing this regularly (approx once every 2 hours) on our high volume ASP.NET site.

    Outside of the charset change you mentioned, is there anything else web developers can do to prevent parser restarts in IE8?

    Any insight is appreciated.

  • @ScottS: I assume you're saying you hit the 4k issue, and not the "BASE" issue, as the latter will occur reliably on every request.

    If you can send me the HTML (or a network capture: www.fiddler2.com) of the affected page, I'll have a look to see if there's any other cause for the parser restart. Email me at microsoft.com, username ericlaw

  • The second bug is an issue for all developers who align their code to 4096 bytes in order to make it run faster on Win98 :o)

  • Thank you Eric for such a comprehensive analysis. I emailed you a few weeks ago about this issue. This is great to see that you guys found the root cause of this problem that is indeed obscure.

  • Wow!  That Missing 4K bug seems to be exactly the problem I've been trying to solve on my high traffic ASP site.  I've spoken to Eric Lawrence during the IE8 expert chats, and am incredibly glad he was able to figure this guy out.

  • Glad to see you tracked down this bug. Thanks for your detailed explanation on all this.

    Do you have an estimation on when the fix will be available ?

  • Also beware that the problem can also occurs for resources other than JavaScript.

    For example CSS and shortcut icon :

    <link rel="stylesheet" href="path-to-stylesheet.css" type="text/css" media="all" />

    <link rel="shortcut icon" href='path-to-favicon.ico' />

  • I had

    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

    in my HEAD tag, and  using Wireshark I could see IE8 requesting the external JavaScript + CSS files only to immediately send a "TCP Reset" to kill the sockets, and then requesting the files again....

    changing the meta to :

    <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>

    fixed the problem....

  • The lack of standard BASE compliance (in a browser touted as being more complient) is a show stopper bug for us.

    I just spent a LOT of time try to figure out why our authentication system was returning a 404 error... and it's because of this bug.

    Here's a common situation that this bug breaks:

    A secure site with only files in the /login directory viewable by the public.

    When a secure file is asked for, the user is redirected to a login page via a "/servlet/login" request.  

    The login page has a BASE tag to get all the CSS, images, etc from the /login directory.  All of which have no security attached.

    However, the IE8 (let me tell you where to go) optimistic code, uses the "/servlet/login" to create a secured non-existant URL, e.g. /servlet/login/login.css.

    AND IT MAKES THIS REQUEST BEFORE the user entered URL is requested.

    So, when the user asks for /foo.html, IE8 actually asks the system for /servlet/login/login.css first.  The security system does what it's supposed to... says, I need you to authenticate before I can confirm or deny this files existance.

    The user login in, thinking they are going to /foo.html, but the authentation system has been given a non-existing URL first.

    The user see's a 404 file not found error, and our support lines light up because they have been told to get /foo.html (wich they can if the request it after authenticating).

    Sorry to sound bitter but this has caused us a lot of grief, especially since IE8 is a highly recommended update.  

    IMHO, this would be marked as a SHOWSTOPPER or CRITICAL bug in anyone issue system.  PLEASE fix it MS.

    Before you say "just fix the login page..."  we use this in 80+ sites used by our clients (mostly fortune 500 companies).  Time is money... and IE8 is costing us by not being compliant with the standards.

  • @CGMonroe: The BASE issue has nothing in common with standards compliance in general. As stated, this bug is a regression, pure and simple.

    I'm not sure I understand the rest of your message. IE makes an invalid speculative download request, which fails. Then, later, it makes a non-speculative download request for the proper URL.

    It's not at all clear why this is a "show stopper" bug for you. Does your 404 response delete the user's cookies or something like that?

    If you have a public URL, I'd be happy to take a look.

  • @CG: Comments with URLs in them are often blocked as spam.

    However, before you resend, please understand two factors:

    1> This is a bug. A plain old boring bug. Bug. You need not bother quoting the HTML spec to show that it's a bug. Everyone agrees that it's a bug.

    2> You seem to assume that the user will see or be redirected to an error page due to this bug. That's not the case, as the result from the failed lookahead download is simply discarded because the parser uses the proper URL.

  • Something in my long reply is causing the comment system to choke.. probably as designed.

    Anyway, here a URL to what I was trying to post here:

    http://dl.getdropbox.com/u/1701672/IE8-Base-bug.txt

  • As noted, please see my comment above.

  • Sigh... but I have documented and personally seen that in the case of a secure site, this bug WILL cause the display a 404 to users... because of this bug.  But, you've made up your mind.. so I'll move on.

  • @CG: I'm more than happy to look at any repros, but if you see a redirection to a 404 page, it's not related to this bug.

    www.fiddlercap.com shows how to gather a network capture for me to look at, if it's not a public site.

Page 1 of 8 (116 items) 12345»
Leave a Comment
  • Please add 7 and 7 and type the answer here:
  • Post