IEInternals

A look at Internet Explorer from the inside out. @EricLaw left Microsoft in 2012, but was named an IE MVP in '13 & an IE userAgent (http://useragents.ie) in '14

Bugs in IE8's Lookahead Downloader

Bugs in IE8's Lookahead Downloader

All bugs mentioned in this post are now fixed. 

Internet Explorer has a number of features designed to render pages more quickly. One of these features is called the "Lookahead Downloader" and it's used to quickly scan the page as it comes in, looking for the URLs of resources which will be needed later in the rendering of the page (specifically, JavaScript files). The lookahead downloader runs ahead of the main parser and is much simpler-- its sole job is to hunt for those resource urls and get requests into the network request queue as quickly as possible. These download requests are called "speculative downloads" because it is not known whether the resources will actually be needed by the time that the main parser reaches the tags containing the URLs. For instance, inline JavaScript runs during the main rendering phase, and such script could (in theory) actually remove the tags which triggered the speculative downloads in the first place. However, this "speculative miss" corner case isn't often encountered, and even if it happens, it's basically harmless, as the speculative request will result in downloading a file which is never used.

IE8 Bugs and their impact
Unfortunately, since shipping IE8, we've discovered two problems in the lookahead downloader code that cause Internet Explorer to make speculative requests for incorrect URLs. Generally this has no direct impact on the visitor's experience, because when the parser actually reaches a tag that requires a subdownload, if the speculative downloader has not already requested the proper resource, the main parser will at that time request download of the proper resource. If your page encounters one of these two problems, typically:

  • The visitor will not notice any problems like script errors, etc
  • The visitor will have a slightly slower experience when rendering the page because the speculative requests all "miss"
  • Your IIS/Apache logs will note requests for non-existent or incorrect resources

If your server is configured to respond in some unusual way (e.g. logging the user out) upon request of a non-existent URL, the impact on your user-experience may be more severe.

The BASE Bug

Update: The BASE bug is now
 
fixed.

The first problem is that the speculative downloader "loses" the <BASE> element after its first use. This means that if your page at URL A contains a tag sequence as follows:

<html><head><base href=B><script src=relC><script src=relD><script src=relE><body>

which requests 3 JavaScript files from the path specified in "B", IE8's speculative downloader will incorrectly request download of URLs "B+relC", and "A+relD" and "A+relE". Correct behavior is to request download of URLs "B+relC", "B+relD", and "B+relE". Hence, in this case, two incorrect requests are sent, usually resulting in 404s from the server. Of course, when the main parser gets to these script tags, it will determine that "B+relC" is already available, but "B+relD", and "B+relE" have not yet been requested, and it will request those correct two URLs and complete rendering of the page.

At present, there is no simple workaround for this issue. Technically, the following syntax will result in proper behavior:

 <html><head><base href=B><script src=relC><base href=B><script src=relD><base href=B><script src=relE><body>

...but this is not standards-compliant and is not recommended. If the page removes its reliance upon the BASE tag, the problem will no longer occur.

Remember: The BASE bug is now fixed.

The Missing 4k Bug

Update: The 4k bug is now fixed. 

The second problem is significantly more obscure, although a number of web developers have noticed it and filed a bug on Connect. Basically, the problem here is that there are a number of tags which will cause the parser and lookahead downloader to restart scanning of the page from the beginning. One such tag is the META HTTP-EQUIV Content-Type tag which contains a CHARSET directive. Since the CHARSET specified in this tag defines what encoding is used for the page, the parser must restart to ensure that is parsing the bytes of the page in the encoding intended by the author. Unfortunately, IE8 has a bug where the restart of the parser may cause incorrect behavior in the Lookahead downloader, depending on certain timing and network conditions.

The incorrect behavior occurs if your page contains a JavaScript URL which spans exactly the 4096th byte of the HTTP response. If such a URL is present, under certain timing conditions the lookahead downloader will attempt to download a malformed URL consisting of the part of the URL preceding the 4096th byte combined with whatever text follows the 8192nd byte, up to the next quotation mark. Web developers encountering this problem will find that their logs contain requests for bogus URLs with long strings of URLEncoded HTML at the end.

As with the previous bug, end users will not typically notice this problem, but examination of the IIS logs will show the issue.

For many instances of this bug, a workaround is available-- the problem only appears to occur when the parser restarts, so by avoiding parser restarts, you can avoid the bug.  By declaring the CHARSET of the page using the HTTP Content-Type header rather than specifying it within the page, you can remove one cause of parser restarts.

So, rather than putting

<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=utf-8">

In your HEAD tag, instead, send the following HTTP response header:

Content-Type: text/html; charset=utf-8

Note that specification of the charset in the HTTP header results in improved performance in all browsers, because the browser's parsers need not restart parsing from the beginning upon encountering the character set declaration. Furthermore, using the HTTP header helps mitigate certain XSS attack vectors.

Unfortunately, however, suspension of the parser (e.g. when it encounters an XML Namespace declaration) can also result in this problem, and it's not feasible for a web developer to avoid suspension of the parser.

But, remember: The 4k bug is now fixed. 

Summary
While these problems are significant, they are not so dire as some readers will conclude at first glance. The second bug, in particular, is quite rarely encountered due to its timing-related nature and the requirement that page have a JavaScript URL spanning a particular byte in the response. Encountering the second issue is not nearly as prevalent as some web developers believe-- for instance, we've heard claims that IE6, 7, and Firefox all have this problem, which is entirely untrue. Readers can easily determine if a page is hitting either bug by examining server logs, or watching network requests with Fiddler.

The IE team will continue our investigation into these bugs and, as with any reported issues, may choose to make available an IE8 update to resolve the issues.

Remember: All bugs mentioned in this post are now fixed. 

Apologies for the inconvenience, and thanks for reading!

-Eric

  • We have had some code in the past that was moving ViewState to the end of the page form to help SEO.

    I've been applying the same to the WebResource and ScriptResource script tags and can move them to the end of the form to get them away from the 4096th bit but because they are now loaded after, say, some Ajax on the page the Ajax doesn't work.

    Trying a few things to get this working but for anyone interested this is what we're trying to do...

                   int startPoint2 = html.IndexOf("<script src=\"/WebResource.axd?");

                   if (startPoint2 >= 0)

                   {

                       int endPoint2 = html.IndexOf("</script>", startPoint2) + 9;

                       string webresourceInput = html.Substring(startPoint2, endPoint2 - startPoint2);

                       int formEndStart2 = html.IndexOf("</form>");

                       if (formEndStart2 >= 0)

                       {

                           html = html.Remove(startPoint2, endPoint2 - startPoint2);

                           html = html.Insert(formEndStart2 - (endPoint2 - startPoint2), webresourceInput);

                       }

                   }

  • For the "Missing 4K bug", I used Response.AddHeader("Content-Type", "text/html; charset=utf-8") in the Page_Load event. But when I view source, I don't see anything about Content-Type.

    Was I doing it right, or that is just expected?

    Thanks.

  • @meta: You're right, it's the expected behavior. The Content-Type directive is added to the HTTP header which can be viewed with excellent tools such as Fiddler (written by the writer of this blog), Firebug or Live HTTP headers among others.

  • This does not work for us: we are still getting the same number of errors per day

  • <meta http-equiv="X-UA-Compatible" content="IE=7" /> does not work for us: we are still getting the same number of errors per day

  • Is there an easy way via Fiddler/whatever to figure out exactly where the 4096 byte is in the response?

    Thanks!

  • @Jeff: Sure thing.

    1> Update to the latest version of Fiddler.

    http://www.fiddler2.com/dl/fiddler2betasetup.exe

    2> For any given response, go to the HexView tab and scroll down until the status bar indicates "4096 (0x1000) of body"

  • Might be a stupid question, but is there an easy way to translate where you are in the hex view to where that corresponds to in the Raw/Text/whatever view?

  • @Jeff: Not really. The problem is that the other views interpret CRLF (end of line) as just that, while in the hex view, they're rendered as encoded bytes.

  • We are constantly hitting this problem. We don't use the "base" tag and also don't have the META tag in our page and fiddler shows we have "Content-Type: text/html; charset=utf-8" so we must be hitting one of the other causes of this bug.

    Thanks for looking at it though. Hopefully you find a solution. As I am now putting in some special code into our exception handling mechanism to not be reported on this bug anymore as it is spamming my mailbox with exception emails.

  • Hi Eric,

    We are seeing this problem, but ONLY in our production environment.  I am unable to reproduce in any other environment.

    IIS configuration is identical on production and staging/dev.  The only difference is that Production is load balanced.  I've verified that the machineKey information is identical on both machines and session variable are stored in a Session database on SQL.

    Any thoughts?

  • We are in the same situation as Chris.  Our production servers are load balanced.  We are significant number of 404 errors due to the bug.  

    Just like others here we cannot reproduce the bug on our developer servers.  

  • I also noticed that when I review my IIS logs, I find entries that dont necessarily match the date/time stamp of the error in the event log.  

    When I am able to find the errors, I see thats that these requests return a 200 status????

  • Jim T -- are you using Google Campaign tracking by any chance?  Im seeing that most of my requests that contain this information is throwing the exception in the logs, but normal generic traffic to the site is not -- Im hoping this explains why DEV/STAGE and Prod are showing 2 different behaviours

  • @Chris/JimT: As our developers investigated the problem, we found that there are a number of timing scenarios which can result in the buggy behavior seen when the parser restarts.

    Unfortunately, this means that in practice, there isn't any viable workaround for web content that would resolve all forms of the problem.

Page 5 of 8 (116 items) «34567»
Leave a Comment
  • Please add 2 and 4 and type the answer here:
  • Post