IEInternals

A look at Internet Explorer from the inside out. @EricLaw left Microsoft in 2012, but was named an IE MVP in '13 & an IE userAgent (http://useragents.ie) in '14

Vary with Care

Vary with Care

  • Comments 19

About the Vary Response Header 

As described in the HTTP/1.1 specification (RFC2616), the Vary response header allows a cache to determine if a cached (still fresh) response may be returned for a subsequent request, based on whether or not the new request's headers match those that were sent when the the previously response was originally cached.

  The Vary field value indicates the set of request-header fields that
  fully determines, while the response is fresh, whether a cache is
  permitted to use the response to reply to a subsequent request
  without revalidation. For uncacheable or stale responses, the Vary
  field value advises the user agent about the criteria that were used
  to select the representation.

The Problem

Unfortunately, the WinINET caching engine (below Internet Explorer and other applications) does not cache outbound request headers.  This limitation makes it impossible for WinINET to perform the request-header matching algorithm. 

Hence, Internet Explorer is conservative and generally will refuse to return a cached Vary response for a new request, except under special circumstances, as detailed below.

Internet Explorer 6

Internet Explorer 6 will treat an uncompressed response with a Vary header as completely uncacheable, unless the Vary header contains only the token User-Agent.  Hence, a subsequent request will be made unconditionally, resulting in a full re-delivery of the unchanged response. This results in a significant performance problem when Internet Explorer 6 encounters Vary headers.

Note: IE6 will ignore the Vary header entirely if the response was delivered with HTTP Compression; the header is dropped when URLMon decompresses the cache file on WinINET's behalf.

Internet Explorer 7

For Internet Explorer 7, the problem was not eliminated, but its impact was mitigated in some common cases. 

When evaluating a cached response that has a Vary, IE7 can make a conditional request (e.g. If-Modified-Since) rather than an unconditional request.

In order to take advantage of this improvement, the original response must contain an ETag.

Even though revalidation of cache response will require one round trip to server, it is still a significant improvement if server responds with a HTTP/304, because the response body is not transmitted.

Note, WinINET will remove the Vary: Accept-Encoding header if it decompressed the response.  Therefore, you should only send a Vary: Accept-Encoding header when you have compressed the content (e.g. Content-Encoding: gzip). 7.21.2009: see the comments below for a caveat on this.

Best Practices

  • Never send Vary: Host.  All responses implicitly vary by hostname, because the hostname is a part of the URI, and all requests vary by URI.
  • Only send a Vary: Accept-Encoding header when you have compressed the content (e.g. Content-Encoding: gzip).

7/14/2010 Update: Improved in IE9; IE9 will ignore Vary: Accept-Encoding, Vary: Host, and Vary: User-Agent, or any combination of these. You should still send an ETAG if you cannot avoid using VARY responses.

  • In a response, Vary indicates content negotiation is occurring, so "Vary: Accept-Encoding" should be sent even when the non-encoded version is sent.

    If you don't, proxies may end up caching only the non-encoded version. <a href="http://wiki.squid-cache.org/KnowledgeBase/VaryNotCaching">Squid does this</a>.

    What I haven't figured out is what to do about IE6 without the "SV1" in the UA. You can't reliably send gzipped JS to it, but if you leave off Vary and the request passes through a proxy, you've eliminated the ability of the cache to store the gzipped version.

  • @Steve Clay: From a protocol-perspective, of course you are correct. The point of this post is to explicitly state that using Vary: Accept-Encoding on uncompressed responses hurts performance in the most popular user-agents.

    Squid could work around their limitation in the most common case (e.g. don't blindly delete compressed, cached variants that are still fresh).

    An IE6 user who is current on patches should be able to properly decompress any response -- XPSP2 (identified by the SV1) shouldn't be required. In contrast, an IE6 user who is not current on patches has probably already been pwned by the bad guys.

  • You wrote : "unless the Vary header contains only the token User-Agent or Accept-Encoding."

    And if the Vary header is:

    "Vary: Accept-Encoding,User-agent"

    does IE6 treat the response as uncacheable?

    Thanks in advance

  • Eric,

    I'm starting to think that IE6's behaviour is preferable to IE7's.

    Since it (presumably) controls the UA and AE headers, and doesn't allow their modification (using XHR, etc.), IE6 can safely strip Vary: UA and AE, and treat those responses as fresh. IE7, OTOH, forces revalidation on these responses.

    Why not combine the behaviours and force revalidation on anything with a Vary that's *not* AE or UA? That seems like the most performance and safest thing you can do without remembering request headers...

    Also, I don't think it's as easy as you say for Squid to correct their behavour; there's no simple way for them to differentiate between a server that doesn't set correct headers and one that has stopped supporting compression.

    Cheers,

  • @Mark: IIRC, IE7 & IE8 also ignore the Vary: User-Agent directive.

    It's also important to recall that IE6 doesn't simply revalidate Vary'ing responses; it doesn't cache them at all.

  • Hi Eric,

    Right, but IE6 will cache something that varies on AE or UA, right? Do 7 and 8 also ignore Vary: Accept-Encoding? If so, this isn't a big problem in the common(est) case...

    Cheers,

  • Can you guys mention IE8 and IE9 preview expected behaviour?

  • @Mark: No, IE7 and IE8 will not ignore "Vary: Accept-Encoding" **unless** the response had a non-identity "Content-Encoding" (aka "if it was compressed"). Hence, if you send a Vary: Accept-Encoding header without compressing your content, it hurts performance in IE7/IE8.

    @Richard: The first IE9 Developer preview behaves the same way as IE8 in this regard.

  • hi Eric,

    Txs for the solid insight, again.

    can you please asnwer the question Benjamin posted on July 30:

    if the Vary header is:

    "Vary: Accept-Encoding,User-agent"

    does IE6 treat the response as uncacheable?

  • I've updated this post with updated analysis of IE6 behavior after code inspection. A future update to this post will cover IE8 and IE9 behavior.

  • I've got a question as someone who's still learning how HTTP caching works. Let's say a resource is set up as follows:

    - Normally, send "Vary: Accept-Encoding, User-agent"

    - If client is IE8 or earlier, and if content is not compressed, don't send the vary header, but send "Cache-Control: private"

    Wouldn't that prevent proxies from storing the uncompressed version?

    And, a separate question: if we're talking about a gzipped page, and if IE supports gzip anyway, why would uncompressed content be sent to IE (with the Vary header) in the first place?

  • @Dan: Yes, Cache-Control: private tells a (shared) proxy not to cache the resource.  

    Just because the client requests GZIP'd content doesn't mean that the server must give it back (for instance, many sites send the first response without compression and spin up a background thread to compress the content and cache it on the server for later use). Additionally, just because IE supports compressed content doesn't mean that it will always ask for it-- users can disable compression, or edge proxies may strip the client request's Accept-Encoding header.

  • I have request being cached even if vary host is set.

    IE6 will send "if modified since" even if vary host is set, is this because I also have cache control max age public set ? Thanks

  • @Vincent: I suspect you're seeing this behavior because perhaps your response is compressed. As noted, IE6 entirely ignores Vary if compression is used. That's the only explanation I see. (I-M-S and If-None-Match will be sent for an expired cached response, or in IE7+ if a response requires validation due to Vary).

  • Just found out that the second server is sending Vary: host while in HTTP 1.0, I guess this is why IE6 can still send if modified since.. :)

    https://gist.github.com/713829 for headers

Page 1 of 2 (19 items) 12
Leave a Comment
  • Please add 3 and 8 and type the answer here:
  • Post