Please read my blog's comment policy here.
Today's post is a collection of technical tidbits about conditional HTTP requests and the behavior of IE's Refresh button. It's probably of limited interest to most readers, but if you need to deeply understand either of these topics, hopefully you will find it helpful!
Web browsers make two types of requests over HTTP and HTTPS—conditional requests and unconditional requests.
An unconditional request is made when the client browser does not have a cached copy of the resource available locally. In this case, the server is expected to return the resource with a HTTP/200 OK response. If the response’s headers permit it, the client may cache this response in order to reuse it later.
If the browser later needs a resource which is in the local cache, that resource’s headers are checked to determine if the cached copy is still fresh. If the cached copy is fresh, then no network request is made and the client simply reuses the resource from the cache.
If the browser later needs a resource which is in the cache, but that response is expired (older than its max-age or past the Expires date), then the client will make a conditional request to the server to determine whether the previously cached response is still valid and should be reused. The conditional request contains an If-Modified-Since and/or If-None-Match header that indicates to the server what version of the content the browser already has in its cache. The server can indicate that the client’s copy is still fresh by returning HTTP/304 Not Modified headers with no body, or it can indicate that the client’s copy is stale by returning a HTTP/200 OK response with the new version of the resource.
Recently, someone asked why they see so many conditional requests in their server logs; they're setting proper far-future Expires headers and thus wouldn't expect IE to make any conditional requests for these resources.
There are a number of reasons why IE might make a conditional request for an item that is already in the cache:
Now, in the case where the user refreshes a page, two of the multiple levels of refresh are relevant:
In the first case (Normal-Refresh), we will perform HTTP requests (conditional, if possible) to revalidate all of the resources on the page, regardless of freshness.
In the second case (Super-Refresh), we perform unconditional HTTP requests to redownload all of the content on the page, bypassing the cache altogether.
Notably, the latest versions of Firefox and Chrome both behave as IE does for both Normal Refresh and Super Refresh cases, so there’s a bit of a du jour standard for this behavior.
There's actually a third type of refresh, which occurs when the user simply puts focus back in the address bar and hits ENTER, as if they were navigating to the page again. In that case, IE will use OLECMDIDF_REFRESH_RELOAD | OLECMDIDF_REFRESH_CLEARUSERINPUT. The first flag allows the browser to pull content from the cache if it's still fresh, while the latter flag clears any input fields on the document and resets the scroll position.
Internally, OLECMDIDF_REFRESH_NO_CACHE maps to the URLMon binding flags BINDF_RESYNCHRONIZE|BINDF_PRAGMA_NO_CACHE while OLECMDIDF_REFRESH_COMPLETELY maps to BINDF_GETNEWESTVERSION|BINDF_PRAGMA_NO_CACHE. (OLECMDIDF_REFRESH_RELOAD doesn't set any URLMon flags.)
These URLMon flags get turned into WinINET flags, and that actually influences the network behavior.
These flags will influence WinINET’s behavior:
When the window.location.reload method is called, the different refresh flags are set depending on the value of the bReloadSource parameter:
Update: IE10 now will apply cache busting flags when XHR is used after a page is refreshed with CTRL+F5.
Nice explanation about Request stack.
We're getting HTTP/304 Not Modified in the following flow:
Normally the page's static resources (js, css etc) are long long time cached (30 days) and works fine.
As soon user opens a tab (IE 7 & 8) and opens an url with some windows or other authentication and logs in.
and comes back to the first tab and do a refresh, we get HTTP/304 Not Modified for the static files?
Any ideas why IE has this behaviour?
@Gopi: The bulk of this post is concerns IE's behavior upon a refresh; you should *expect* to see conditional requests for all of the page's resources upon Refresh, regardless of "other windows" "authentication", etc.
Eric, Regular & conditional request works as you've explained, no issues here. We can do any number of regular refresh there wont be any 304's, if we do conditional requests all the files are re-downloaded... all are ok.
Now as soon as we open another tab and do an auth and login (diferent sites) and come back to first tab, and now do a Regular Refresh, now we're seeing all this 304's for (only) static files...?!! Cannot fig why only after opening (with auth) a tab, the first site throws 304's.
@Gopi: I think you misunderstand. If you refresh, you SHOULD be seeing conditional requests and HTTP/304 responses. If a conditional request cannot be made (because no Last-Modified or ETAG header was on the original response) you should see an unconditional request and a HTTP/200 response.
Now, the behavior in the face of *non-refresh* navigations DOES differ if a new tab is open and the original response did not have headers that specified cache lifetime. This is due to the behavior of Heuristic Expiration in IE8 which had a bug such that the "once per session" logic was interpreted as "Once per session or until a new tab is opened."
If you have a repro URL, I'd be happy to have a look.
Eric, do you have more information or plan to write on how calls at the browser level map to wininet? For example when a user clicks on a link, how does that map behind the scenes - would it be the same as OLECMDIDF_REFRESH_RELOAD | OLECMDIDF_REFRESH_CLEARUSERINPUT
@Tony: Link navigation wouldn't map to a refresh constant. Trident passes BINDF_HYPERLINK to URLMon, which gets converted to INTERNET_FLAG_HYPERLINK for WinINET.
Thanks Eric! Is all this information available somewhere - or do we have to pick your brain everytime :)
[EricLaw]: Much of it is implied, but sadly not directly stated, in the MSDN documentation. I'm afraid picking my brain is often your best option.
Hi Eric, this is a great post and has filled in most of the large gaps in my knowledge.
One further question I'd like to understand the answer to is what IE does with the 304 response, other than getting the content for the original request from its cache? Does it do anything to "freshen" the cache following the 304, as if it just downloaded the content for the first time and was setting its expiry based on max-age? - therefore avoiding future conditional requests until the content is once again not fresh
Thanks in advance!
@DaveK: Great question! When it gets a 304, WinINET will update the content's expiration using the Cache-Control or Expires headers specified on the 304. It also updates the "last-checked" time inside WinINET's metadata. The code does not update anything else on the cached response.
We are facing issues with 304 which is resulting in Script Error on IE and the page does not load. We use the thirdpart FCKEditor in our pages, and the css, jss, and xml files have not changed in our side for a while. When ever users (not for all only for few) users hit the page, it does not load the text areas we have have in the webpage, troubleshooting this issue we found the following message. URL requested ... URL to the resource....then Staus 304.
Any help would be appreciated.
Can you please help understand why IE differs from FF and Chrome if user use F5 or refresh button to reload the page in following scenario?
Original request: Post, Original Response: PDF
New Request on Refresh/F5 in IE: Get
New Request on Refresh/F5 in FF and Chrome: Post
When you hit F5 and IE isn't showing HTML, the word gets infinitely more complicated, because it's up to the document object (e.g. the PDF DLL, in your case) to decide how it wants to interpret the request to refresh.
Wow...you've opened up a whole can of worms for me, but I'm hoping you can point me in the direction of what I should be looking for more than just thinking you can provide an answer. I'm a network guy, not a web developer, but we have an issue in our company that is driving me nuts.
We have a web application that users use. The application is behind a load balancer and there are three web servers that traffic is sent to. Essentially a user makes a request to http://site.com and gets redirected (via http 302) to https://site.com. From there the user enters their username/password/domain and selects login.
The problem I'm having is that a handful of users (all running windows 7 and all running IE9) will intermittently hit "Login" and the screen flashes and is back at the login prompt (read: it doesn't authenticate). The weird part is that no one has experienced the issue using FF or Chrome, but a good amount of people see it with IE9. I've had limited opportunity to debug it since I am one of the people that rarely if ever sees the traffic. I did do some packet captures, but it doesn't look like lost packets so since the traffic is HTTPS my captures don't do much for me. I have been using fiddler, but havn't been able to capture a failure since. I did ask one co-worker who experiences the issue to use the "network capture" in the IE tools (F12) and they captured a succeessful and failed login. The only thing I can see (and this jives with what MS claimed since we had them in house to run a bunch of tests) is that in the successful test the user hits "login" and then receives an https 302 redirecting them to "site.com/test.msv" and on the failed login attempt I just see an https 200 status "OK".
Interestingly enough (and the way I stumbled across this site) is that if the user opens another tab then they seem to be able to authenticate and login just fine.
Once again, the problem never occurs with other browsers, yet MS is adamant that one of the servers is sending back a bad header (or the load balancer, which is why I'm now involved)...yet the problem is so random and intermittent I'm at a loss as to where to focus on next.
Any guidance would be appreciated.
@David: I'm no longer at Microsoft. Ping me using the Help link in Fiddler and I'll help if I can.
In the F12 Tools' Network view that I see 304's on pretty much every static resource requested.
[EricLaw] One of the architectural limitations in the implementation of the F12 tools makes it difficult for them to tell the difference between when a resource was fetched from the cache without revalidation and when a revalidation/304 occurred. As a consequence, F12 shows 304s in many cases where network traffic didn't occur. Similarly, F12 doesn't properly show when the server used HTTP Compression. Fiddler does not suffer from these limitations.
PS It also appears now in IE10 that there is no combination of F5/Refresh/LocationBarPressEnter that will issue a conditional request issuing an If-Modified... type header any more
[EricLaw] If you have a public test page with this problem, I'd be happy to look at it. I cannot reproduce any problems here.