Please read my blog's comment policy here.
Recently, a user sent in the following screenshot of a security warning they encountered when attempting to download the Microsoft Zune software:
Obviously, we immediately attempted to reproduce the reported problem, and we found we were unable to do so – the program was recognized as legitimate software and no security warnings were shown.
However, the user in question was able to provide a big clue as to what went wrong. When they tried to download the file with Chrome, they also saw a security warning:
When they (unwisely) clicked through the warning message and elected to “Run" anyway, the program failed to launch and the following dialog was shown:
This message indicates that the downloaded file was corrupt. It turns out that the local downloaded copies of ZuneSetupPkg.exe were only 6 megabytes in size, although the actual file is supposed to be 119mb. With this information, it was then easy to understand why the security warning was shown.
The file transfer was interrupted when only a small portion of the file had been been downloaded. The user didn’t notice that the file was smaller than expected (as I mentioned last month, no common browser warns the user about incomplete file downloads). Before opening the incomplete file, it was analyzed by the SmartScreen Application Reputation feature. SmartScreen determined that the file was unsigned (the signature comes at the end of the file, beyond the portion that the user had successfully downloaded) and the hash of the downloaded file did not match any known-safe program. As expected, the user was then warned that the file was unrecognized, and of a potentially dangerous type.
If you ever encounter this problem, simply click Delete from the warning notification and download the file again. If you see a warning again, check the size of the file and contact the owner of the website for help. If you are behind a proxy, you may need to talk to your IT Administrator about flushing a corrupted or incomplete download from your proxy cache.
Well, this is unfortunate -- I think UAs should try harder to do the right thing here. For IE a smallish improvement would be to remember that it *thinks* that the download was truncated, and to adjust the aforementioned message accordingly.
[EricLaw: The "right" thing is always a bit of a loaded statement. But yes, as I expressed in my other post, I'm very dismayed that virtually all browsers ignore incomplete downloads, which significantly increases the interoperability risk of any browser choosing to treat "fewer-bytes-than-expected" as an error.]
Eric, is it possible to have IE to:
- verify whether a downloaded file is completely downloaded or just partially downloaded (truncated)
- if the file is partially downloaded, notify user that it's truncated and offer an option to re-download it, instead of showing it as "a.exe is not commonly downloaded and could harm your computer."
Don't you think the error message irrelevant to the situation: the file is partially downloaded, not that it's completely downloaded and then could harm your computer?
- after a failed (partial) download happen, force to download a fresh copy from the server directly instead of from middle-party caches, such as proxy servers you mentioned?
@Maximilian: You should go read the post I linked above; the text is "as I mentioned last month".
Eric, instead of relying on Content-Length, is there any mechanism in HTML-related protocols to verify using file hash, e.g.: SHA1, MD5, etc.? It seems Content-Length isn't reliable enough to justify the verification.
I notice these days, some, if not so many, web sites provide file hash for downloaded files. Personally, I manually use two steps to verify a downloaded file. First by its digital signature and second by file hash, if the digital signature doesn't exist. I'm talking about bringing this manual steps into automatic steps in IE, although I'm not sure if it's currently technically possible. Some download managers I know use this technique.
BTW, why do I have to click Post button TWICE to submit comment here in MSDN blogs? The first submit always doesn't work.
@Maximilian: in general, Content-Length *is* reliable, otherwise HTTP wouldn't work. I believe, Eric is referring to edge cases (hopefully!). The right thing to do here is IMHO to notify the user that the download is likely truncated, but to let him/her proceed (in order to deal with broken servers). Displaying a misleading unrelated warning is not really helpful...
@Julian: I'm not saying Content-Length isn't reliable, but I said it isn't reliable *enough*. The case Eric told here is an example of how unreliable *enough* it's, but I have more concern on the repercussion: Content-Length failure is used as a justification of an partially downloaded app that could harm the computer. Is it relevant and helpful? I don't think so.
I agree that it's reliable most of the time, but there are some (edge) cases when it isn't *that* reliable. Thus, in addition of using Content-Length "verification", it's also better to use file hash verification in case Content-Length doesn't work as expected, especially for this file download case.
@Julian: I'm inclined to agree, but sadly there's not a lot of data about the incidence of "otherwise working" scenarios with bad C-L and we found at least a few in our IE9 testing. This is an area that we're pretty likely to revisit.
@Maximilian: I think you're confused. Partial downloads are not "used as a justification" that they "could harm the computer." Instead, partial downloads are inherently unrecognizable, and unrecognized executables are inherently of much higher risk than those that are recognized. The fact that the executable will subsequently fail to execute isn't something that can be readily determined before the warning is presented.
HTTP has a "Content-MD5" header but it's basically unused in practice, and as a percentage, very few sites make this information available in other ways (e.g. human readable).
As I stated originally, the user best practice of clicking "Delete" will typically resolve the problem.
As to why you often have to click "Post" twice-- Do you have more than one tab open to MSDN blogs? I'm guessing that there's a CSRF mitigation in place that triggers if more than one tab is open on an MSDN post. But generally, no, the blog software isn't very good.
OK, Eric. I should have been clearer about what I said.
I think of two possibilities. The first is a completely-downloaded malware app (a non-corrupted bad app), and the second is a partially-downloaded good application (a corrupted good app), like the case you presented.
Don't they both cause IE to trigger the same error message: "The program is not commonly downloaded and could harm your computer."? What I meant is that they both are different things happening because of different situations, but why do they have the *same* error message?
It's correct for the first case, but I don't think so for the second one.
In the second case, firstly, the error message is irrelevant to the actual situation happening. And secondly, it made me confused of what was actually happening and thus lead me to delete it but *not* tell me to re-download it. Have I been any clearer?
As to your question, no, I only have one tab opening to MSDN blog: only this post. And it's just happening (again) as I type this extra paragraph.
@Maximilian: No, these two situations do not trigger the same message.
If you attempt to download known malware, SmartScreen interrupts the download with a "This download is unsafe" message. Known malware explicitly *does not* show the "We don't know what this is, but the file type is dangerous" warning that offers a "Run anyway" link.
As for your problems with the blog software-- what browser/version are you using?
OK, then why doesn't IE *simply* say it's a corrupted file? And further, shouldn't IE add a Re-download action aside from the Delete one?
I'm using IE9 RTM on Windows 7 SP1 RTM (clean install and fully patched). I haven't tried it yet using FF nor Chrome.
@Maximilian: As I explained in this post, there's no way for IE to *know* that it's a corrupted file. And as I explained in the post I linked, claiming a file is corrupt/incomplete just because the C-L is incorrect is something that we considered, but no browser with significant marketshare currently does that, and we know the compatibility impact is non-zero.
Sorry, I've got no idea why you're seeing the problem with the blog software. Do you have any browser add-ons enabled (e.g. the Windows Live Signin Assistant)?
OK, thanks Eric for your great explanation so far. It helps me understand better the issue and the context around it. I really appreciate it.
The add-ons currently enabled are Shockwave ActiveX Control, Shockwave Flash Object, XML DOM Document, and Free Download Manager. I disabled Windows Live ID Sign-in Helper. IIRC, I had the same issue in IE8 and I still can't figure out what the root cause is.