IEInternals

A look at Internet Explorer from the inside out. @EricLaw left Microsoft in 2012, but was named an IE MVP in '13 & an IE userAgent (http://useragents.ie) in '14

Proxy-Authentication breaks many applications

Proxy-Authentication breaks many applications

  • Comments 6

When I first joined Office, I worked on the team responsible for delivering Help, Templates, and ClipArt into the client applications. As we were testing our work in various simulated customer environments, we found a big problem. At least one big customer (tens of thousands of licenses) had a network environment in which their users were forced to enter a username and password in order to authenticate to the proxy server. Without authenticating to the proxy, all HTTP/HTTPS requests were blocked.

Now, this was a fairly uncommon architecture, even then, and is perhaps more so now. In most environments, either the proxy server doesn’t require authentication, or the proxy relies upon the NTLM/Kerberos authentication schemes which permit users’ Windows logon credentials to be automatically used to respond to challenges from the proxy server. Environments that relied upon BASIC or DIGEST authentication require that the user explicitly submit their credentials, typically once per process (because most networking components, e.g. WinINET would cache these credentials for the lifetime of the process).

The problem with my features in Office was that they all passed the INTERNET_FLAG_NO_UI flag to WinINET, or ran atop WinHTTP, which explicitly doesn’t include any user-interface, including dialogs. The result of this was that in an environment with a BASIC/DIGEST proxy, all requests failed. In order to work properly in such environments, the application must itself supply the needed credentials to the network stack (e.g. for WinINET, call InternetSetOption, passing the INTERNET_OPTION_PROXY_PASSWORD and INTERNET_OPTION_PROXY_USERNAME option flags) to avoid the need to prompt the user.

I added a new rule to Fiddler that made it simple to test products for this problem: 

image

When the Require Proxy Authentication box is checked, Fiddler automatically responds to any request lacking a Proxy-Authorization header with a HTTP/407 response containing a Proxy-Authenticate header specifying the authentication scheme required:

GET /ua.aspx HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)
Host: www.enhanceie.com


HTTP/1.1 407 Proxy Auth Required
Connection: close
Proxy-Authenticate: Basic realm="FiddlerProxy (username: 1, password: 1)"
Content-Type: text/html

<html><body>[Fiddler] Proxy Authentication Required.<BR> </body></html>

A client that supports manual proxy authentication will then prompt the user for the username and password:

image

The client will then reissue the same request, supplying the provided credentials (base64-encoded) in the Proxy-Authorization header:

GET /ua.aspx HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
Proxy-Authorization: Basic MTox
User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)
Host: www.enhanceie.com

If the client fails to collect the credentials, it will typically treat the HTTP/407 response as fatal and will show an error message or fail silently.

When you try this, you can find broken scenarios all over. For instance, when I tried to post this blog using Windows Live Writer, the following error message was shown:

image

Afterward, I was prompted to re-enter my credentials for the web server—there was no way to supply the credentials required by the proxy!

Sometimes, an otherwise failing scenario may pass depending on what happens earlier in a process. For instance, if you enable the Fiddler rule, then launch IE to about:blank you will find that Search Suggestions from the Address bar don’t work, showing “An error occurred.”

image

Notably, if you subsequently navigate the tab to a web page, IE will prompt you for proxy credentials using the CredUI dialog box shown above. After you supply those credentials, the Search Suggestions feature starts working—that’s because the proxy credentials are cached for the lifetime of the process.

In other cases, failure are silent and there’s no notice to the user. For instance, many background updaters are based on BITS/WinHTTP and will fail silently when a HTTP/407 is encountered. Similarly, Windows’ CAPI component’s Certificate Revocation Checks will fail because the svchost.exe process doesn’t have the required proxy credentials.

If you need to sell your software into an enterprise that uses proxies, or just want to make your software robust against even uncommon network configurations, be sure to test manual proxy authentication scenarios!

-Eric

  • Back in 2006, I once suggested a way for Windows to handle proxy authentication better to Windows Networking team in Connect (the address was connect.microsoft.com/WNDP, if I'm not mistaken). But they never responded to it, and somehow the team page was missing/removed from Connect, and so did with my feedback.

    My suggestion was entitled "Windows Network Architecture for Proxy Authentication". Basically, I was suggesting to move proxy authentication support down from application level to Windows network level.

    Well, I have honestly no idea if it would be technically feasible, but I hoped (and still hope) it could be somehow implemented in Windows vNext. So here's the full idea:

    ==== pasted from my archive =====

    - Move support for proxy server authentication down to the Windows networking level

    - Create a network connection manager so that a user can have a multiple list of connections. He then can set a default connection he wants to use for ALL application that needs Internet connection, whether direct ones (I mean multiple connections) or behind proxy servers (I also mean multiple ones). Specifically say about proxy server, please provide a way so that a user doesn’t keep to be prompted for such credential; not as you can see in IE now (it made me tired to enter the credentials over and over again in every time I launched IE). As you can see in every download managers, for example Free Download Manager, it provides a way to set a proxy authentication credential; simply to say: SET ONCE, CONNECT EVERYTIME, not SET ONCE, PROMPT EVERYTIME nor SET ONCE, FAIL EVERYTIME.

    Here’s the example:

    Connection List:

    -- Direct Connection A

       Properties of connection A

    -- Direct Connection

       Properties of connection B

    -- Proxy Server C (Default Connection)

       Proxy server address: XXX, user name: YYY, password: ZZZ

    -- Proxy Server D

       Proxy server address: AAA, user name: BBB, password: CCC

    Note that XXX and AAA include DNS/IP Address plus port number and support IP version 6 addresses.

    - Provide a way (using API maybe) that an application can choose to connect to the Internet using other than the default connection. This creates an abstraction on how an application should be connected to the Internet and leaves the detail of the connection to Windows.

    ========

  • @Maximilian-- Indeed, that idea has arisen periodically for the last decade or so. As I mentioned, manually-authenticating proxies have always been rare (and some of those who use them would not want this "automatic" behavior). These are two of the key reasons why this investment has never been made.

  • Having the proxy at the application level causes very many issues in the IT world... moving it below layer 7 would solve so many headaches.

    The key problem is that every application needs to deal with the "system default proxy", and if it doesn't then there is often no solution for the end user (other than asking the vendor to "fix" their program).

    Case in point: Most video players running in Microsoft Silverlight will fail when behind any kind of proxy (we're behind an authenticating proxy that transparently uses the user's Windows credentials).  Our company has several sites with Silverlight training videos, and we need to give users static routes to the video hosts as the Silverlight Player does not respect the system default proxy.

    It's a ridiculous situation really... and if Microsoft is capable of fixing it, then it would make a lot of people very happy.  (Looking at this as an issue that only affects manually authenticating proxies ignores a huge chunk of the problems with proxies at the application level.)

  • New KB Article: "Using authenticated proxy servers together with Windows 8" - support.microsoft.com/.../2778122

    Summary: Don't.

  • For my Windows 7, I ended up with using Squid to act as an intermediate proxy server, put parent proxy server's basic authentication info into Squid configuration file, redirected the whole internet connection to it, and it just works without the annoying prompts.

    Haven't validated it yet against Windows 8/8.1, but I think it should just work as well.

  • If you have a .NET application that fails, you may be able to update its config file:

    <system.net>

       <defaultProxy useDefaultCredentials="true" >

       </defaultProxy>

    </system.net>

    chentiangemalc.wordpress.com/.../case-of-the-system-net-webclient-407-proxy-authentication-required

Page 1 of 1 (6 items)
Leave a Comment
  • Please add 4 and 6 and type the answer here:
  • Post