Please read my blog's comment policy here.
Over the last decade, I’ve come to learn a lot about web proxies, having chosen to implement my web debugger as a proxy. In today’s post, I’ll provide an overview of proxy-related information, including information on changes in Internet Explorer 11 / Windows 8.1.
Unlike on other systems like Mac OSX, Windows doesn’t really have the concept of a “system” proxy. Most applications respect the WinINET proxy setting, but a few do not.
Historically, most applications were built on WinINET and thus respected its proxy settings directly, but today more and more applications are built on WinHTTP (or a descendent architecture like BITS) and System.NET and thus may require additional configuration.
WinINET proxy settings are typically per-user rather than per-machine. This means that individual (even non-admin) users can set their own proxy settings without impacting the proxy settings of other user-accounts. WinINET can be configured to apply proxy changes on a machine-wide basis by creating a registry DWORD named HKLM\SOFTWARE\Policies\Microsoft\Windows\CurrentVersion\Internet Settings\ProxySettingsPerUser with value 0. Subsequently, changes to the proxy settings can only be made by elevated applications running with Administrative permissions.
The WinINET proxy setting can be configured in Internet Explorer by clicking Tools > Internet Options > Connections. On the connections tab, select a Connection and click the Settings button. For most users, the proxy is configured by pressing the LAN Settings button at the bottom of the dialog:
Settings Precedence: Part 1: While the proxy settings for the Dialup/VPN/RAS connection and the LAN settings button can be configured independently, internally WinINET will use the settings from any active/connected dialup/VPN/RAS Connection. Otherwise, the proxy setting specified in LAN settings is used. Many users incorrectly expect WinINET to first determine which network interface is used to establish a given connection before selecting a proxy; it doesn’t work like this.
When you configure your proxy settings, you’re presented with the following dialog box:
Settings Precedence: Part 2: The settings in this dialog box are presented in the order of their precedence. First, the Automatically detect settings option is consulted, next the automatic configuration script option is evaluated. If neither of those options is enabled or neither specifies a proxy to use, only then are the fixed proxy server settings consulted.
When Automatically Detect Settings is enabled, Internet Explorer performs a process called Web Proxy Auto-Detection (WPAD). This process consists of two phases:
Some browsers (e.g. Firefox) only undertake #1 and skip #2.
If the client is able to automatically detect a proxy server, it will attempt to download a proxy configuration script.
The user may also directly specify the URL of a proxy configuration script using the second checkbox in the dialog. The URL field below points directly at the target script (e.g. http://proxy.contoso.com/proxy.pac).
WinINET also offers support for another function FindProxyForURLEx, an extension which supports IPv6-aware proxy scripts.
Warning: proxy scripts is that they impact the Internet Explorer Security Zone determination. To summarize my long MSDN article and blog post, by default, if a proxy script is in use and returns DIRECT, the target site will be mapped to the Local Intranet Zone.
In IE11, the WinINET team has made some changes to attempt to improve interoperability of proxy scripts.
Internet Explorer 10 and other WinINET-based clients would use the specified proxy script. However, WindowsUpdate and any other application built on a WinHTTP-based network stack would silently ignore the specified proxy script because WinHTTP has never supported the use of the file:// protocol as the origin of a proxy configuration script.
In Internet Explorer 11, the WinINET team has disabled WinINET’s support for file:// based scripts to promote interoperability across network stacks. Corporations are advised to instead host their proxy configuration scripts on a HTTP or HTTPS server. As a temporary workaround, this block can be removed by setting the following registry key:
Key: HKLM\SOFTWARE\Policies\Microsoft\Windows\CurrentVersion\Internet Settings\ Value: EnableLegacyAutoProxyFeatures Type: REG_DWORD Data: 1
Keep in mind that this should only be a temporary measure, as this block was introduced for good reasons, and removing the block won’t magically fix your WinHTTP-based applications.
While you shouldn’t use a file://-based proxy script, if you do so, IE requires that you use the legacy “unhealthy syntax” for file URIs in the proxy configuration dialog.
As you can see, the “Healthy” syntax is incorrectly parsed as a relative URL.
The legacy “unhealthy” syntax works properly:
But, to reiterate, don’t use file:// to host your proxy script.
When downloading a proxy configuration script, it’s possible that the server will demand credentials from the client application. A problem arises if the server requests credentials using HTTP BASIC or HTTP DIGEST authentication, as these authentication methods require that the user respond to a credential prompt dialog box. (In contrast, NTLM and Negotiate authenticate silently).
Many applications cannot handle showing prompts as a part of this workflow and will fail silently. To promote interoperability, IE11 also blocks the use of Proxy Configuration scripts that require authentication. As a temporary workaround, this block can be removed by setting the following registry key:
Keep in mind that this should only be a temporary measure, as this block was introduced for good reasons, and removing the block won’t magically fix applications which cannot show UI prompts.
Generally speaking, you should attempt to migrate away from BASIC/DIGEST authentication both for access to the proxy script and for access to use the proxy itself. As I’ve noted previously, Manual Proxy-Authentication breaks many applications.
Beyond automatic proxy detection, the user may manually configure “fixed” proxies. By default, the simple UI is shown:
…but clicking the Advanced button shows more options:
You can either specify a different proxy server/port for each protocol, or you may use the default option which applies the same proxy server to all protocols.
The Exceptions box allows you to specify what hostnames are configured to bypass the proxy. Beyond hostnames, it supports two special tokens:
Note that Internet Explorer only supports SOCKSv4.
Fiddler and some other applications attempt to programmatically adjust the WinINET proxy setting. Rather than attempting to “poke” the registry directly, the proper way to update the proxy setting is to use the InternetSetOption API.
Using the API is straightforward, whether you’re using C/C++ or .NET.
As a part of Internet Explorer 10, proxy configuration was centralized in an existing system service (still labeled “WinHTTP Web Proxy Auto-Discovery Service”) so that proxy configuration code no longer needs to run in every process using WinINET (e.g. each IE tab is in its own process).
Unfortunately, as a part of this update, a regression was inadvertently introduced. Code that attempts to change the WinINET proxy setting during a system shutdown (like Fiddler’s shutdown code, for instance) will find that the APIs return success, but upon restart you will see that the proxy settings change was discarded. No fix for this issue (also present in IE11) is yet available. A possible workaround for this problem would be to create an out-of-proc COM server and run a method (DllHostKeepAlive) that doesn’t return until the client calls a shutdown method (CancelDllHostKeepAlive). This would open a DLLHost that can be used to adjust the proxy settings during shutdown.
There’s still quite a bit more to say about proxies. I’ll update this post as time permits and questions arise.
How do I inspect a proxy script obtained via WPAD?
Proxy scripts are simply text files that can be examined in any text editor. If you're looking for an actual PAC-script debugger, check out chentiangemalc.wordpress.com/.../c-proxy-pacdbg-pac-debugging.
Great article - we came across this as we are facing a problem due to this change and we have a valid reason for wanting to have a file based proxy.
Our product uses a bunch of URLs that go to our data center and integrates with a bunch of URLs that belong to the clients of our product. Our URLs have our proxy rules and client URLs have their own rules -- we make both work by downloading both scripts and merging them to a combined script file on the desktop. We then call InternetSetOption and point it to the local file... It seems this is no longer working
I am behind a proxy server providing free internet via my town hall. Since upgrading to windows 8.1 and explorer 11 I receive error 8024401C when I try to connect to windows update. I believe this is a proxy server problem is there anything you can suggest to help? thanks
EricLaw - What are your proxy settings? Did you download and run the WindowsUpdate Troubleshooter?
"WinHTTP-based applications only respect the WinINET setting when configured to do so (netsh winhttp import proxy)."
This isn't quite accurate. For starters, the netsh command only imports static proxies and doesn't handle auto-detect or autoconfig.
Also the netsh config is really only meant for WinHttp apps running as system without a user to impersonate.
A well written WinHttp app will, figure out a way to get user to impersonate if they aren't running in a users process, and use WinHttpGetIEProxyConfigForCurrentUser and then plug in the settings to WinHttpGetProxyForUrl if needed.
There's a sample for how to do this on msdn: code.msdn.microsoft.com/.../WinHTTP-proxy-sample-eea13d0c
In WPAD, DHCP is prioritized first and DNS second.
A common troubleshooting point we often hit is that the Content-Type of the proxy script is not "application/x-ns-proxy-autoconfig" or that the file extension is not one of .js .pac or .dat which causes problems with WinHttp's implementation of WPAD
It looks like IE10 and above doesn't look at dialup/VPN/RAS proxy setting any more, it seems LAN setting always override that. The testing shows this behaviour. Could you confirm if this is expected?
[EricLaw] The WinINET team indicates that this issue was addressed by http://www.microsoft.com/en-us/download/details.aspx?id=40532.
I am using an Automatic Configuration Script for my proxy settings that is working fine for IE, but I have a .NET application that attempts to make an SSL connection to a website. The application doesn't see any Proxy settings and tries to connect to the website directly. If I populate the Fixed Proxy Configuration, the application uses the proxy and connects without issue. What are the implications with using BOTH the PAC script and the Fixed Proxy settings?
[EricLaw]: Settings "flow" from the top of the box downward. So, if the PAC script is inaccessible, then WinINET/IE will fallback to using the fixed proxy settings instead of going direct.