Http Client Protocol Issues : WinInet

Using HttpWebRequest with Credential Manager

jpsanders — Thu, 03 Sep 2009 21:39:00 GMT

A little know fact is that the .NET framework will use the stored credentials in the Credential Manager when accessing a network resource if the credentials exist for that particular resource (host). I intend to clear up how this functionality works for the HttpWebRequests (you could extend this to WebService calls as well).

In Internet Explorer, you can access a website that challenges you for credentials by filling in a username and password when the dialog is presented and asks you for authentication. If you check ‘Remember my password’ these credentials will be stored in protected storage (Credential Manager). You can see the stored credentials by running this command: Control Keymgr.dll. These credentials are stored in protected storage in each user’s My folder. You can store these credentials by using control Keymgr.dll, by using the interface available in Internet Explorer or by using C++/C and the Credentials API’s (Such as CredWrite). If you do not check ‘Remember my password’ then these credentials are not stored in Credential Manager and .NET will not be able to use these credentials.

The .NET framework can use the currently logged on user’s credentials to authenticate but if you have a web site that requires different credentials you can use store these credentials in Credential Manager and .NET will use these credentials instead of the currently logged on user credentials. To illustrate this I put together a real simple sample.

The target web server in my sample is jpsandershv2003. I only enabled Windows Integrated Authentication in IIS on that box. I then created a User ID JPSGuest on that IIS machine. Finally I wrote a real simple .NET console application to create an HttpWebRequest and get the response.

HttpWebRequest aReq;

HttpWebResponse aResp;

aReq = WebRequest.Create("http://jpsandershv2003/") as HttpWebRequest;

aReq.UseDefaultCredentials = true;

aResp = aReq.GetResponse() as HttpWebResponse;

aResp.Close();

When you run this code it will first look in the credential manager first for the credentials for attaching to jpsandershv2003 and if found use those, otherwise the Currently Logged on User credentials will be used. In my example since I was logged on as jpsanders to my machine I expect that the credentials used should be my domain credentials jpsanders. I turned on logging and ensured the IIS log was logging the username and sure enough, jpsanders was authenticated in IIS.

My next goal was to see if I could change this default behavior and make it log on using the jpsandershv2003\JPSGuest user credentials. Using the command control keymgr.dll I was able to add the credential as a Windows logon credential:

Now when I ran the console application I saw success in the IIS log! I was logging in now as JPSGuest.

Taking a fiddler (http://fiddlertool.com) trace of Internet Explorer also confirmed I was using the stored credentials and these were NTLM credentials.

Domain: jpsandershv2003
User: JPSGuest
Host: JPSANDERS3

Can I store credentials and have it use Kerberos? In theory yes, simply type domain based credentials in. However I could not test this because I only have one logon to my domain and you cannot save your currently logged on credentials in the Credential Manager.

What about basic and digest? NO, this would be a huge security hole. Can you tell me why (it should be obvious)?

Note that the credentials used are independent of the PORT used. So if I change the server to bind HTTP traffic on port 8089 then change my code to access http://jpsandershv2003:8089 the code will still access the web site using the stored credentials.

To revert back to the interactive user, simply delete the credentials stored for jpsandershv2003.

Further observations

What is the second radio button for (A Web site or program credentials)? This is for programs that are writing against the credentials API directly. Some examples of this are Terminal Server (RDP) and Windows Live credentials. Here are some great links about the APIs:
http://msdn.microsoft.com/en-us/library/aa374789(VS.85).aspx
http://msdn.microsoft.com/en-us/library/aa480470.aspx

NOTE: Because the credentials are stored in the context of the logged on user, this technique will not work for non interactive user accounts such as the service accounts and ASP.NET applications.

Please send me a quick note if you found this Blog helpful!

WinInet used in Thread Impersonation

jpsanders — Thu, 09 Jul 2009 22:48:00 GMT

First and foremost, this is NOT supported for reasons stated in this article: http://support.microsoft.com/default.aspx/kb/238425. Primarily, problems occur because of information that is stored in the HKEY_CURRENT_USERS registry key and because of threading concerns. Furthermore, due to security implications and the design of WinInet in Windows 7, this absolutely will not work in Windows 7. Finally, in Vista, there is a Low IL cache as well as the normal (Medium IL) cache so this sample is ineffective at best. This blog entry is mearly an excercise to help you understand how the Registry works with thread impersonation as I discovered when working on a WinInet problem.

That said, if you have a service running in the Local System Account, you might be able to work around some of the problems by ensuring the registry is loaded with the currently logged on user's registry information. If the currently logged on user has used the browser, most information should be populated in that user's registry.

If running with UAC enabled and you attempt to enumerate the cache while running under the Local System account you will fail and GetLastError will return ERROR_INVALID_PARAMETER. This error is thrown because internally there are some registry settings missing that WinInet expects to be there for every user (again, WinInet is not designed, tested or supported when running under the System Account or when using thread impersonation). I found however if you use thread impersonation and load the registry HKCU for the currently logged on user, you can access the cache. I do not know if this will work in all scenarios and in future WinInet releases, but I was able to get it to work for me. I used PsExec from Sysinternals to run my tests.

Details
I downloaded and installed PsExec from http://technet.microsoft.com/en-us/sysinternals/bb896649.aspx

I created a new C++ Console application in Visual Studio and added code to:
1. Get the Currently Logged on user token in the SessionID from where this program was launched under the SYSTEM account using WTSQueryUserToken
2. Impersonate this user using ImpersonateLoggedOnUser
3. Load the impersonated HKU registry in HKCU by using RegDisablePredefinedCache
4. Iterate through the IE cache using FindFirstUrlCacheEntry and FindNextUrlCacheEntry
5. I added code that would also show the cache if the WTSQueryUserToken failed. This will fail if you are running the program from the command line (no PsExec). This allows you to see the case running in the context of the current user.

I then added the WinInet.lib and Wtsapi32.lib to the linker input files and built the project.

I then tested the code with PsExec using: PsExec –i –s <<full path and program name>>. I opened a command prompt with Administrator permissions to run the tests. An interesting use of PsExec allows you to see the cache when running as an Low IL process: PsExec –l <<full path and program name>>. Note that this is a different set of values because the Low IL cache is in a different location then the Medium (normal) IL cache.
For details on protected mode cache see: http://blogs.msdn.com/ie/archive/2006/02/09/528963.aspx

Here is the code for your enjoyment. Of course you would need to add your code to test the success of calls, trap exceptions etc... But it should get you started:

C++ code listing for sample (Copy Code):

// ImpersonateWinInet.cpp : Defines the entry point for the console application.
//

#include "stdafx.h"
#include <windows.h>
#include <WinInet.h>
#pragma comment(lib, "WinInet.lib")
#include <Wtsapi32.h>
#pragma comment(lib, "Wtsapi32.lib")
#include <iostream>
#include <conio.h> // for _getch
int showCache()
{
    // another gotcha... The actual cache hit here depends on the IL that the program is running under.
    // to see this run psexec with the -l option in a cmd prompt
    // Local variables
    DWORD cacheEntryInfoBufferSizeInitial = 0;
    DWORD cacheEntryInfoBufferSize = 0;
    int *cacheEntryInfoBuffer = 0;
    INTERNET_CACHE_ENTRY_INFO *internetCacheEntry;
    HANDLE enumHandle = NULL;
    BOOL returnValue = false;
    int aiNumEntries=0;
    DWORD dwError;
    // get the size of the buffer required
    enumHandle = FindFirstUrlCacheEntry(NULL, 0, &cacheEntryInfoBufferSizeInitial);
    if (enumHandle == NULL && ERROR_NO_MORE_ITEMS == GetLastError())
    {
        return aiNumEntries;
    }
    cacheEntryInfoBufferSize = cacheEntryInfoBufferSizeInitial;
    internetCacheEntry = (INTERNET_CACHE_ENTRY_INFO *)malloc(cacheEntryInfoBufferSize);
    enumHandle = FindFirstUrlCacheEntry(NULL, internetCacheEntry, &cacheEntryInfoBufferSizeInitial);
    if (enumHandle == NULL)
    {
        std::cout << "could not get first URL entry. Error :" << GetLastError();
        // ERROR_INVALID_PARAMETER is thrown because of incorrect registry info
        return aiNumEntries;
    }
    returnValue=true;
    while(1)
    {
        cacheEntryInfoBufferSizeInitial = cacheEntryInfoBufferSize;
        if (returnValue)
        {
            if (internetCacheEntry != NULL)
            {
                std::wcout<<(LPWSTR)((INTERNET_CACHE_ENTRY_INFO *)internetCacheEntry->lpszSourceUrlName)<< _T("\r\n");
                returnValue = FindNextUrlCacheEntry(enumHandle, internetCacheEntry, &cacheEntryInfoBufferSizeInitial);
                aiNumEntries++;
            }
            else
            {
                // this should not happen!
                break;
            }
        }

        dwError = GetLastError();
        if (!returnValue && ERROR_NO_MORE_ITEMS == dwError)
            break; //now more items!

        // if buffer not big enough, grow it
        if (!returnValue && cacheEntryInfoBufferSizeInitial > cacheEntryInfoBufferSize)
        {
            cacheEntryInfoBufferSize = cacheEntryInfoBufferSizeInitial;
            // note test for buffer here (OOM condition)
            internetCacheEntry = (INTERNET_CACHE_ENTRY_INFO *)realloc(internetCacheEntry, cacheEntryInfoBufferSize);
            returnValue = FindNextUrlCacheEntry(enumHandle, internetCacheEntry, &cacheEntryInfoBufferSizeInitial);
        }
    }
    free(internetCacheEntry);
    std::wcout<<_T("Number of entries processed: ")<<aiNumEntries<<_T("\r\n");
    return aiNumEntries;
}

int _tmain(int argc, _TCHAR* argv[])
{
    // note, this below will get the session id that this program was run from.
    DWORD dwActiveSessionId = WTS_CURRENT_SESSION;
    HANDLE hUserToken = INVALID_HANDLE_VALUE;
    DWORD dwErr=0;

    //showCache();
    BOOL bSuccess = WTSQueryUserToken(dwActiveSessionId, &hUserToken);
    if (bSuccess)
    {
        std::cout << "Got User Token\r\n";
        bSuccess = ::ImpersonateLoggedOnUser(hUserToken);
        if (bSuccess)
        {
            std::cout << "Impersonating\r\n";
            // This is necessary because we want to load the impersonated user registry into HKCU
            // comment this out whey running under the system acct and you will get ERROR_INVALID_PARAMETER
            if( ERROR_SUCCESS==RegDisablePredefinedCache())
            {
                showCache();
            }
            CloseHandle(hUserToken);
        }
        else
        {
            std::cout << "Failed Impersonation\r\n";
        }
    }
    else
    {
        std::cout << "Did not Get User Token\r\n";
        dwErr= ::GetLastError();
        std::cout << "Error" << dwErr << "\r\n";
        if (ERROR_PRIVILEGE_NOT_HELD==dwErr)
        {
            std::cout << "To call WTSQueryUserToken successfully, the calling application must be running within the context of the LocalSystem account and have the SE_TCB_NAME privilege\r\n";
            std::cout << "Trying in the context of the user running this program...\r\n";
            std::cout << "Hit a key or enter to start";
            _getch();
            showCache();
        }
    }

    std::cout<< "Hit a key or enter to exit";
    _getch();

    return 0;
}

I hope this sample helps you out. Please drop me a comment if you found this useful!

Understanding Connection Limits and New Proxy Connection Limits in WinInet and Internet Explorer

jpsanders — Mon, 29 Jun 2009 18:01:00 GMT

Because of RFC 2616 section 8.1.4 (http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html) we have traditionally limited the number of persistent connections to 2 per server. This is because of the strong language in this RFC: " A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy."

Practically speaking with today's modern web servers, there is no reason for such a small number of client connections to a server. This is especially true with the rich WebPages and WebBased applications hosted in WebPages. Indeed, at times you can get into a situation where you are attempting to download resources for a WebPage and a script is executed to download additional data giving the appearance of a web browser. The ability to download several resources simultaneously has huge advantages when it comes to downloading resources quickly and gives the ability to provide a more responsive web experience.

While you could always change the number of connections WinInet (and Internet Explorer) used by default, in the past, you could not also control the number of Proxy Connections separately (also covered in RFC 2616). There is now a setting that will allow you to set the proxy connection limit independent from the number of persistent connections to a server.

Here is a brief Q&A I put together to help you get your head around these settings:

What does setting the max Server and Proxy connections do? This limits the number of possible connection at one time to a given host either direct or through a proxy.

What happens if I try to make 4 connections and the default number of persistent connections (proxy or server) is set to 2? Two connections will be made and two will be put in a queue to ‘wait’ for an available connection to be freed. Once one of the first two connections gets a response, another ‘waiting’ connection will use that available connection in a FIFO manner.

If my app has 2 requests to one host and 2 to another, does that mean only 2 connections at a time will be used total, and there will be two connections waiting? No, this means that 2 connections per host will be used so in this example there will be no connections waiting.

Why limit the proxy connections in WinInet to begin with? We always have as a function of limiting the HTTP connections to the end point server. We did this because the RFC 2616 tells us to:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html (see 8.1.4 Practical Considerations)

Why add a new limit for the proxy connections? In the event you need to customize the number of proxy connections. The default is set to 4 which is enough to satisfy the defaults for WinInet which is 2 for HTTP 1.1 servers and 4 for HTTP 1.0 servers.

What was it before? The value did not exist before so it is the same as the server settings. If the proxy was Http 1.0 it was 4. If it was http 1.1 it was 2.

What if I want to have more server connections, should I also set the MaxConnections to proxy to follow this? No! The number of proxy connections will automatically follow the MaxConnections for the server, your application should probably do the same. In most cases (unless your proxy has some other strange requirement) it should be the same as your server connections. If for some reason you want this different then set the MaxConnectionsPerProxy manually. If you do not have a reason for this to be different then do not use this setting.

How do I set the MaxConnectionsPerProxy? Either in Code (see this blog: http://blogs.msdn.com/jpsanders/archive/2009/06/08/understanding-the-new-wininet-option-internet-option-max-conns-per-proxy.aspx) or with this registry setting:

Important This section, method, or task contains steps that tell you how to modify the registry. However, serious problems might occur if you modify the registry incorrectly. Therefore, make sure that you follow these steps carefully. For added protection, back up the registry before you modify it. Then, you can restore the registry if a problem occurs. For more information about how to back up and restore the registry, click the following article number to view the article in the Microsoft Knowledge Base:

322756 (http://support.microsoft.com/kb/322756/) How to back up and restore the registry in Windows

To change the number of files that you can download through your proxy at one time to 2, follow these steps:

1. Start Registry Editor.

2. Locate the following key in the registry:
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings

3. On the Edit menu, point to New, click DWORD Value, and then add the following registry values:
Value name: MaxConnectionsPerProxy
Value data: 2
Base: Decimal

4. Exit Registry Editor.

Let me know if this article was useful to you!

Understanding the new WinInet options: INTERNET_SUPPRESS_COOKIE_PERSIST and INTERNET_SUPPRESS_COOKIE_PERSIST_RESET

jpsanders — Mon, 08 Jun 2009 19:33:00 GMT

These options are well documented. Important notes:

These flags affect the process that you set this option from.

You do not need to pass an InternetHandle (but no error if you do).

Sample code:

bool abRes = false;
DWORD adOption = INTERNET_SUPPRESS_COOKIE_PERSIST;
abRes= InternetSetOption(hInternet, INTERNET_OPTION_SUPPRESS_BEHAVIOR ,&adOption,sizeof(adOption));
adOption = INTERNET_SUPPRESS_COOKIE_PERSIST_RESET;
abRes= InternetSetOption(hInternet, INTERNET_OPTION_SUPPRESS_BEHAVIOR ,&adOption,sizeof(adOption));

Understanding the new WinInet option: INTERNET_OPTION_MAX_CONNS_PER_PROXY

jpsanders — Mon, 08 Jun 2009 18:52:00 GMT

The documentation on this is very straight forward (http://msdn.microsoft.com/en-us/library/aa385328(VS.85).aspx). At the time of publishing however the documentation has a slight error. You cannot pass a handle in for the first argument. It must be NULL or the call will return false and GetLastError() will show the error is: ERROR_INTERNET_INVALID_OPERATION.

The default value for this option is 4 connections.

Also see this post: http://blogs.msdn.com/jpsanders/archive/2009/06/29/understanding-connection-limits-and-new-proxy-connection-limits-in-wininet-and-internet-explorer.aspx

Let me know if this article was useful to you!

Understanding the new WinInet option INTERNET_OPTION_SERVER_CERT_CHAIN_CONTEXT

jpsanders — Fri, 05 Jun 2009 23:03:00 GMT

With the release of Internet Explorer 8 comes a new option you can query for with programming with the WinInet APIs: INTERNET_OPTION_SERVER_CERT_CHAIN_CONTEXT. The MSDN documentation tells you it allows you to get the PCCERT_CHAIN_CONTEXT, and not much more. It does not show you how to properly get the Context. I like to see things in action so here is some sample code.

This option is documented here: http://msdn.microsoft.com/en-us/library/aa385328(VS.85).aspx (note that if you are not using the latest SDK headers, the value for this option is also documented here).

To sum it up, use this option to get the Server Certificate Chain Context to use with other Crypto API functions To understand this option you can modify the HttpDump example in the Platforms SDK and see how this option can be used. Since this is an option for the request you get this option using the request handle after you execute the request. I am not a Crypto API guru so I stole some code examples from MSDN just to dump out some information about the Certificate Chain.

The most important (and undocumented) part of getting this option is to pass a Pointer to a PCCERT_CHAIN_CONTEXT. Below is a code snippet that I added to the HttpDump sample in the Windows 7 SDK (RC).

First, there is a problem with that sample when compiled for unicode. I fixed it with this replacement code:

C++ code listing for sample (Copy Code):

dwSize=0;
TCHAR *myBuff = NULL;

// First time we will find out the size of the headers.
HttpQueryInfo (hReq,HTTP_QUERY_RAW_HEADERS_CRLF, NULL, &dwSize, NULL);
myBuff = new TCHAR [dwSize+1];
// Now we call HttpQueryInfo again to get the headers.
if (!HttpQueryInfo (hReq,HTTP_QUERY_RAW_HEADERS_CRLF, (LPVOID) myBuff,
&dwSize, NULL))
{
    ErrorOut (GetLastError(), TEXT("HttpQueryInfo"));
}
else
{
    *(myBuff + dwSize) = '\0';
    wcout << myBuff << endl;
}

delete myBuff;

Next I added this code just before closing the request handle ( 'if (!InternetCloseHandle (hReq) )' ).

C++ code listing for sample (Copy Code):

PCCERT_CHAIN_CONTEXT CertCtx=NULL;
DWORD cbCertSize = sizeof(&CertCtx);

//GetCertificate information
if (InternetQueryOption(hReq,INTERNET_OPTION_SERVER_CERT_CHAIN_CONTEXT ,(LPVOID)&CertCtx,&cbCertSize))
{
//---------------------------------------------------------------
// Display some of the contents of the chain.
PCCERT_CHAIN_CONTEXT pChainContext=CertCtx;
printf("The size of the chain context is %d. \n",pChainContext->cbSize);
printf("%d simple chains found.\n",pChainContext->cChain);
printf("\nError status for the chain:\n");
switch(pChainContext->TrustStatus.dwErrorStatus)
{
case CERT_TRUST_NO_ERROR :
printf("No error found for this certificate or chain.\n");
break;
case CERT_TRUST_IS_NOT_TIME_VALID:
printf("This certificate or one of the certificates in the certificate chain is not time-valid.\n");
break;
case CERT_TRUST_IS_NOT_TIME_NESTED:
printf("Certificates in the chain are not properly time-nested.\n");
break;
case CERT_TRUST_IS_REVOKED:
printf("Trust for this certificate or one of the certificates in the certificate chain has been revoked.\n");
break;
case CERT_TRUST_IS_NOT_SIGNATURE_VALID:
printf("The certificate or one of the certificates in the certificate chain does not have a valid signature.\n");
break;
case CERT_TRUST_IS_NOT_VALID_FOR_USAGE:
printf("The certificate or certificate chain is not valid in its proposed usage.\n");
break;
case CERT_TRUST_IS_UNTRUSTED_ROOT:
printf("The certificate or certificate chain is based on an untrusted root.\n");
break;
case CERT_TRUST_REVOCATION_STATUS_UNKNOWN:
printf("The revocation status of the certificate or one of the certificates in the certificate chain is unknown.\n");
break;
case CERT_TRUST_IS_CYCLIC :
printf("One of the certificates in the chain was issued by a certification authority that the original certificate had certified.\n");
break;
case CERT_TRUST_IS_PARTIAL_CHAIN:
printf("The certificate chain is not complete.\n");
break;
case CERT_TRUST_CTL_IS_NOT_TIME_VALID:
printf("A CTL used to create this chain was not time-valid.\n");
break;
case CERT_TRUST_CTL_IS_NOT_SIGNATURE_VALID:
printf("A CTL used to create this chain did not have a valid signature.\n");
break;
case CERT_TRUST_CTL_IS_NOT_VALID_FOR_USAGE:
printf("A CTL used to create this chain is not valid for this usage.\n");
} // End switch

printf("\nInfo status for the chain:\n");
switch(pChainContext->TrustStatus.dwInfoStatus)
{
case 0:
printf("No information status reported.\n");
break;
case CERT_TRUST_HAS_EXACT_MATCH_ISSUER :
printf("An exact match issuer certificate has been found for this certificate.\n");
break;
case CERT_TRUST_HAS_KEY_MATCH_ISSUER:
printf("A key match issuer certificate has been found for this certificate.\n");
break;
case CERT_TRUST_HAS_NAME_MATCH_ISSUER:
printf("A name match issuer certificate has been found for this certificate.\n");
break;
case CERT_TRUST_IS_SELF_SIGNED:
printf("This certificate is self-signed.\n");
break;
case CERT_TRUST_IS_COMPLEX_CHAIN:
printf("The certificate chain created is a complex chain.\n");
break;
case CERT_TRUST_HAS_PREFERRED_ISSUER:
printf("The certificate chain has a preferred issuer.\n");
break;
case CERT_TRUST_HAS_ISSUANCE_CHAIN_POLICY:
printf("The certificate chain has issuance chain policy.\n");
break;
case CERT_TRUST_HAS_VALID_NAME_CONSTRAINTS:
printf("The certificate chain valid name contraints.\n");
break;
case CERT_TRUST_IS_PEER_TRUSTED:
printf("The certificate chain is peer trusted.\n");
break;
case CERT_TRUST_HAS_CRL_VALIDITY_EXTENDED:
printf("The certificate chain has CRL validity extended.\n");
break;
case CERT_TRUST_IS_FROM_EXCLUSIVE_TRUST_STORE:
printf("The certificate chain was found in a store specified by hExclusiveRoot or hExclusiveTrustedPeople.\n");
break;
} // end switch

CERT_SIMPLE_CHAIN *simpleCertificateChainWithinContext = NULL;
for (int i=0; i<pChainContext->cChain; i++)
{
    simpleCertificateChainWithinContext=pChainContext->rgpChain[i];
    // for each certificate chain in this context...
    for (int simpleCertChainIndex = 0;
    simpleCertChainIndex < simpleCertificateChainWithinContext->cElement;
    simpleCertChainIndex++)
    {

        // get the CertContext from the array
        PCCERT_CONTEXT pCertContext =
        simpleCertificateChainWithinContext->rgpElement[simpleCertChainIndex]->pCertContext;

        //-------------------------------------------------------------------
        // Find and print the name of the subject of the certificate
        // just retrieved.
        TCHAR pszNameString[256];
        if(CertGetNameString(
        pCertContext,
        CERT_NAME_SIMPLE_DISPLAY_TYPE,
        0,
        NULL,
        pszNameString,
        128))
        {
            wprintf(L"Certificate for %s has been retrieved.\n", pszNameString);
        }
        else
        {
            wprintf(L"CertGetName failed. \n");
        }

        // Get the issuer now...
        if(CertGetNameString( pCertContext,
        CERT_NAME_SIMPLE_DISPLAY_TYPE,
        CERT_NAME_ISSUER_FLAG,
        NULL,
        pszNameString,
        128))
        {
            wprintf(L"Certificate issuer is %s.\n\n", pszNameString);
        }
        else
        {
            wprintf(L"CertGetName failed. \n");
        }
    }
}

//important! Free the CertCtx
CertFreeCertificateChain(CertCtx);
}
else
{
ErrorOut (GetLastError(), TEXT("HttpQueryInfo"));
}

The above code simply illustrates getting the CertContext. I will leave the actual implementation and usage of this context to all of you Crypto API types! Please drop me a message if you found this blog useful!

Understanding the New WinInet flag: INTERNET_COOKIE_HTTPONLY

jpsanders — Thu, 04 Jun 2009 18:56:00 GMT

There are a couple of new Cookie flags introduced with the Internet Explorer 8 WinInet.dll. The INTERNET_COOKIE_HTTPONLY flag allows you to read the HttpOnly cookies in your WinInet Code. This flag is documented here: http://msdn.microsoft.com/en-us/library/aa384714(VS.85).aspx. As always, I like to see examples of how this flag works!

Here is a sample ASPX page to create some standard and httponly cookies:

aspx code listing for sample (Copy Code):

<%@ Page Language="C#" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<script runat="server">
    void Page_Load(object sender, EventArgs e)
    {
        // Create a new HttpCookie.
        HttpCookie myHttpCookie = new HttpCookie("LastVisit", "sometime");
myHttpCookie.Expires = DateTime.Now.AddYears(1);

// By default, the HttpOnly property is set to false
// unless specified otherwise in configuration.

myHttpCookie.Name = "MyHttpCookie";
myHttpCookie.Path = "/";
Response.AppendCookie(myHttpCookie);

// Show the name of the cookie.
Response.Write(myHttpCookie.Name);

// Create an HttpOnly cookie.
HttpCookie myHttpOnlyCookie = new HttpCookie("LastVisit", "sometime later");

// Setting the HttpOnly value to true, makes
// this cookie accessible only to ASP.NET.

        myHttpOnlyCookie.HttpOnly = true;
myHttpCookie.Expires = DateTime.Now.AddYears(1);
        myHttpOnlyCookie.Name = "MyHttpOnlyCookie";
        Response.AppendCookie(myHttpOnlyCookie);

        // Show the name of the HttpOnly cookie.
        Response.Write(myHttpOnlyCookie.Name);
        Response.Write("jeff");
    }
</script>

<html >
<head runat="server">
    <title>ASP.NET Example</title>
</head>
<body>
<script type="text/javascript">
function getCookie(NameOfCookie)
{
    if (document.cookie.length > 0)
{
    begin = document.cookie.indexOf(NameOfCookie+"=");
    if (begin != -1)
   {
    begin += NameOfCookie.length+1;
      end = document.cookie.indexOf(";", begin);
      if (end == -1) end = document.cookie.length;
      return unescape(document.cookie.substring(begin, end));
      }
}
return null;
}
</script>

    // This code returns the cookie name.
    alert("Getting HTTP Cookie");
    alert(getCookie("MyHttpCookie"));

    // Because the cookie is set to HttpOnly,
    // this returns null.
    alert("Getting HTTP Only Cookie");
    alert(getCookie("MyHttpOnlyCookie"));

</script>

</body>
</html>

When you run this page you will note InternetExplorer jscript will not allow you to read the value of MyHttpOnlyCookie. This new flag will allow you to read that cookie from code however!

To investigate this I decided to use my favorite sample 'httpauth' from the Platform SDK.

I added this code to the end of the function, just before closing the handles (note the empty error conditions that you need to fill in):

C++ code listing for sample (Copy Code):

fprintf (stderr, "\n");
char szCookieBuf[512];
DWORD ccCookieBufSize=512;
DWORD dwErr=0;

if (!InternetGetCookieEx("http://jsanders4/","MyHttpCookie",szCookieBuf,&ccCookieBufSize,0,NULL))
{

    dwErr=GetLastError();
    switch (dwErr)
    {

case ERROR_INSUFFICIENT_BUFFER:
break;

case ERROR_NO_MORE_ITEMS:
break;

default:
break;

};

}
else
{
fprintf (stderr, "Cookie found: %s\n", szCookieBuf);
}

ccCookieBufSize=512;

if (!InternetGetCookieEx("http://jsanders4/","MyHttpOnlyCookie",szCookieBuf,&ccCookieBufSize,INTERNET_COOKIE_HTTPONLY,NULL))
{

    dwErr=GetLastError();
    switch (dwErr)
    {

case ERROR_INSUFFICIENT_BUFFER:
break;

case ERROR_NO_MORE_ITEMS:
break;

default:
break;

};

}
else
{
fprintf (stderr, "Cookie found: %s\n", szCookieBuf);
}

I put the page to write the cookies on one of my servers and pointed the httpauth.exe to that page. This code works fine and does read the HttpOnly cookie. Try and remove the flag and you will see the call fail!

Let me know if this was a help to you!

Understanding the New WinInet option: INTERNET_OPTION_SUPPRESS_SERVER_AUTH

jpsanders — Thu, 04 Jun 2009 17:45:00 GMT

With the release of Internet Explorer 8 comes a new option for WinInet programming: INTERNET_OPTION_SUPPRESS_SERVER_AUTH. The MSDN documentation is very specific and describes how the option affects authorization, but I like to see things in action! How about some sample code for INTERNET_OPTION_SUPPRESS_SERVER_AUTH?

To sum it up, use this option so your WinInet application will allow you to use credentials to authorize through a proxy, but don't pass credentials to the endpoint server.

To understand this option you can modify the HttpAuth example in the Platforms SDK and see how this option can be used to create a sample. Then you can use Fiddler (http://www.fiddlertool.com) to request proxy credentials and verify that you cannot pass credentials to an endpoint server.

Since this is an option for the request you set this on the request handle just before you execute the request:

InternetSetOption(hRequest,INTERNET_OPTION_SUPPRESS_SERVER_AUTH,NULL,0);

// Send request.
fRet = HttpSendRequest( hRequest, // request handle
"", // header string
0, // header length
NULL, // post data
0 // post length
);

Next configure Fiddler to require proxy authentication by selecting the menu item 'Rules' and check the 'Require Proxy Authentication' option. If you look at the help documentation on this feature you will discover the password and user id is '1' for this setting.

Finally run the HttpAuth sample and see that it will prompt you for the Proxy authorization, and once you enter these credentials you cannot send credentials to the end point server. Even if you use the custom UI and use InternetSetOption to set the username and password, WinInet will not send these credentials.

Note that in the documentation for this option, it suggests you use the INTERNET_OPTION_NO_COOKIES option as well to prevent Cookie based Authentication to the end point server.

Let me know if this was useful to you!

How to get Certificate Information Using WinInet APIs

jpsanders — Fri, 17 Apr 2009 17:10:00 GMT

There are two different structures you can query in order to retrieve server certificate information. You must do some sort of request to complete the SSL server certificate exchange, and then you can retrieve this information. Here is an example of that technique.

C++ code listing for sample (Copy Code):

// CertificateInfo.cpp : Defines the entry point for the console application.
// This is sample code. You are responsible for input validation, and error processing.
// The following macros define the minimum required platform. The minimum required platform
// is the earliest version of Windows, Internet Explorer etc. that has the necessary features to run
// your application. The macros work by enabling all features available on platform versions up to and
// including the version specified.
// Modify the following defines if you have to target a platform prior to the ones specified below.
// Refer to MSDN for the latest info on corresponding values for different platforms.
#ifndef _WIN32_WINNT // Specifies that the minimum required platform is Windows Vista.
#define _WIN32_WINNT 0x0600 // Change this to the appropriate value to target other versions of Windows.
#endif
#ifndef WIN32_LEAN_AND_MEAN
#define WIN32_LEAN_AND_MEAN
#endif
#include <windows.h>
#include <stdio.h>
#include <iostream>
#include <tchar.h>
#include <wininet.h> // make sure you link with WinInet.lib

int _tmain(int argc, _TCHAR* argv[])
{

if (argc < 2)
{
    std::cout << "Please specify and HTTPS address to query the certificate information for" << std::endl;
  }
else
{
    // Get Scheme and HostName from URL passed in
    ::URL_COMPONENTS urlComp;
    ::ZeroMemory((void *)&urlComp,sizeof(URL_COMPONENTS));
    urlComp.dwSchemeLength = -1;
    urlComp.dwHostNameLength = -1;
    urlComp.dwStructSize = sizeof(URL_COMPONENTS);
    if (!::InternetCrackUrl(argv[1],wcslen(argv[1]),0,&urlComp))
    {
        std::cout << "InternetCrackUrl failed" << std::endl;
    }
    else
    {
        if (urlComp.nScheme != INTERNET_SCHEME_HTTPS)
        {
            std::cout << "Please specify and HTTPS address to query the certificate information for: https://www.someserver.com" << std::endl;
        }
        else
        {
            std::wcout << "opening connection to host: " << (LPWSTR)urlComp.lpszHostName << std::endl;

            HINTERNET hInternet = ::InternetOpen(_T("Test Certificate Info"), INTERNET_OPEN_TYPE_PRECONFIG, NULL, NULL, 0);
            if (hInternet)
            {
                HINTERNET hConnect = ::InternetConnect( hInternet, urlComp.lpszHostName, INTERNET_DEFAULT_HTTPS_PORT, NULL, NULL, INTERNET_SERVICE_HTTP, INTERNET_FLAG_SECURE, NULL);
                if (hConnect)
                {
                    // you could use HEAD,GET or POST here. HEAD will not return the body of the page so will be more efficient
                    HINTERNET hRequest = ::HttpOpenRequest(hConnect,_T("HEAD"), argv[1], NULL, NULL,
NULL, INTERNET_FLAG_SECURE, NULL);

                    if (NULL != hRequest)
                    {
                        // Send
                        BOOL fRet = ::HttpSendRequest( hRequest, _T(""), 0, NULL, 0);

                        if (fRet)
                        {
                            char certificateInfoStr[2048];
                            certificateInfoStr[0] = '\0';
                            DWORD certInfoLength = 2048;
                            if ( TRUE == ::InternetQueryOption( hRequest, INTERNET_OPTION_SECURITY_CERTIFICATE, &certificateInfoStr, &certInfoLength) )
                            {
                                if ( certificateInfoStr )
                                {
                                    std::cout << (char *) certificateInfoStr << std::endl;
                                }
                            }
                            else
                            {
                                // process error here
                                DWORD error = GetLastError();
                            }

                            INTERNET_CERTIFICATE_INFO certificateInfo;
                            certInfoLength = sizeof(INTERNET_CERTIFICATE_INFO);
                            if ( TRUE == InternetQueryOption( hRequest,
                                                                              INTERNET_OPTION_SECURITY_CERTIFICATE_STRUCT,
                                                                              &certificateInfo,
                                                                              &certInfoLength) )
                            {
                                // free up memory with LocalFree()

                                if ( certificateInfo.lpszEncryptionAlgName )
                                {
                                    std::cout << (char *) certificateInfo.lpszEncryptionAlgName << std::endl;
                                    LocalFree( certificateInfo.lpszEncryptionAlgName);
                                }

                                if ( certificateInfo.lpszIssuerInfo )
                                {
                                    std::cout << (char *) certificateInfo.lpszIssuerInfo << std::endl;
                                    LocalFree( certificateInfo.lpszIssuerInfo );
                                }

                                if ( certificateInfo.lpszProtocolName )
                                {
                                    std::cout << (char *) certificateInfo.lpszProtocolName << std::endl;
                                    LocalFree( certificateInfo.lpszProtocolName );
                                }

                                if ( certificateInfo.lpszSignatureAlgName )
                                {
                                    std::cout << (char *) certificateInfo.lpszSignatureAlgName << std::endl;
                                    LocalFree( certificateInfo.lpszSignatureAlgName );
                                }

                                if ( certificateInfo.lpszSubjectInfo )
                                {
                                    std::cout << (char *) certificateInfo.lpszSubjectInfo << std::endl;
                                    LocalFree( certificateInfo.lpszSubjectInfo );
                                }

                            }
                            else
                            {
                                    // process error
                                    DWORD error = GetLastError();
                            }
                        }

                        ::InternetCloseHandle(hRequest);
                    }
                    else
                    {
                        std::cout << "HttpOpenRequest failed" << std::endl;
                    }

::InternetCloseHandle(hConnect);

                }
                else
                {
                    std::cout << "InternetConnect failed" << std::endl;
                }
                ::InternetCloseHandle(hInternet);
            }
            else
            {
                 std::cout << "InternetOpen failed" << std::endl;
            }
        }
    }
  }

return 0;
}

Please Drop me a comment if you found this useful!

Workaround For WinInet DNS not refreshing when using KB 263558

jpsanders — Fri, 14 Nov 2008 17:45:00 GMT

If you are making Synchronous calls in WinInet, the DNS timeouts don't appear to work when set (see: http://support.microsoft.com/kb/263558 ).

Async calls do however work correctly.

One workaround is to simply restart the application using WinInet.

I found a code workaround for the issue as well. Simply open and close the handle you get with the InternetOpen call then open it again:

hOpen= ::InternetOpen(.....);

InternetCloseHandle(hOpen);

hOpen= ::InternetOpen(.....);

// now use handle...

The performance impact will be very low.

-Jeff

Troubleshooting code that uses the HTTP protocol

jpsanders — Wed, 25 Jun 2008 21:02:00 GMT

Overview

You can solve issues encountered when using the HTTP protocol by applying the principles in this article. By simplifying the issue into a small reproducible problem you can effectively troubleshoot and resolve most problems encountered in code that uses the HTTP protocol. Crucial steps to troubleshooting code are: Gather Information, Simplify the Problem and Detailed Troubleshooting.

Technologies Covered

WinHttp, WinInet, Internet Explorer, XMLHTTP (MSXML), ServerXMLHTTP, System.Net classes such as WebClient, HttpWebRequest, SOAP classes, WebServices, System.Xml classes such as XmlUrlResolver, SoapHttpClientProtocol.

Error Messages Encountered (so search will find this document)

The operation timed out, 80072EE2, -2147954402, ERROR_WINHTTP_TIMEOUT, ERROR_WINHTTP_INVALID_SERVER_RESPONSE, The server name or address could not be resolved, 0x80072EE7, -2147012889, ERROR_WINHTTP_NAME_NOT_RESOLVED, 0x80072EFD

-2147012867, A connection with the server could not be established, -2146697208 (800c0008), The download of the specified resource has failed, Cannot connect to Remote server, Access is Denied, System error: -2146697191, 401, 500, 407, Server refused connection, Http protocol violation, Unauthorized, Cannot find server.

Step 1. Gather Information

You may solve the issue or get close to the root cause quickly by gathering information about the problem. Then you can continue to the Simplify the Problem section of this document to further isolate the issue and determine where the problem is occurring. Do this only after Gathering the information listed below.

1. What HTTP Verb are you using (EX: GET, POST, PUT)?

2. What is the destination URL?

3. Did this ever work? What changed if it did work previously (moved to production server, applied Service pack or Anti Virus update)?

4. What technology are you using to reach the URL that fails?

5. Can you reach the destination URL with another Technology such as Internet Explorer, other Browser, scripted Http Components, a simple console application?

6. Can you reach ANY URL from the computer (Perhaps DNS is not set, ports blocked, or you need a proxy setting)?

7. What is the failure (exact error message)?

8. What is the call stack of the failure?

9. Is this a Desktop or service based application (ASP and ASP.NET is a service)?

10. Are you running with the latest Service Packs and fixes etc…(Go to Windows Update)?

Step 2. Simplify the Problem

Armed with the answers from Step 1, reproduce the problem with the simplest code possible. In the process of simplifying the issue you may discover the issue is not in the technology you are using but a problem being obscured by your program’s logic or system configuration. For example, In one case I assisted in, numerous troubleshooting actions had been taken in the ASP pages to determine what was going wrong. When I was brought in to assist, I simply asked, “Can you hit the target URL in a Browser”? The answer (“no”) instantly revealed to all of us that the issue was the DNS server setting on that machine and not the code.

How to simplify the issue:

1. Duplicate the issue with a technology that is easier to troubleshoot such as the Web Browser or System.Net 2.0 which has the ability to log the HTTPS traffic.

2. If the problem occurs with HTTPS, see if you can reproduce with HTTP as well then troubleshoot HTTP traffic instead of HTTPS traffic (which is securely encoded). If you cannot duplicate the issue with HTTP then the problem may lie with your HTTPS configuration.

3. If this is a service based application, such as ASP or ASP.NET, move the code out of the Service and test. If it works, then the issue is not with the technology you are using. The issue is how you have your service configured or the context it is running in.

4. Try simple code from script or a command line application to see if that succeeds. Then use that same simple code in your application and test. If it fails in either case you have a much smaller problem to deal with.

5. Simplify the code by executing the simplest possible HTTP call in a command line script or console application. Start with using a HEAD or GET verb to a resource (perhaps an image or static file on the server) and add complexity until you get the failure.

Below are some code samples you can use to simplify the problem. Simplify! For example if your application is doing a POST verb, start with a GET verb to start your troubleshooting and then only if you cannot duplicate the issue using the GET verb, move to the POST verb.

VBScript samples:

Dim aRequest

Set aRequest = CreateObject ("Microsoft.XMLHTTP")

aRequest.Open "GET","http://www.microsoft.com/",False

aRequest.Send