Welcome to MSDN Blogs Sign in | Join | Help

Impersonation code in C#

While investigating a customer issue, I needed code to impersonate a different user on the current thread.  A quick search of the msdn blogs turned up this post, which got me most of the way there.  I decided to wrap the code into a class that inherits from IDisposable just to make it easily usable in a using statement.  Here is what I came up with (note that it is not thread safe):

using System;

using System.ComponentModel;

using System.Runtime.InteropServices;

using System.Security.Principal;

 

public class ImpersonatedUser : IDisposable

{

    IntPtr userHandle;

    WindowsImpersonationContext impersonationContext;

 

    public ImpersonatedUser(string user, string domain, string password)

    {

        userHandle = IntPtr.Zero;

        bool loggedOn = LogonUser(

            user,

            domain,

            password,

            LogonType.Interactive,

            LogonProvider.Default,

            out userHandle);

 

        if (!loggedOn)

            throw new Win32Exception(Marshal.GetLastWin32Error());

 

        // Begin impersonating the user

        impersonationContext = WindowsIdentity.Impersonate(userHandle);

    }

 

    public void Dispose()

    {

        if (userHandle != IntPtr.Zero)

        {

            CloseHandle(userHandle);

            userHandle = IntPtr.Zero;

            impersonationContext.Undo();

        }

    } 

 

    [DllImport("advapi32.dll", SetLastError = true)]

    static extern bool LogonUser(

        string lpszUsername,

        string lpszDomain,

        string lpszPassword,

        LogonType dwLogonType,

        LogonProvider dwLogonProvider,

        out IntPtr phToken

        );

 

    [DllImport("kernel32.dll", SetLastError = true)]

    static extern bool CloseHandle(IntPtr hHandle);

 

    enum LogonType : int

    {

        Interactive = 2,

        Network = 3,

        Batch = 4,

        Service = 5,

        NetworkCleartext = 8,

        NewCredentials = 9,

    }

 

    enum LogonProvider : int

    {

        Default = 0,

    }

}

Posted by joncole | 2 Comments
Filed under:

Sample Asynchronous SslStream Client/Server Implementation

I was recently asked about sample code for the System.Net.Security.SslStream using the asynchronous APIs.  I searched the web and couldn't find anything significant, so I decided to write some that included the asynchronous usage of the TcpListenter and TcpClient objects (both in the System.Net.Sockets namespace).  In this post, I will show both the client and server side of the picture.

The person that was requesting this sample code seemed pretty anxious to get it, so I pumped this out pretty quickly, and as such, there are probably some bugs in it – please let me know of any you find and I will fix them.  My goal here is to introduce the concepts and is not to provide production ready code.  There are some things that I plan on changing about this code, but this should be a good starting place for anyone unfamiliar with the usage.  Watch for “To Do Items” sections where I try to call out some of the things that I would like to add at some point, but you may need to add to your own code in the mean time…

To start, here is some helper code that is shared between client and server code. 

Common Code:

The delegate function definition is used to notify you that a secure connection attempt is either ready for use or that an asynchronous error occurred.  The SecureConnectionResults class is merely a holder for the results.  You will need to provide an implementation of this delegate function.

When you delegate function is called, you will need to check for any asynchronous exception that may have occurred by looking at the SecureConnectionResults.AsyncException property.  If it is null, then the SslStream object should be ready for use.   Also, you will want to note that the SecureConnectionResultsCallback will be invoked on a Threadpool thread, so you won’t want to hold onto that thread permanently as it is a limited resource.

To Do Items:

·         May also want to expose the underlying socket through the SecureConnectionResults object.  This would allow you to change socket options or switch back to an unencrypted network connection if you so desire.

 

using System;

using System.Net.Security;

 

public delegate void SecureConnectionResultsCallback(object sender, SecureConnectionResults args);

 

public class SecureConnectionResults

{

    private SslStream secureStream;

    private Exception asyncException;

 

    internal SecureConnectionResults(SslStream sslStream)

    {

        this.secureStream = sslStream;

    }

 

    internal SecureConnectionResults(Exception exception)

    {

        this.asyncException = exception;

    }

 

    public Exception AsyncException{get{return asyncException;}}   

    public SslStream SecureStream{get{return secureStream;}}   

}

 

Server Code:

Here is my implementation of a TcpListener that accepts a connection and then initiates the SSL handshake immediately (by calling SslStream.BeginAuthenticateAsServer from within the OnAcceptConnection callback).  This code automatically handles both IPv6 and IPv4 for you – make sure to read the “To Do Items”.  If you aren't interested in reading all of the code, the bulk of the work is done inside the StartConnecting, OnAcceptConnection and OnAuthenticateAsServer functions.

To Do Items:

·         I haven’t added any substantial error handling to this code. 

o   For example, you might be able to bind the IPv4 socket to the port but an error could occur when trying to bind the IPv6 socket to that same port (i.e. - it is taken by another application on IPv6 but not on IPv4). 

·         When on Windows Vista or later, use a single socket to listen for both IPv4 and IPv6 – this avoids the binding problem previously discussed.

·         Another example would be that a fatal socket error could happen in OnAcceptConnection or OnAuthenticateAsServer that should result in the entire object being invalidated.  I will leave it up to you to add these details for now, but will try to update the code when I have time.

·         Thread safety.  I am not doing any locking inside the object and so you could see funny results.

o   For example, you shutdown the object while a callback is in progress.

using System;

using System.Net;

using System.Net.Sockets;

using System.Net.Security;

using System.Security.Cryptography.X509Certificates;

using System.Security.Authentication;

 

public class SecureTcpServer : IDisposable

{ 

    X509Certificate serverCert;

    RemoteCertificateValidationCallback certValidationCallback;

    SecureConnectionResultsCallback connectionCallback;

    AsyncCallback onAcceptConnection;

    AsyncCallback onAuthenticateAsServer;

 

    bool started;

 

    int listenPort;

    TcpListener listenerV4;

    TcpListener listenerV6;

    int disposed;

    bool clientCertificateRequired;

    bool checkCertifcateRevocation;

    SslProtocols sslProtocols;

 

    public SecureTcpServer(int listenPort, X509Certificate serverCertificate,

        SecureConnectionResultsCallback callback)

        :this(listenPort,serverCertificate,callback,null)

    {

    }

 

    public SecureTcpServer(int listenPort, X509Certificate serverCertificate,

        SecureConnectionResultsCallback callback,

        RemoteCertificateValidationCallback certValidationCallback)

    {

        if (listenPort < 0 || listenPort > UInt16.MaxValue)

            throw new ArgumentOutOfRangeException("listenPort");

 

        if (serverCertificate == null)

            throw new ArgumentNullException("serverCertificate");

 

        if (callback == null)

            throw new ArgumentNullException("callback");

 

        onAcceptConnection = new AsyncCallback(OnAcceptConnection);

        onAuthenticateAsServer = new AsyncCallback(OnAuthenticateAsServer);

 

        this.serverCert = serverCertificate;

        this.certValidationCallback = certValidationCallback;

        this.connectionCallback = callback;

        this.listenPort = listenPort;

        this.disposed = 0;

        this.checkCertifcateRevocation = false;

        this.clientCertificateRequired = false;

        this.sslProtocols = SslProtocols.Default;

    }

 

    ~SecureTcpServer()

    {

        Dispose();

    }

 

    public SslProtocols SslProtocols

    {

        get { return sslProtocols; }

        set { sslProtocols = value; }

    }

 

    public bool CheckCertifcateRevocation

    {

        get { return checkCertifcateRevocation; }

        set { checkCertifcateRevocation = value; }

    }

 

 

    public bool ClientCertificateRequired

    {

        get { return clientCertificateRequired; }

        set { clientCertificateRequired = value; }

    }

 

    public void StartListening()

    {

        if(started)

            throw new InvalidOperationException("Already started...");

 

        IPEndPoint localIP;

        if (Socket.SupportsIPv4 && listenerV4 == null)

        {

            localIP = new IPEndPoint(IPAddress.Any, listenPort);

            Console.WriteLine("SecureTcpServer: Started listening on {0}", localIP);

            listenerV4 = new TcpListener(localIP);

        }

 

        if(Socket.OSSupportsIPv6 && listenerV6 == null)

        {

            localIP = new IPEndPoint(IPAddress.IPv6Any, listenPort);

            Console.WriteLine("SecureTcpServer: Started listening on {0}", localIP);

            listenerV6 = new TcpListener(localIP);

        }

 

        if(listenerV4 != null)

        {

            listenerV4.Start();

            listenerV4.BeginAcceptTcpClient(onAcceptConnection,listenerV4);

        }

 

        if(listenerV6 != null)

        {

            listenerV6.Start();

            listenerV6.BeginAcceptTcpClient(onAcceptConnection,listenerV6);

        }

 

        started = true;

    }

 

    public void StopListening()

    {

        if (!started)

            return;

 

        started = false;

 

        if (listenerV4 != null)

            listenerV4.Stop();

        if (listenerV6 != null)

            listenerV6.Stop();

    }

 

    void OnAcceptConnection(IAsyncResult result)

    {

        TcpListener listener = result.AsyncState as TcpListener;

        TcpClient client = null;

        SslStream sslStream = null;

 

        try

        {

            if (this.started)

            {

                //start accepting the next connection...

                listener.BeginAcceptTcpClient(this.onAcceptConnection, listener);

            }

            else

            {

                //someone called Stop() - don't call EndAcceptTcpClient because

                //it will throw an ObjectDisposedException

                return;

            }

 

            //complete the last operation...

            client = listener.EndAcceptTcpClient(result);

 

 

            bool leaveStreamOpen = false;//close the socket when done

 

            if (this.certValidationCallback != null)

                sslStream = new SslStream(client.GetStream(), leaveStreamOpen, this.certValidationCallback);

            else

                sslStream = new SslStream(client.GetStream(), leaveStreamOpen);

 

            sslStream.BeginAuthenticateAsServer(this.serverCert,

                this.clientCertificateRequired,

                this.sslProtocols,

                this.checkCertifcateRevocation,//checkCertifcateRevocation

                this.onAuthenticateAsServer,

                sslStream);

 

 

        }

        catch (Exception ex)

        {

            if (sslStream != null)

            {

                sslStream.Dispose();

                sslStream = null;

            }

            this.connectionCallback(this, new SecureConnectionResults(ex));

        }

    }

 

    void OnAuthenticateAsServer(IAsyncResult result)

    {

        SslStream sslStream = null;

        try

        {

            sslStream = result.AsyncState as SslStream;

            sslStream.EndAuthenticateAsServer(result);

 

            this.connectionCallback(this, new SecureConnectionResults(sslStream));

        }

        catch (Exception ex)

        {

            if (sslStream != null)

            {

                sslStream.Dispose();

                sslStream = null;

            }

 

            this.connectionCallback(this, new SecureConnectionResults(ex));

        }

    }

 

    public void Dispose()

    {

        if (System.Threading.Interlocked.Increment(ref disposed) == 1)

        {

            if (this.listenerV4 != null)

                listenerV4.Stop();

            if (this.listenerV6 != null)

                listenerV6.Stop();

 

            listenerV4 = null;

            listenerV6 = null;

 

            GC.SuppressFinalize(this);

        }

    }

}

 

Client Code:

The client code is similar to the server code in that it tries to abstract you from handling the connecting and SSL negotiation.  The remoteHostName parameter is used when validating the server’s certificate.  If the name you pass in doesn’t match the name on the server’s certificate, you will get certificate errors (i.e. System.Net.Security.SslPolicyErrors.RemoteCertificateNameMismatch).  Again, if you really only want to look at the most important parts of the code, focus on the StartConnecting, OnConnected and OnAuthenticateAsClient functions.

To Do Items:

·         Change StartConnecting so that it doesn’t require a single IPEndpoint.  It would be nice if the code just did an asynchronous DNS lookup for you based on the host name and then automatically connected to any of the resolved addresses (regardless of IPv4 or IPv6)

·         I haven’t added any substantial error handling to this code. 

o   An example would be that a fatal error could happen that should result in the entire object being invalidated.  I will leave it up to you to add these details for now, but will try to update the code when I have time.

·         Thread safety.  I am not doing any locking inside the object and so you could see funny results.

using System;

using System.Net;

using System.Net.Sockets;

using System.Net.Security;

using System.Security.Cryptography.X509Certificates;

using System.Security.Authentication;

 

public class SecureTcpClient : IDisposable

{

    X509CertificateCollection clientCertificates;

    RemoteCertificateValidationCallback certValidationCallback;

    SecureConnectionResultsCallback connectionCallback;

    bool checkCertificateRevocation;

 

    AsyncCallback onConnected;

    AsyncCallback onAuthenticateAsClient;

    TcpClient client;       

    IPEndPoint remoteEndPoint;

    string remoteHostName;

    SslProtocols protocols;

    int disposed;

 

    public SecureTcpClient(SecureConnectionResultsCallback callback)

        : this(callback,null,SslProtocols.Default)

    {

    }

    public SecureTcpClient(SecureConnectionResultsCallback callback,

        RemoteCertificateValidationCallback certValidationCallback)

        : this(callback, certValidationCallback, SslProtocols.Default)

    {

    }

 

    public SecureTcpClient(SecureConnectionResultsCallback callback,

        RemoteCertificateValidationCallback certValidationCallback,

        SslProtocols sslProtocols)

    {

        if (callback == null)

            throw new ArgumentNullException("callback");

 

        onConnected = new AsyncCallback(OnConnected);

        onAuthenticateAsClient = new AsyncCallback(OnAuthenticateAsClient);

 

        this.certValidationCallback = certValidationCallback;

        this.connectionCallback = callback;

        protocols = sslProtocols;

        this.disposed = 0;

    }

 

    ~SecureTcpClient()

    {

        Dispose();

    }

 

    public bool CheckCertificateRevocation

    {

        get { return checkCertificateRevocation; }

        set {checkCertificateRevocation = value;}

    }

 

    public void StartConnecting(string remoteHostName, IPEndPoint remoteEndPoint)

    {

        StartConnecting(remoteHostName,remoteEndPoint,null);

    }

 

    public void StartConnecting(string remoteHostName, IPEndPoint remoteEndPoint,

        X509CertificateCollection clientCertificates)

    {

        if (string.IsNullOrEmpty(remoteHostName))

            throw new ArgumentException("Value cannot be null or empty","remoteHostName");

 

        if (remoteEndPoint == null)

            throw new ArgumentNullException("remoteEndPoint");

 

        Console.WriteLine("Client connecting to: {0}", remoteEndPoint);

       

        this.clientCertificates = clientCertificates;

        this.remoteHostName = remoteHostName;

        this.remoteEndPoint = remoteEndPoint;

 

        if (client != null)

            client.Close();

       

        client = new TcpClient(remoteEndPoint.AddressFamily);

       

        client.BeginConnect(remoteEndPoint.Address,

            remoteEndPoint.Port,

            this.onConnected,null);       

    }

 

    public void Close()

    {

        Dispose();

    }

 

    void OnConnected(IAsyncResult result)

    {

        SslStream sslStream = null;

 

        try

        {

            bool leaveStreamOpen = false;//close the socket when done

 

            if (this.certValidationCallback != null)

                sslStream = new SslStream(client.GetStream(), leaveStreamOpen, this.certValidationCallback);

            else

                sslStream = new SslStream(client.GetStream(), leaveStreamOpen);

 

                         

            sslStream.BeginAuthenticateAsClient(this.remoteHostName,

                    this.clientCertificates,

                    this.protocols,

                    this.checkCertificateRevocation,

                    this.onAuthenticateAsClient,

                    sslStream);

        }

        catch (Exception ex)

        {

            if (sslStream != null)

            {

                sslStream.Dispose();

                sslStream = null;

            }

 

            this.connectionCallback(this,new SecureConnectionResults(ex));

        }

    }

 

    void OnAuthenticateAsClient(IAsyncResult result)

    {

        SslStream sslStream = null;

        try

        {

            sslStream = result.AsyncState as SslStream;

            sslStream.EndAuthenticateAsClient(result);

 

            this.connectionCallback(this,new SecureConnectionResults( sslStream));

        }

        catch (Exception ex)

        {

            if (sslStream != null)

            {

                sslStream.Dispose();

                sslStream = null;

            }

 

            this.connectionCallback(this, new SecureConnectionResults(ex));

        }

    }

 

    public void Dispose()

    {

        if (System.Threading.Interlocked.Increment(ref disposed) == 1)

        {

            if (client != null)

            {

                client.Close();

                client = null;

            }

            GC.SuppressFinalize(this);

        }

    }

}

 

 

Sample Program:

 

Here is a very simple program that demonstrates the use of these objects.

 

using System;

using System.IO;

using System.Net;

using System.Threading;

using System.Net.Sockets;

using System.Security.Cryptography;

using System.Security.Cryptography.X509Certificates;

using System.Net.Security;

 

 

class Program

{

    static void Main(string[] args)

    {

        SecureTcpServer server = null;

        SecureTcpClient client = null;

 

        try

        {

            int port = 8889;

 

            RemoteCertificateValidationCallback certValidationCallback = null;

            certValidationCallback = new RemoteCertificateValidationCallback(IgnoreCertificateErrorsCallback);

 

            string certPath = System.Reflection.Assembly.GetEntryAssembly().Location;

            certPath = Path.GetDirectoryName(certPath);

            certPath = Path.Combine(certPath, "serverCert.cer");

            Console.WriteLine("Loading Server Cert From: " + certPath);

            X509Certificate serverCert = X509Certificate.CreateFromCertFile(certPath);

 

            server = new SecureTcpServer(port, serverCert,

                new SecureConnectionResultsCallback(OnServerConnectionAvailable));

           

            server.StartListening();

 

            client = new SecureTcpClient(new SecureConnectionResultsCallback(OnClientConnectionAvailable),

                certValidationCallback);

 

            client.StartConnecting("localhost", new IPEndPoint(IPAddress.Loopback, port));

        }

        catch (Exception ex)

        {

            Console.WriteLine(ex);               

        }

 

        //sleep to avoid printing this text until after the callbacks have been invoked.

        Thread.Sleep(4000);

        Console.WriteLine("Press any key to continue...");

        Console.ReadKey();

 

        if (server != null)

            server.Dispose();

        if (client != null)

            client.Dispose();

       

    }

 

    static void OnServerConnectionAvailable(object sender, SecureConnectionResults args)

    {

        if (args.AsyncException != null)

        {

            Console.WriteLine(args.AsyncException);

            return;

        }

 

        SslStream stream = args.SecureStream;

 

        Console.WriteLine("Server Connection secured: " + stream.IsAuthenticated);

       

       

       

        StreamWriter writer = new StreamWriter(stream);

        writer.AutoFlush = true;

 

        writer.WriteLine("Hello from server!");

 

        StreamReader reader = new StreamReader(stream);

        string line = reader.ReadLine();

        Console.WriteLine("Server Recieved: '{0}'", line == null ? "<NULL>" : line);

 

        writer.Close();

        reader.Close();

        stream.Close();

    }

 

    static void OnClientConnectionAvailable(object sender, SecureConnectionResults args)

    {

        if (args.AsyncException != null)

        {

            Console.WriteLine(args.AsyncException);

            return;

        }

        SslStream stream = args.SecureStream;

 

        Console.WriteLine("Client Connection secured: " + stream.IsAuthenticated);

 

        StreamWriter writer = new StreamWriter(stream);

        writer.AutoFlush = true;

 

        writer.WriteLine("Hello from client!");

       

        StreamReader reader = new StreamReader(stream);

        string line = reader.ReadLine();

        Console.WriteLine("Client Recieved: '{0}'", line == null ? "<NULL>" : line);

 

        writer.Close();

        reader.Close();

        stream.Close();

    }

 

    static bool IgnoreCertificateErrorsCallback(object sender,

        X509Certificate certificate,

        X509Chain chain,

        SslPolicyErrors sslPolicyErrors)

    {

        if (sslPolicyErrors != SslPolicyErrors.None)

        {

 

            Console.WriteLine("IgnoreCertificateErrorsCallback: {0}", sslPolicyErrors);

            //you should implement different logic here...

 

            if ((sslPolicyErrors & SslPolicyErrors.RemoteCertificateChainErrors) != 0)

            {

                foreach (X509ChainStatus chainStatus in chain.ChainStatus)

                {

                    Console.WriteLine("\t" + chainStatus.Status);

                }

            }

        }

 

        //returning true tells the SslStream object you don't care about any errors.

        return true;

    }

}

 

Note that if you don’t have a certificate for testing this code, you can create a certificate using the MakeCert.exe tool, which ships with the Visual Studio SDK. 

Creating a Process Memory Dump

 

When trying to help customers debug some issues it is periodically necessary to get a memory dump (snapshot) of a process for analysis at another time or place.  This is frequently the case when an error is intermittent or a live debug session is not feasible (like in a production environment).  I will discuss the steps necessary to install the right debuggers, attach to the process in question, looking for the right event to occur (knowing when to save the dump) and saving a dump to disk.  In this case I will be looking at BtsNtSvc.exe, which is the BizTalk NT Service process.

Installing the Debuggers

Many debuggers have support for creating (or opening) memory dumps, including Visual Studio 2005.  However, because of the size of Visual Studio and license requirements, I will discuss the use of the freely downloadable debugging tools that Microsoft provides - Debugging Tools for Windows.  Make sure you install the correct version for you processor architecture (x86 vs x64).

You may want to read the Getting Started page for supplemental information.  These tools come with several debuggers, most of which support the same types of operations.  Windbg.exe is the GUI version, but you can also use cdb.exe (a command line based debugger) and accomplish pretty much the same things because they are built on top of the same underlying debugging libraries.  In this article I will focus on WinDbg. 

Setting your Symbol Path

The symbol path (path to pdb files) can be set in several ways.  You can set it before you start the debugger by using the _NT_SYMBOL_PATH environment variable or at runtime.  To set the symbol path at runtime, select File->Symbol File Path.  This symbol path is basically just a semicolon (;) delimited list of search paths that the debugger should use when trying to locate debug symbols for the exe/dlls in your process.  I will try to write more about setting up your symbol path at a later date. 

You can also set the symbol path by using the .sympath command (more information below on how to run commands).  See the debugger help files for more information.

Check out KB article 311503 for more information on setting symbols, including how to get publicly available symbols for microsoft products.  It aslo includes a video tuturial on how to set the symbol path in the Visual Studio.Net debugger.

Attaching to the process

You can attach to the process in a couple of ways.  Choose one of the following:

  1.  From the command line you can type any of the following:
    1. "Windbg.exe -p <processID>"  (example: "windbg.exe -p 1234")
    2. "windbg.exe -pn <processName>"  (example: "windbg.exe -pn btsntsvc.exe")
    3. Note that you can add "-g" (lower case g) to any of the above to avoid breaking into the debugger immediately after attaching.
  2. After starting WinDbg:
    1. Select File->Attach to a Process from the menu
    2. Select the Process from the list in the new windows that comes up  

Once you have the debugger open, you can either type ".hh" in the command box or by selecting Help-> Contents from the menu.  Note that you will have to break into the debugger before you can run any commands in the command box.  Hit "Ctrl-Break" on your keyboard or click the pause button on the toolbar to break into the debugger.

FYI, here is a snapshot of the command box (found in the lower left hand corner of the window).  I have circled it in red.   Note that the  "0 : 003" indicates that I am currently examining thread number 3. 

 

Looking for the right event

There may be several types of events you are interested in and several ways to locate the event you are interested in.  I will only speak of three types of events in this article.  Some will result in the debugger breaking for you automatically, others you will have to tell the debugger you are interested in stopping if it occurs. I will primarily focus on .NET exceptions in this article.  

  1. Access Violation (a.k.a. AV).  This type of error will result in the process breaking into the debugger without any special setup on your part.  AVs occur when your program tries to access memory that it doesn't have permission to access.  Examples would be reading/writing protected memory or trying to use an object that is null.  In .NET, these will result in either an AccessViolationException (read/write protected memory) or NullReferenceException (null object referenced in code). 
  2. .NET exception at the time it is thrown (a.k.a first chance exception).  Windbg will not break into the debugger for this one automatically.  You have to tell it that you want it to stop.  You can do this by running the command sxe clr.  If you need to stop breaking into the debugger every time a .NET exception is thrown, you can use the sxi clr command.  Keep in mind that just because an exception is thrown does not mean that it is unexpected and unhandled.  The process may expect to get exceptions in certain recoverable situations and the user may be completely unaware that anything has happened.
  3. .NET exception is thrown and is not handled.  This will result in your process crashing.  This is especially true in .NET 2.0 and later.  In earlier versions of the framework, an exception being thrown and unhandled on a thread pool thread did not necessarily cause the application to shut down - it was swallowed.  This made debugging applications more difficult, so in V2.0 of the framework an unhandled exception causes the application to shutdown immediately to make the system easier to debug.

Now that we know how to get the system to break into the debugger, we need to be able to look at the exception that occurred to determine if it is the one we care about.  I will discuss how to do this in V2.0 of the framework.  It can be done in earlier versions of the framework also, but it isn't quite as easy.

Because Windbg is primarily a native application debugging tool, you have to load what is called a debugger extension to enable you to debug managed code.  This extension is called sos.dll and it sits in the .net v2.0 installation dir (same dir as mscorwks.dll).  You can use the following command to easily load it

   .loadby sos mscorwks

This tells windbg to look in the same directory as mscorwks.dll (always loaded into .NET applications) to find sos.dll.  Command that are loaded from a debugger extension dll are always prefixed with an exclamation mark (!).

When you have sos.dll loaded you can run !help.  Note that if you have multiple extensions loaded, you may have to prefix your commands with the specific extension that has the command you want.  For example: !sos.help

Now, when the debugger breaks, we can do run two commands to help us figure out where we are and what type of exception occured. !sos.printexception (or !pe is the shorthand version) and !clrstack.  If the exception type, message and stack trace match the one you see in logs, event viewer, etc., then you have (hopefully) found the right exception. 

Here is some sample output when a first chance .NET exception occurs:

(650.1c9c): CLR exception - code e0434f4d (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=0012f298 ebx=00181a60 ecx=00000000 edx=00000025 esi=0012f324 edi=e0434f4d
eip=77e55e02 esp=0012f294 ebp=0012f2e8 iopl=0 nv up ei pl nz na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202
KERNEL32!RaiseException+0x53:
77e55e02 5e pop esi

 

0:000> !pe
Exception object: 01355858
Exception type: System.InvalidOperationException
Message: This is a sample exception message for an InvalidOperationException
InnerException: <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 80131509

 

0:000> !clrstack
OS Thread Id: 0x1c9c (0)
ESP EIP
0012f370 77e55e02 [HelperMethodFrame: 0012f370]
0012f414 00d8021e Test.AnotherFunctionThatThrowsAnException(System.String)
0012f420 00d801c2 Test.SomeFunction()
0012f428 00d800b2 Test.Main()
0012f69c 79e88f63 [GCFrame: 0012f69c]

 

If the exception you are currently looking at is not the one you were looking for, then you can tell the debugger to let the process continue until the next event by using the g command (go).  Of course, these debuggers support break points also, but that is a discussion for another day.

Saving the Dump File

Now that you have the process broken into the debugger at the right place, you can use .dump command to write the memory dump out to a file. 

.dump <options> <path to file>

example: .dump /mfh c:\temp\myDump.dmp

 

I would recommend the /mfh options for this command as I show above.  You can read the help files for the .dump command to get more information on the options you can use and what they imply.

Just a reminder...

One last warning about memory dumps before we are done.  Memory dumps contain the memory snapshot of the process.  This memory can contain private information that you and/or your customers wouldn't want to give out to just anyone.  Some examples:  usernames and passwords, credit card numbers, etc - anything that is in memory at the time the dump is taken.  Consider yourself warned - don't give a copy of the memory dump to just anyone.

Posted by joncole | 2 Comments
Filed under:

Be consistent in your configuration

I recently worked with a customer using BizTalk 2004.  They were experiencing some very odd behavior where a WebService client would call into a BizTalk WebService and experience a long wait before the client finally got a timeout error.  The timeout happend to correspond to the 120 second timeout configured in IIS. 

Being fairly new to the Soap Receive Adapter code, I spent a couple hours looking over the code and some trace logs trying to understand what was happening.  I finally noticed one very simple (yet easy to miss) mistake that the customer was making.  They weren't being consistent in the configuration of the WebService.  The Receive Port in BizTalk was configured as a Request-Response (two-way) port, as was the Web Method defined in the WSDL.  However, within the WebMethod implementation, when they were calling the base.Invoke(...) function on the ServiceProxy base class they were passing true for the oneway parameter.  This inconsistency broke the contract between the WebService Implementation and the code that interacted with the Message Box, thus leading to the inconsistent behavior.

Posted by joncole | 0 Comments
Filed under: ,

Suggested Practices for proxy configuration when using System.Net.HttpWebRequest

I have periodically received questions on how to configure code using HttpWebRequest in an application that may be run in multiple types of environments (home user, corporate domain, with or with out a proxy, etc).  This is what I would suggest.

  1. Don't set a hard coded proxy unless you know the network configuration that you application will run on under all circumstances. In cases where you know the configuration settings for the network, it is better to use config than a hard coded proxy because proxy names could change and config will allow you to handle this case.
  2. Rely on Internet explorer (IE) settings for proxy settings.  Most of the time, the user will have IE configured correctly.  The caveat to this is that IE may decide to automatically send default credentials to the proxy where HttpWebRequest will not without you enabling it to do so. Also keep in mind that if your code is running in a system service, then it may not be able to get any settings from IE because the account the service is running under may not have a profile loaded with any proxy settings.
  3. In V2.0 of the framework, we now support Automatic proxy detection (this is the same as the "Automatically detect settings" check box in IE.
  4. Don't set credentials on the proxy until you receive a 407 for the first time.  At that point, you can try sending default credentials.  If default credentials don't work for the proxy, then you should prompt the user for the credentials.  Note that you should give the user the ability to disable the automatic sending of default credentials because of possible security concerns.  Note that in V2.0 of the framework, we added the ability for a user to specify when credentials should be sent to using the ICredentialPolicy interface. If you are running in a system service, prompting the user is not really applicable.
  5. If ISA Firewall Client is installed and enabled, advise users of your application to correct (remove) IE settings for the proxy server.  This will allow all requests to go through the Firewall Client (which will handle authentication for you automatically).  If the user is unwilling to do this, then the user should disable the proxy for your application through a config file.

For a great read on how the System.Net Proxy objects work, take a look at this article that was written by Durga Gorti (a former manager of mine).

Posted by joncole | 2 Comments
Filed under:

Simple Message Framing Sample for TCP Socket - Part 2 (Asynchronous)

As a follow up to my last post (http://blogs.msdn.com/joncole/archive/2006/03/20/555721.aspx) I decided to add some sample code for implementing an asynchronous version of ReadMessage.  All of the same rules apply as discussed in my previous post, but we now have more code we have to create in order to maintain state across calls to our handling code.  I have chosen not to implement the sending of a message asynchronously because the reading of a message asynchronously is much more interesting. 

 

We also have the same basic logic as the synchronous implementation (read size, then read the body) but we have to keep state across multiple calls to our code that handles the arrival of data. 

 

Note: There are some special cases related to this sample code that you will need to handle (or at least take into consideration)

1) Local socket is closed by another thread.  This could result in an ObjectDisposedException being thrown from this code when accessing the socket.

2) Remote side resets the connection.  This results in a SocketException being thrown when using the socket to send or receive data

3) Simultaneous duplicate calls to ReadMessage or ReadMessageAsync on the same socket should not be allowed.  Doing this will result very odd behavior.  Some examples are: you could potentially run into chopped messages, apparent hangs (one side expecting to read more data than the other side is going to send), or OutOfMemoryExceptions (because you interpret a portion of the message body of one request to be the message size of another request and allocate too much memory). Basically you are causing an alignment problem in your stream protocol parsing, which breaks the entire app.

 

First, we need a couple of objects for holding our state and some helper code to handle the asynchronous notification to the user that a message has arrived.

//**********************************************

//the definition of a function that the user has to give us for us to

//notify him/her that a message has arrived.

//**********************************************
public delegate void ReadMessageAsyncCallback(ReadMessageEventArgs e);

//**********************************************
//This is the argument passed to the above delegate
//**********************************************
public
class ReadMessageEventArgs : EventArgs
{

    internal ReadMessageEventArgs(string msg)

    {

        Message = msg;

        Error = null;

    }

 

    internal ReadMessageEventArgs(Exception err)

    {

        Error = err;

        Message = null;

    }

 

    public readonly Exception Error;

    public readonly string Message;

 

}

 

//**********************************************
//this class keeps track of our state during the read operation

//**********************************************
internal
class ReadMessageAsyncState : IDisposable

{

    public ReadMessageAsyncState(Socket socket, ReadMessageAsyncCallback callback)

    {

        if (socket == null)

            throw new ArgumentNullException("socket");

        if (callback == null)

            throw new ArgumentNullException("callback");

 

        userSocket = socket;

        userCallback = callback;

        buffer = new byte[4];

                      

        bytesReceived = 0;

        messageSize = -1;

    }

 

    public void Dispose()

    {

        userSocket = null;

        userCallback = null;

        buffer = null;

    }

               

    public Socket userSocket;

    public ReadMessageAsyncCallback userCallback;

    public int bytesReceived;

    public byte[] buffer;

    public int messageSize;

}

 

 

 

Now that we have defined all of the helper objects lets get to the new ReadMessageAsync function implementation.  This is the function that would be called to start the async read message process.  It basically just creates the async state object and then starts an async read on the socket.  Note that in this implementation I have not provided a way to cancel the asynchronous operation in case where other side of stream is behaving badly.  You may need to consider doing so for your application.

public static void ReadMessageAsync(Socket socket,

            ReadMessageAsyncCallback callback)

{

    ReadMessageAsyncState state = new ReadMessageAsyncState(socket, callback);

 

    socket.BeginReceive(state.buffer,

        0, //offset

        state.buffer.Length, //how much data can be read

        SocketFlags.None, new AsyncCallback(OnReceive), state);

}

Now, all the magic really happens in the OnReceive function (my callback passed to the BeginReceive calls on the socket).  This fuction is what is responsible for starting the next read on the socket if more data is needed, it calls the user callback when the entire message has been received and it handles any error and sets it on the ReadMessageEventArgs object.

static void OnReceive(IAsyncResult ar)

{

    ReadMessageAsyncState state = ar.AsyncState as ReadMessageAsyncState;

    try

    {

        int count = state.userSocket.EndReceive(ar);

        state.bytesReceived += count;

 

        if (state.messageSize == -1)//we are still reading the size of the data

        {

            if (count == 0)

                throw new ProtocolViolationException("The remote peer closed the connection while reading the message size.");

 

            if (state.bytesReceived == 4)//we have received the entire message size information

            {

                //read the size of the message

                state.messageSize = BitConverter.ToInt32(state.buffer, 0);

 

                if (state.messageSize < 0)

                {                           

                    throw new ProtocolViolationException("The remote peer sent a negative message size.");

                }

 

                //we should do some size validation here also (e.g. restrict incoming messages to x bytes long)

                state.buffer = new Byte[state.messageSize];

 

                //reset the bytes received back to zero

                //because we are now switching to reading the message body

                state.bytesReceived = 0;

            }

 

            if (state.messageSize != 0)

            {

 

                //we need more data - could be more of the message size information

                //or it could be the message body.  The only time we won't need to

                //read more data is if the message size == 0

                state.userSocket.BeginReceive(state.buffer,

                    state.bytesReceived, //offset where data can be written

                    state.buffer.Length - state.bytesReceived, //how much data can be read into remaining buffer

                    SocketFlags.None, new AsyncCallback(OnReceive), state);

            }

            else

            {

                //we have received a zero length message, notify the user...

                ReadMessageEventArgs args = new ReadMessageEventArgs(String.Empty);

                state.userCallback(args);

                state.Dispose();

            }            

        }

        else //we are reading the body of the message

        {

            if (state.bytesReceived == state.messageSize) //we have the entire message

            {

                //notify the user

                state.userCallback(new ReadMessageEventArgs(Encoding.ASCII.GetString(state.buffer)));

               

                //free up our reference to the socket, buffer and the callback object.

                state.Dispose();

            }

            else //need more data.

            {

                if (count == 0)

                    throw new ProtocolViolationException("The remote peer closed the connection before the entire message was received");

 

                state.userSocket.BeginReceive(state.buffer,

                    state.bytesReceived, //offset where data can be written

                    state.buffer.Length - state.bytesReceived, //how much data can be read into remaining buffer

                    SocketFlags.None, new AsyncCallback(OnReceive), state);

            }                   

        }

    }

    catch (Exception ex)

    {

        ReadMessageEventArgs args = new ReadMessageEventArgs(ex);

        state.userCallback(args);

        state.Dispose();

    }

}

I would also recommend that you read Malar Chinnusamy's blog on socket programming considerations.

Simple Message Framing Sample for TCP Socket

A common misunderstanding for developers new to network programming over TCP sockets is how messages are sent and received.  I frequently hear the statement that "my data is not arriving on the other side of the socket in the same format that I sent it."  The most common cause of this is because the TCP protocol does not guarantee that it will keep message boundaries.  In other words, you could send "Hello World" in a single call to Send(), but the other side of the socket stream may have to do multiple Receive() calls on the socket to get all of the data (the first Receive might return "He" and the second "llo World").  Data might also be concatenated together into a single TCP packet on the wire.  The client may send "Hello World"  and "I am in Seattle" in different calls to Send(), but the other side of the socket may receive it with a single call to Receive() in the format "Hello WorldI am in Seattle".

I thought I would give a simple example that shows how to frame your message so that it comes out on the other side of the network stream in the same format in which it was sent.  Your implementation may need to be more complex than what I have given here. 

Also, I have written this in C# and please be aware that this code does not handle errors in the stream and does not do parameter checking.  If the connection get's dropped in the middle of a mesage or if the msg being sent is greater than what can be sent with an integer size, you will see funny results.  Here, I am trying to keep the code simple and to the point.

      static void SendMessage(Socket socket, string msg)
        {
            byte[] data = Encoding.ASCII.GetBytes(msg);
            byte[] sizeinfo = new byte[4];

            //could optionally call BitConverter.GetBytes(data.length);
            sizeinfo[0] = (byte)data.Length;
            sizeinfo[1] = (byte)(data.Length >> 8);
            sizeinfo[2] = (byte)(data.Length >> 16);
            sizeinfo[3] = (byte)(data.Length >> 24);

            socket.Send(sizeinfo);
            socket.Send(data);
        }


        //note, this function does not handle closed connections in the middle of a message...
        static string ReadMessage(Socket socket)
        {
            byte[] sizeinfo = new byte[4];

            //read the size of the message
            int totalread = 0, currentread = 0;

            currentread = totalread = socket.Receive(sizeinfo);

            while (totalread < sizeinfo.Length && currentread > 0)
            {
                currentread = socket.Receive(sizeinfo,
                          totalread, //offset into the buffer
                          sizeinfo.Length - totalread, //max amount to read
                          SocketFlags.None);

                totalread += currentread;
            }

            int messagesize = 0;

            //could optionally call BitConverter.ToInt32(sizeinfo, 0);
            messagesize |= sizeinfo[0];
            messagesize |= (((int)sizeinfo[1]) << 8);
            messagesize |= (((int)sizeinfo[2]) << 16);
            messagesize |= (((int)sizeinfo[3]) << 24);
           
            //create a byte array of the correct size
            //note:  there really should be a size restriction on
            //              messagesize because a user could send
            //              Int32.MaxValue and cause an OutOfMemoryException
            //              on the receiving side.  maybe consider using a short instead
            //              or just limit the size to some reasonable value
            byte[] data = new byte[messagesize];

            //read the first chunk of data
            totalread = 0;
           
currentread = totalread = socket.Receive(data,
                         totalread, //offset into the buffer
                        data.Length - totalread, //max amount to read
                        SocketFlags.None);

            //if we didn't get the entire message, read some more until we do
            while (totalread < messagesize && currentread > 0)
            {
                currentread = socket.Receive(data,
                         totalread, //offset into the buffer
                        data.Length - totalread, //max amount to read
                        SocketFlags.None);
                totalread += currentread;

            }

            return Encoding.ASCII.GetString(data, 0, totalread);                   
        }

I would also recommend that you read Malar Chinnusamy's blog on socket programming considerations 

Posted by joncole | 11 Comments
Filed under: ,

Debugging a memory leak in managed code: Ping - SendAsync

Included in version 2.0 of the .NET framework is a Ping class (System.Net.NetworkInformation namespace) that can be used to monitor the up/down status of a network connection to a server.  We recently realized that we had a minor bug in the class that can present itself as a major problem - a memory leak.  Fortunately there is an easy workaround to this problem. I thought I would take this as an oportunity to demonstrate how to use the windbg debugger as I explain how we figured out what was going wrong.

Lets consider this simple console ping application that sends a series of ping requests to a server using the SendAsync method.

using System;
using System.Threading;
using System.Net.NetworkInformation;


class program {
 static int PingsOutstanding;

 static void Main(string[] args)
 {
  string targetServer = "www.contoso.com";
  for (int i = 0; i < 1000; i++) {
   PingServer(targetServer);
  }

  int currentcount = PingsOutstanding;
  while (currentcount > 0) {
   Console.WriteLine("waiting for {0} ping requests to complete", currentcount);
   Thread.Sleep(1000);
   currentcount = PingsOutstanding;
  }

  //the calls to the GC are just for debugging purposes
  Console.WriteLine("Calling GC.Collect()");
  GC.Collect();
  GC.WaitForPendingFinalizers();

  Console.WriteLine("Press <enter> to exit application");
  Console.ReadLine();
 }

 static void PingServer(string hostnameoraddress)
 {
  try {
   Ping pingsender = new Ping();
   pingsender.PingCompleted += new PingCompletedEventHandler(OnPingCompleted);
   pingsender.SendAsync(hostnameoraddress, null);
   Interlocked.Increment(ref PingsOutstanding);
  } catch (Exception ex) {
   Console.WriteLine(ex);
  }
 }


 static void OnPingCompleted(object sender, PingCompletedEventArgs e)
 {
  Interlocked.Decrement(ref PingsOutstanding);

  try {
   Ping pingsender = (Ping)sender;

   pingsender.Dispose();
  } catch (Exception ex) {
   Console.WriteLine(ex);
  }
 }
}

This application will not consume a ton of memory under normal operation but if you increase the number of ping requests that are sent, you will see that memory consumption also climbs.  Given that the .Net framework uses a garbage collection approach to memory allocation and cleanup, you would expect that the memory would be cleaned up properly within a reasonable amount of time.  You should also notice that I am calling pingSender.Dispose() which we would hope would release the resources being held by this object.

For demonstration purposes I have added a Console.ReadLine() call at the end of the application so that it won't exit.  I have also added a call to GC.Collect() and GC.WaitForPendingFinalizers() to make sure that when I go to debug this I am certain that the GC has recently run and cleaned up any objects that are available for cleanup.  It is not usually advisable to force GC collections except for debugging purposes.  See Maoni's blog for great information on how the GC works and how to use it effectively.

Using Windbg.exe

I chose to use Windbg because I have found it to be the best debugger for memory leaks.  It is a very powerful GUI debugger that is used very frequently inside of Microsoft for debugging everything from application bugs to OS bugs.  You can download it from here: http://www.microsoft.com/whdc/devtools/debugging/default.mspx

A couple of quick notes before I begin:

  1. updates to these debugging tools are available regularly, so make sure to update periodically.
  2. I will color commands that are to be run inside the debugger in green
  3. I have access to all of the debugger symbols for the .Net framework, but you don't.  However, most of the information you need should be accessible by using ".symfix" as noted below.  The principles discussed here apply to debugging a memory leak in any managed application.
    • public microsoft symbols can loaded into windbg using the .symfix  followed by a .reload command (requires internet connection to a microsoft http server)
  4. I may trim sample output (colored in purple) from the debugger in order to focus on the important information.

Now, Windbg is a native windows debugger and doesn't really know much about the .Net framework, so you have to load an extension to the debugger that understands managed applications.  SOS.dll is the debugger extension we are looking for. In the command window this will load the sos extension:

   .loadby sos mscorwks

This command tells the debugger to load the sos dll by looking in the same directory as the mscorwks dll, which is a core dll that is always loaded in a managed application.  Note that Windbg breaks into the debugger immediately upon loading the application, so mscorwks will not be loaded at this point.  You will have to run the application long enough for the .NET framework code to be loaded.  You can actually tell the debugger to break when mscorwks is loaded by running this in the command window:

   sxe ld:mscorwks

There are two ways that you can run commands from the sos this extension dll:

   !<command> or !sos.<command>

Here are examples for the help function found in sos (it prints usage information):

   !help
   !sos.help

The second one is useful when you have multiple extensions that share common function names.

When debugging this leak, I actually let the app run until it blocked on the call to Console.ReadLine() so that GC should have cleaned up.  The first thing I do when dealing with a managed memory leak is to look at all the objects that have been created by the runtime and see if I see any that jump out at me.  Lets look at the heap and see what it contains. 

   !dumpheap -stat
total 20600 objects
Statistics:
      MT    Count    TotalSize Class Name
7a76eb14        1           12 System.Net.DnsPermission
7a755834        1           12 System.Diagnostics.PerformanceCounterCategoryType
7a753394        1           12 System.Diagnostics.TraceOptions
7a71a710        1           12 System.Net.TimeoutValidator
7912b908        1           12 System.Collections.Generic.GenericEqualityComparer`1[[System.String, mscorlib]]
79112c90        1           12 System.Collections.Comparer
7910db30        1           12 System.Threading.SynchronizationContext
7910a718        1           12 System.DefaultBinder
79107f40        1           12 System.RuntimeTypeHandle
79107ac4        1           12 System.Reflection.__Filters
79102f48        1           12 System.__Filters
79102ef8        1           12 System.Reflection.Missing
79101ca8        1           12 System.RuntimeType+TypeCacheQueue
79100800        1           12 System.Text.DecoderExceptionFallback
...(output trimmed)
0105a514       77         4004 System.Configuration.FactoryRecord
791242ec       47         8136 System.Collections.Hashtable+bucket[]
79160a3c     1000        16000 System.Threading.RegisteredWaitHandle
79124228       58        19216 System.Object[]
7a779154     1000        20000 System.Net.SafeLocalFree
7a778ec0     1000        20000 System.Net.SafeCloseHandle
7a761bb0     1000        20000 System.ComponentModel.AsyncOperation
79160b84     1000        20000 System.Threading._ThreadPoolWaitOrTimerCallback
7910cf3c     1001        20020 Microsoft.Win32.SafeHandles.SafeWaitHandle
791609c8     1000        24000 System.Threading.RegisteredWaitHandleSafe
79115d0c     1000        24000 System.Threading.ManualResetEvent
7a7811f4     1000        32000 System.Net.NetworkInformation.PingCompletedEventHandler
79160aa8     1000        32000 System.Threading.WaitOrTimerCallback
7915ff38     1000        32000 System.Threading.SendOrPostCallback
7910d2f4     1001        36036 System.Threading.ExecutionContext
79124418     1004        44820 System.Byte[]
7a7812e0     1000        88000 System.Net.NetworkInformation.Ping
00150c90      421       108132      Free
790fa3e0     5262       304552 System.String
Total 20600 objects

Looking at the columns in the output, MT stands for Method Table and is basically a pointer to the table that describes that type of object.  Count is the number of objects that exist in the heap of the given type.  TotalSize is the amount of memory that is being consumed by any one type of object.  The last column is obviously the fully typed name of the object.

The first thing that jumps out at me in this case is the fact that we still have a large number of ping objects in the heap even though we aren't holding onto any references in our code.  We also called dispose on these objects, so this is a huge red light telling me that something went wrong.  Also, the fact that we sent exacly 1000 ping requests and there are exactly 1000 ping objects still hanging around calls my attention.  Lets take a look at these ping objects and see why they aren't getting cleaned up (Below I show two ways of doing this, you really only need to do one of them).

   !dumpheap -type Ping

This does a substring match on any object with Ping in it.  In this case it will list both System.Net.NetworkInformation.Ping objects and System.Net.NetworkInformation.PingCompletedEventHandler objects.  I really wanted just the Ping objects, so we could have run this command (using the MT for the ping object - obtained above)

   !dumpheap -mt 7a7812e0    

 Address       MT     Size
01271ff0 7a7812e0       88    
012720f8 7a7812e0       88    
012721d0 7a7812e0       88    
012722a8 7a7812e0       88    
01272380 7a7812e0       88    
01272458 7a7812e0       88    
01272530 7a7812e0       88    
01272608 7a7812e0       88    
012726e0 7a7812e0       88    
012727b8 7a7812e0       88    
01272890 7a7812e0       88    
01272968 7a7812e0       88    
...(output trimmed) 
012fada0 7a7812e0       88    
012faeec 7a7812e0       88    
012fb038 7a7812e0       88    
012fb184 7a7812e0       88    
012fb2d0 7a7812e0       88    
012fb41c 7a7812e0       88    
012fb568 7a7812e0       88    
012fb6b4 7a7812e0       88    
012fb800 7a7812e0       88    
012fb94c 7a7812e0       88    
012fba98 7a7812e0       88    
012fbbe4 7a7812e0       88    
012fbd30 7a7812e0       88    
012fbe7c 7a7812e0       88    
total 1000 objects
Statistics:
      MT    Count    TotalSize Class Name
7a7812e0     1000        88000 System.Net.NetworkInformation.Ping
Total 1000 objects

 Notice that the MT for all of the above objects matches that for the ping object.  The left hand column is the memory address where the object resides.  We can use this address to examine the actual ping object.  To take a look at the first object in the above list we would use this command:

   !dumpobject 01271ff0

I am not going to go into details at this point because this won't lead us where the problem lies.  I thought I would list the dumpobject command just to point it out to you.  The problem in this case is that objects that we think should be cleaned up are not cleaned up as expected.  This means that there must be a reference to the object somewhere.  The following command will help you to see what objects hold references to an object in question.  We will use the same object address we used in the dumpobject example:

   !gcroot 01271ff0

Note: Roots found on stacks may be false positives. Run "!help gcroot" for
more info.
Scan Thread 0 OSTHread 2f8
Scan Thread 2 OSTHread fcc
Scan Thread 3 OSTHread de8
Scan Thread 4 OSTHread eb8
Scan Thread 8 OSTHread c8c
Scan Thread 12 OSTHread 464
Scan Thread 13 OSTHread 1e0
DOMAIN(0014BF60):HANDLE(Strong):8f15b8:Root:0128d2c0(System.Threading._ThreadPoolWaitOrTimerCallback)->
01271ff0(System.Net.NetworkInformation.Ping)

This tool scans the threads to find any object that holds a reference to the given memory address.  We can see that some threadpool object is holding onto us.  If we go back to the objects on the heap we can see that there are these _ThreadPoolWaitOrTimerCallback objects on the heap also.  Now, I don't know a ton about the thread pool, but I would not expect those callback objects to still be on the heap either because they are most likely specific to a single usage of a threadpool thread. We could dump all of the _ThreadPoolWaitOrTimerCallback objects and check their roots (the objects holding onto them), but why dump the more objects from the heap?  The output from !gcroot gave us a pointer to an instance that we can quickly examine.

   !gcroot 0128d2c0

Note: Roots found on stacks may be false positives. Run "!help gcroot" for
more info.
Scan Thread 0 OSTHread 2f8
Scan Thread 2 OSTHread fcc
Scan Thread 3 OSTHread de8
Scan Thread 4 OSTHread eb8
Scan Thread 8 OSTHread c8c
Scan Thread 12 OSTHread 464
Scan Thread 13 OSTHread 1e0
DOMAIN(0014BF60):HANDLE(Strong):8f15b8:Root:0128d2c0(System.Threading._ThreadPoolWaitOrTimerCallback)->
01271ff0(System.Net.NetworkInformation.Ping)

We would have hoped that the Dispose() function would have cleaned these objects up, but it didn't.  So, what about the GC?  Why didn't GC take care of them when dispose didn't? Notice how this gave us the same exact output from the gcroot of the ping class.  This must mean that we have a circular reference (ping has a reference to this thread pool object and visa versa) and that is why the GC didn't clean it up.

Workaround (and reason why calling Dispose didn't clean this up)

The simple workaround to this problem is to cast the object to the IDisposable interface and then call Dispose on that casted object.

   ((IDisposable)pingSender).Dispose();

Why does this solve the problem?  Take a look at the declaration of the class:

   public class Ping:Component,IDisposable

Notice that it inherits from Component and it also implementes the IDisposable interface.  Now lets look at the function declaration for Dispose on the Ping class:

   void IDisposable.Dispose ()

In the implementation, the Dispose function was defined as part of the IDisposable interface but the fact that the base class Component also implements a Dispose function was missed.  The Ping class doesn't override the Component.Dispose method and so if you call Dispose directly on the Ping object without casting it to an IDisposable Interface, the runtime calls the base class implementation of Dispose instead of the one that was intended (the one for IDisposable). 

Note: There is another great blog on debugging managed memory leaks here: http://blogs.msdn.com/mvstanton/archive/2005/10/11/479861.aspx

Posted by joncole | 19 Comments
Filed under: ,

Managed classes to view/manipulate the Windows Firewall

I have been needing a set of classes to give me access to the settings for the Windows Firewall.  I did a quick search and didn't find much that would help me in my quest to view or manipulate these settings, so I ended up writing my own managed wrapper classes for the underlying COM objects.  I thought I would post my code here so that others can take advantage of the work I did.  Note that at this point I have only done cursory testing, so I don't promise that it is bug free.  Please let me know if you see any errors/improvements that need to be addressed.  Note that towards the bottom I have included a simple Main function that shows some sample usage.

using System;
using System.Collections;
using System.Text;
using System.Globalization;
using System.Security.Permissions;
using System.Security.Principal;
using System.ComponentModel;
using System.Runtime.InteropServices;

namespace WindowsFirewallTools
{


 [ComImport, ComVisible(false), Guid("304CE942-6E39-40D8-943A-B913C40C9CD4")]
    public class NetFwMgr {

    }

    [ComImport, ComVisible(false), Guid("F7898AF5-CAC4-4632-A2EC-DA06E5111AF2"), System.Runtime.InteropServices.InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
    public interface INetFwMgr {
       
   INetFwPolicy LocalPolicy {get;}

   FirewallProfileType CurrentProfileType {get;}

   void RestoreDefaults();

         void IsPortAllowed(string imageFileName,
          IPVersion ipVersion,
          long portNumber,
          string localAddress,
          IPProtocol ipProtocol,
          [Out] out bool allowed,
          [Out] out bool restricted);

        void IsIcmpTypeAllowed(IPVersion ipVersion,
          string localAddress,
          byte type,
          [Out] out bool allowed,
          [Out] out bool restricted);
    }

 [ComImport, ComVisible(false), Guid("D46D2478-9AC9-4008-9DC7-5563CE5536CC"), System.Runtime.InteropServices.InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
 public interface INetFwPolicy
 {

   INetFwProfile CurrentProfile{get;}
         INetFwProfile GetProfileByType(FirewallProfileType profileType);
 }

 [ComImport, ComVisible(false), Guid("174A0DDA-E9F9-449D-993B-21AB667CA456"), System.Runtime.InteropServices.InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
 public interface INetFwProfile
 {

  
   FirewallProfileType Type {get;}
         bool FirewallEnabled {get;set;}
   bool ExceptionsNotAllowed {get;set;}
   bool NotificationsDisabled {get;set;}
   bool UnicastResponsesToMulticastBroadcastDisabled {get;set;}
   INetFwRemoteAdminSettings RemoteAdminSettings {get;}
         INetFwIcmpSettings IcmpSettings {get;}
   INetFwOpenPorts GloballyOpenPorts {get;}
   INetFwServices Services {get;}
   INetFwAuthorizedApplications AuthorizedApplications {get;}
       
      
 }

 [ComImport, ComVisible(false), Guid("D4BECDDF-6F73-4A83-B832-9C66874CD20E"), System.Runtime.InteropServices.InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
 public interface INetFwRemoteAdminSettings
 {
         IPVersion IpVersion {get;set;}
        
   Scope Scope{get;set;}
       
   string RemoteAddresses{get;set;}
       
   bool Enabled {get;set;}
 }

 [ComImport, ComVisible(false), Guid("A6207B2E-7CDD-426A-951E-5E1CBC5AFEAD"), System.Runtime.InteropServices.InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
 public interface INetFwIcmpSettings
 {
         bool AllowOutboundDestinationUnreachable{get;set;}
       
   bool AllowRedirect{get;set;}
       
   bool AllowInboundEchoRequest{get;set;}

   bool AllowOutboundTimeExceeded{get;set;}

   bool AllowOutboundParameterProblem{get;set;}
       
   bool AllowOutboundSourceQuench{get;set;}

   bool AllowInboundRouterRequest{get;set;}
  
   bool AllowInboundTimestampRequest{get;set;}
       
         bool AllowInboundMaskRequest{get;set;}

   bool AllowOutboundPacketTooBig{get;set;}
       
 }

 [ComImport, ComVisible(false), Guid("C0E9D7FA-E07E-430A-B19A-090CE82D92E2"), System.Runtime.InteropServices.InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
 public interface INetFwOpenPorts
 {
         long Count {get;}
       
   void Add(INetFwOpenPort port);

   void Remove(long portNumber, IPProtocol ipProtocol);
       
   INetFwOpenPort Item(long portNumber, IPProtocol ipProtocol);
       
   System.Collections.IEnumerator _NewEnum{get;}
 }

 [ComImport, ComVisible(false), Guid("E0483BA0-47FF-4D9C-A6D6-7741D0B195F7"), System.Runtime.InteropServices.InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
 public interface INetFwOpenPort
 {


  string Name{get;set;}
       
  IPVersion IpVersion{get;set;}
  
  IPProtocol Protocol{get;set;}

  long Port {get;set;}

  Scope Scope{get;set;}
  
  string RemoteAddresses{get;set;}
  
  bool Enabled{get;set;}
    
  bool BuiltIn {get;}
       
 }

 [ComImport, ComVisible(false), Guid("79649BB4-903E-421B-94C9-79848E79F6EE"), System.Runtime.InteropServices.InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
 public interface INetFwServices
 {
         long Count {get;}
       
   INetFwService Item(ServiceType svcType);
       
         System.Collections.IEnumerator _NewEnum{get;}

 }

 [ComImport, ComVisible(false), Guid("79FD57C8-908E-4A36-9888-D5B3F0A444CF"), System.Runtime.InteropServices.InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
 public interface INetFwService
 {
         string Name{get;}
       
   ServiceType Type{get;}

   bool Customized{get;}

   IPVersion IpVersion{get;set;}
  
   Scope Scope{get;set;}
  
   string RemoteAddresses{get;set;}

         bool Enabled{get;set;}
       
         INetFwOpenPorts GloballyOpenPorts {get;}

 }

 [ComImport, ComVisible(false), Guid("644EFD52-CCF9-486C-97A2-39F352570B30"), System.Runtime.InteropServices.InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
 public interface INetFwAuthorizedApplications
 {
         long Count {get;}
       
   void Add(INetFwAuthorizedApplication port);

   void Remove(string imageFileName);
       
   INetFwAuthorizedApplication Item(string imageFileName);
       
   System.Collections.IEnumerator _NewEnum{get;}
 }

 [ComImport, ComVisible(false), Guid("EC9846B3-2762-4A6B-A214-6ACB603462D2")]
    public class NetFwAuthorizedApplication {

    }

 [ComImport, ComVisible(false), Guid("B5E64FFA-C2C5-444E-A301-FB5E00018050"), System.Runtime.InteropServices.InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
 public interface INetFwAuthorizedApplication
 {
         string Name{get;set;}
       
   string ProcessImageFileName{get;set;}

   
   IPVersion IpVersion{get;set;}
  
   Scope Scope{get;set;}
  
   string RemoteAddresses{get;set;}

         bool Enabled{get;set;}
 }

    public enum FirewallProfileType
 {
  Domain = 0,
  Standard = 1,
  Current = 2,
  Max = 3
 }

    public enum IPVersion
 {
  IPv4 = 0,
  IPv6 = 1,
  IPAny = 2,
  IPMax = 3
 }
    public enum IPProtocol
 {
  Tcp= 6,
  Udp= 17
 }

 public enum Scope
 {
  All = 0,
  Subnet = 1,
  Custom = 2,
  Max = 3
 }

 public enum ServiceType
 {
  FileAndPrint = 0,
  UPnP = 1,
  RemoteDesktop = 2,
  None = 3,
  Max = 4
 
 }


#if COMPILE_MAIN 
 public class App {
  //a sample main application that shows the usage of these objects.
  public static void Main()
  {
   try {

    INetFwMgr mgr = (INetFwMgr)new NetFwMgr();

    Console.WriteLine("CurrentProfileType: " +mgr.CurrentProfileType);

    INetFwProfile profile = mgr.LocalPolicy.CurrentProfile;
    Console.WriteLine("FirewallEnabled: " + profile.FirewallEnabled);


    System.Collections.IEnumerator e = null;

    e = profile.AuthorizedApplications._NewEnum;


     

    Console.WriteLine("\r\n-----  Applications  -----  ");
    while (e.MoveNext()) {
     INetFwAuthorizedApplication app = e.Current as INetFwAuthorizedApplication;
     Console.WriteLine("\t{0}\r\n\t\tImageFilename={1}\r\n\t\tEnabled={2}\r\n\t\tIpVersion={3}\r\n\t\tScope={4}\r\n\t\tRemoteAddresses={5}",
           app.Name,
           app.ProcessImageFileName,
           app.Enabled,
           app.IpVersion,
           app.Scope,
           app.RemoteAddresses
          );
    }


    e = profile.Services._NewEnum;
    Console.WriteLine("\r\n-----  Services  -----  ");
    while (e.MoveNext()) {
     INetFwService service = e.Current as INetFwService;
     Console.WriteLine("\t{0}\r\n\t\tType={1}\r\n\t\tEnabled={2}\r\n\t\tIpVersion={3}"+
           "\r\n\t\tScope={4}\r\n\t\tcustomized={5}\r\n\t\tRemoteAddresses={6}",
           service.Name,
           service.Type,
           service.Enabled,
           service.IpVersion,
           service.Scope,
           service.Customized,
           service.RemoteAddresses
          );
    }

    e = profile.GloballyOpenPorts._NewEnum;
    Console.WriteLine("\r\n-----  Globally Open Ports  -----  ");
    while (e.MoveNext()) {
     INetFwOpenPort port = e.Current as INetFwOpenPort;
     Console.WriteLine("\t{0}\r\n\t\tIsBuiltIn={1}\r\n\t\tEnabled={2}\r\n\t\tIpVersion={3}"+
           "\r\n\t\tScope={4}\r\n\t\tProtocol={5}\r\n\t\tRemoteAddresses={6}",
           port.Name,
           port.BuiltIn,
           port.Enabled,
           port.IpVersion,
           port.Scope,
           port.Protocol,
           port.RemoteAddresses
          );
    }


   } catch (Exception ex) {
    Console.WriteLine(ex);
   }
  }
 }

#endif //COMPILE_MAIN
}

Posted by joncole | 11 Comments
Filed under:

HttpWebRequest and Authentication - Part 1

Authentication over HTTP is a way for a client to prove its identity to the server so that it can gain access to a resource. In this series I will discuss the various types of authentication to servers and to proxy servers. I assume you have a basic understanding of HTTP including the use of headers and status codes. Note that all Url’s listed are examples and may or may not exist.

HTTP Authentication to a web server
When a client requests a resource to which the server does not allow anonymous access, the server will send a 401 error response to the client. Included in that response will be information that the client can use to determine which types of authentication the server supports. Here is an example request/response where the server responds with a 401 (some data has been left out so we can focus on the important information).


-------  Request -------
GET /protectedResource/file.htm HTTP/1.1
Host: www.contoso.com

------- Response ------- HTTP/1.1 401 Unauthorized WWW-Authenticate: NTLM,Negotiate,Kerberos,Digest qop="auth",algorithm=MD5-sess, nonce="c48850fd6e86c4013da6abceeb068c10ad9a668010af099834b51942bbb13f9ab5f6e65f2d89a802", opaque="39e6d50b360984701e23589a4a684347",charset=utf-8,realm="Digest",Basic realm="www.contoso.com"

Note the response contains the header “www-authenticate” which includes the types of authentication the server supports. It lists NTLM, Negotiate, Kerberos, Digest (plus some input parameters for Digest – explained in a later post) and Basic as supported authentication schemes.

The client then chooses from this list an authentication type that it also supports (if any). HttpWebRequest will choose the most secure protocol supported by both the client and server. The server and client then exchange information (possibly multiple times depending on the authentication type) so that the client can prove its identity to the server (which can also prove the identity of the server also). The client then resubmits the same request it issued earlier, but adds the “Authorization” header that contains the keyword “Basic” so that the server knows which type of authentication the client chose followed by a blob of data that the server should use to identify the client. In this example, the client chooses Basic authentication for simplicity’s sake (even though in the real world HttpWebRequest would not have chosen this authentication type above the others).


-------  Request -------
GET /protectedResource/file.htm HTTP/1.1
Host: www.contoso.com
Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=
The server then either sends back another 401 letting the client know that it failed to authenticate or it sends back 200 response status code so that the client knows that it passed the test.


-------  Response -------
HTTP/1.1 200 OK
Content-Length: 512

HTTP Authentication to a proxy server
Authentication to a proxy server is similar with a few modifications. The client still sends the same request to the proxy but instead of getting a 401 response status code it will get a 407 status code indicating that the proxy (as opposed to the end server) wants the client to authenticate. The proxy will also send back a “Proxy-Authenticate” header in place of the “www-authenticate” header. When the client goes to send the blob back to the proxy it will use the “proxy-authorization” header instead of the “authorization” header.

Posted by joncole | 0 Comments
Filed under:

HttpWebRequest and Expect 100 Continue

How do the ServicePointManager.Expect100Continue and ServicePoint.Expect100Continue properties change the behavior of my HttpWebRequest? This is a question that seems to come up from time to time and I thought I would give a little more information.

First, lets talk about the difference between the two versions of this property.  ServicePointManager is the master object that controls the creation and lookup of ServicePoints.  As a general rule, properties set on ServicePointManager apply to newly created ServicePoints from that point forward.  If you set the property on ServicePointManager after a ServicePoint is created, it will not be propogated to the already existing ServicePoints.

Now, having said that, lets look at what this property does.  In a nutshell, it tells the HttpWebRequest object whether or not to send the "Expect: 100-continue" header with the request on the wire.  What does this mean?  When this header is sent on the wire this tells the server that the client is going to delay sending the body of the request for some period of time because it wants the server to give the client the OK (a 100 continue response) to upload the data.  In current implementations, the HttpWebRequest object waits 350 milliseconds.  Why is this important?  There are many things that can go wrong during the HTTP request process.  The server could redirect the user to a different location, it could require authentication, the connection could have been dropped in between requests, etc.  In the case of redirection or authentication, this allows the client to try to avoid uploading the data (which it knows is useless) if possible.  One example of where the client can avoid uploading the data is when the client sets the HttpWebRequest.SendChunked property to true.  This means that the client will not tell the server the upload size, but will send the data in chunks.  At the beginning of each chunk is information on how big the chunk is and the client sends a zero length chunk to advise the server that there is no more data coming.

What about when the connection is dropped between requests?  Because HttpWebRequest keeps a pool of connections to any given server, the connection could be closed by the remote server without the client knowing that the connection has been dropped.  Due to the fact that upload requests cannot be assumed to be idempotent, the client cannot safely retry the request if an error occurs beyond a certain point in the request process.  If the client has sent even one byte of the upload body on the wire, then HttpWebRequest will fail if an error occurs in the connection.  How does HttpWebRequest know if it has sent the body on the wire?  It uses the delay between sending the headers (including the "Expect: 100-Continue" header) and sending they body as it's way figuring it out.  If Expect100Continue functionality is enabled, this gives a small window where upload requests can be retried safely.  If it is disabled, then an error that occurs when sending the request will cause the entire request to fail.  This is true even if the data never hits the wire -- all HttpWebRequest knows is that it sent the data down to the underlying socket layer and can't tell if the error occured before or after the data went out on the wire.

I typically suggest that developers leave Expect100Continue set to true unless they are desperate for performance gains or are trying to work around some issue with the server or client.

Posted by joncole | 5 Comments
Filed under:
 
Page view tracker