How to use SafeHandle in a Resilient Library [Brian Grunkemeyer]

How to use SafeHandle in a Resilient Library [Brian Grunkemeyer]

  • Comments 4

SafeHandle is the preferred mechanism for controlling the lifetime of an OS resource (such as a handle to a file, semaphore, or block of memory*) and guaranteeing that the resource is eventually freed.  Using some advanced reliability features (ie, constrained execution regions & critical finalization), SafeHandle lets you easily encapsulate OS resources without worrying about leaks caused by async exceptions.  This increased resiliency is crucial for server availability.  For a more theoretical discussion on the benefits, look here:  http://blogs.msdn.com/bclteam/archive/2005/03/16/396900.aspx 

 

Several library authors have asked me how to use SafeHandle in their own resource-wrapping classes, to improve their resiliency.  Here’s an example tailored precisely for library authors.  I’ve tried to make this example as close to a real-world example.  Since most people are familiar with file handles, my example mimics FileStream.  This way, we can look at what changes, if any, should be made to the finalizer & Dispose code paths.  And this may serve as an amusing diversion into the intricacies of writing a simple class, for those that aren’t familiar with our security model or design guidelines for P/Invoke methods or IDisposable, etc.  Lastly, my demo includes a low-tech fault injection mode, which runs this code repeatly in a loop, injecting thread aborts to demonstrate a race condition that causes a handle leak.

 

When dealing with OS resources, the methods you must call fall into three camps:  allocation, usage, and freeing.  Today, library code will define some P/Invoke methods to access the native resources, using an IntPtr (for running correctly on 32 bit and 64 bit machines).  Then the methods for freeing the resource are called in at least a finalizer, and usually through a Dispose method as well.  This unfortunately can leak resources in a few ways as well as causes correctness & security concerns. 

 

1)       An asynchronous exception can occur right after CreateFile returns, and before we move the handle from a register into our _handle field, causing us to leak the handle.

2)       During rude appdomain unloads in hosted environments like SQL Server, both finalizers & finally blocks aren’t guaranteed to run, so we might never get a chance to free the handle.

3)       In the ReadContents method below, we need to use GC.KeepAlive or HandleRef to prevent the GC from finalizing this instance during a P/Invoke call.  Otherwise, we may start reading from an invalid file handle, or worse yet, a recycled file handle that refers to a completely different file!

 

Go ahead and read the V1.1 code, see how it looks perfectly reasonable, and run it with the -fault option & watch it leak handles in perfmon!

 

To fix this, we invented SafeHandle, as a core reliability primitive that is baked into our P/Invoke marshaling layer.  You provide a subclass and override the ReleaseHandle method, putting in all your appropriate cleanup code.  Then you change your P/Invoke methods to use your SafeHandle subclass wherever you were using IntPtr’s for handles before.

 

SafeHandle’s cleanup code is guaranteed to run during even rude appdomain unloads, allowing you to free your resources.  This happens because of two CLR features – critical finalization, and constrained execution regions.  The CLR will eagerly prepare SafeHandle’s finalizer and ReleaseHandle method before allocating the first instance, ensuring that during an AD unload, the CLR can’t run out of memory trying to JIT your code.  Then since the critical finalizer’s callgraph is a constrained execution region, the CLR also doesn’t inject thread aborts in that code.  Note that being a constrained execution region, you have some special constraints on your code.  You must have a suitably strong reliability contract on the methods you call, you can’t call virtual methods (without extra work), and you can’t do anything that may fail, such as allocate an object, without some other mitigation.  You can read more about CER’s here: http://blogs.msdn.com/bclteam/archive/2005/06/14/429181.aspx

 

SafeHandle’s reliability come from the fact that it controls the lifetime of the handle, and all usages of the handle.  That way, the underlying OS resource cannot be freed underneath us, or be used by another thread while the SafeHandle is being closed.  While there is a DangerousGetHandle method to extract the handle value, this should only be used in constrained execution regions where you have explicitly incremented & decremented SafeHandle’s ref count, using DangerousAddRef and DangerousRelease.  Otherwise, you run into the same set of problems described above.

 

You may want to run windiff to compare these two files, to appreciate the changes.  We make resiliency fun!

 

 

 

// SafeHandle demo, V1.1 version.  This example shows the best state of

// the art as of v1.1 (without SafeHandle) for writing a class that wraps

// an OS handle.  This sample shows how to harden an existing class library

// to correctly free resources even in the presence of async exceptions.

// This tries to mimic a real library as close as possible, to show you

// precisely what design & implementation considerations may arise.

// Note the P/Invoke method signatures, and the Dispose & finalizer below.

// Also, below I have a primitive fault injection harness for testing this

// code, to demonstrate the handle leak in v1.1.

 

using System;

using System.Runtime.InteropServices;

using System.IO;

using System.ComponentModel;

using System.Security.Permissions;

using System.Security;

using System.Threading;

 

namespace SafeHandleDemoV1_1

{

    // We've reviewed this class & take responsibility for security reviews, etc.

    [SuppressUnmanagedCodeSecurity()]

    internal static class NativeMethods

    {

        // Win32 constants for accessing files

        internal static readonly IntPtr InvalidHandleValue = new IntPtr(-1);

        internal const int GENERIC_READ = unchecked((int)0x80000000);

 

        // Allocate a file object in the kernel, then return a handle to it.

        [DllImport("kernel32", CharSet=CharSet.Auto, SetLastError=true)]

        internal extern static IntPtr CreateFile(String fileName,

           int dwDesiredAccess, System.IO.FileShare dwShareMode,

           IntPtr securityAttrs_MustBeZero, System.IO.FileMode dwCreationDisposition,

           int dwFlagsAndAttributes, IntPtr hTemplateFile_MustBeZero);

 

        // Use the file handle

        [DllImport("kernel32", SetLastError=true)]

        internal extern static int ReadFile(IntPtr handle, byte[] bytes,

           int numBytesToRead, out int numBytesRead, IntPtr overlapped_MustBeZero);

 

        // Free the kernel's file object (close the file)

        [DllImport("kernel32", SetLastError=true)]

        internal extern static bool CloseHandle(IntPtr handle);

    }

 

    // This is a sample class that accesses an OS resource and implements

    // IDisposable.  This is useful to show the types of transformation that

    // are required to make your resource-wrapping classes more resilient.

    // Note the Dispose & finalizer implementations.  Consider this a very

    // good simulation of System.IO.FileStream.

    public class MyFileReader : IDisposable

    {

        // _handle is set to NativeMethods.InvalidHandleValue to indicate

        // that we've disposed of this instance.

        private IntPtr _handle;

 

        public MyFileReader(String fileName)

        {

            // Security permission check

            String fullPath = Path.GetFullPath(fileName);

            new FileIOPermission(FileIOPermissionAccess.Read, fullPath).Demand();

 

            // Open the file, and save the file handle in _handle

            // Note that the most optimized code turns into two processor

            // instructions: 1) a call, and 2) moving the return value into

            // the _handle field.

            // We can get an async exception in that window, and leak a handle.

            // I'm making this call then mov sequence more explicit here.

            IntPtr tmpHandle;

            tmpHandle = NativeMethods.CreateFile(fileName, NativeMethods.GENERIC_READ,

                FileShare.Read, IntPtr.Zero, FileMode.Open, 0, IntPtr.Zero);

            // An async exception here will cause us to leak the handle. 

            // I'm sleeping here to explicitly allow an abort to happen in this

            // window, making a rare race condition much easier to hit in our

            // fault injection harness (below in Main)

            Thread.Sleep(500);

            _handle = tmpHandle;  // moves eax into wherever _handle is stored.

 

            // Ensure we successfully opened the file.

            if (_handle == NativeMethods.InvalidHandleValue)

                throw new Win32Exception(Marshal.GetLastWin32Error(), fileName);

        }

 

        public void Dispose()  // Follow the Dispose pattern - public nonvirtual

        {

            Dispose(true);

            GC.SuppressFinalize(this);

        }

 

        ~MyFileReader()

        {

            Dispose(false);

        }

 

        protected virtual void Dispose(bool disposing)

        {

            // Note there are three interesting states here:

            // 1) CreateFile failed, handle is InvalidHandleValue

            // 2) We called Dispose already

            // 3) _handle is zero-initialized, due to an async exception before

            //    calling CreateFile.  The finalizer runs if the .ctor fails!

            if (_handle != NativeMethods.InvalidHandleValue &&

                _handle != IntPtr.Zero)

            {

                // Free the handle

                bool r = NativeMethods.CloseHandle(_handle);

                if (!r)

                    throw new Win32Exception(Marshal.GetLastWin32Error());

            }

            // Record the fact that we've closed the handle.

            _handle = NativeMethods.InvalidHandleValue;

        }

 

        public byte[] ReadContents(int length)

        {

            if (_handle == NativeMethods.InvalidHandleValue)  // Disposed?

                throw new ObjectDisposedException("FileReader is closed");

 

            // This is just sample code & won't work for all files.

            byte[] bytes = new byte[length];

            int numRead = 0;

            int r = NativeMethods.ReadFile(_handle, bytes, length, out numRead, IntPtr.Zero);

            // The GC can decide to run here, and may determine that this instance

            // is no longer accessible, and it may run our finalizer.  If that

            // happens during the call to ReadFile above, we may be passing in an

            // invalid handle.  In even more rare cases, another thread might use

            // that same handle number for a completely unrelated file!

            // Calling GC.KeepAlive or using HandleRef diligently will help prevent

            // this problem, by telling the GC this instance is still live.

            GC.KeepAlive(this);

            if (r == 0)

                throw new Win32Exception(Marshal.GetLastWin32Error());

            if (numRead < length) {

                byte[] newBytes = new byte[numRead];

                Array.Copy(bytes, newBytes, numRead);

                bytes = newBytes;

            }

            return bytes;

        }

    }

 

    static class Program

    {

        // For building a fault injection tester

        private static bool _printToConsole = false;

        private static bool _workerStarted = false;

 

        private static void Usage() {

            Console.WriteLine("Usage:");

            Console.WriteLine("HexViewer <fileName> [-fault]");

            Console.WriteLine(" -fault   Run the hex viewer over & over in a loop, injecting faults.");

        }

 

        private static void ViewInHex(Object fileName)

        {

            _workerStarted = true;

            byte[] bytes;

            using (MyFileReader reader = new MyFileReader((String)fileName)) {

                bytes = reader.ReadContents(20);

            }  // Using block calls Dispose() for us here

 

            if (_printToConsole) {

                int printNBytes = Math.Min(20, bytes.Length);

                Console.WriteLine("First {0} bytes of {1} in hex", printNBytes, fileName);

                for (int i = 0; i < printNBytes; i++)

                    Console.Write("{0:x} ", bytes[i]);

                Console.WriteLine();

            }

        }

 

        static void Main(string[] args)

        {

            if (args.Length == 0 || args.Length > 2 ||

                args[0] == "-?" || args[0] == "/?") {

                Usage();

                return;

            }

 

            String fileName = args[0];

            bool injectFaultMode = args.Length > 1;

            if (!injectFaultMode) {

                _printToConsole = true;

                ViewInHex(fileName);

            }

            else {

                Console.WriteLine("Injecting faults - watch handle count in perfmon (press Ctrl-C when done)");

                int numIterations = 0;

                while (true) {

                    _workerStarted = false;

                    Thread t = new Thread(new ParameterizedThreadStart(ViewInHex));

                    t.Start(fileName);

                    Thread.Sleep(1);

                    while (!_workerStarted) {

                        Thread.Sleep(0);

                    }

                    t.Abort();  // Normal apps shouldn't do this, of course.

                    numIterations++;

                    if (numIterations % 10 == 0)

                        GC.Collect();

                    if (numIterations % 10000 == 0)

                        Console.WriteLine(numIterations);

                }

            }

        }

    }

}

 

 

----------------------------------------------------

 

 

 

// SafeHandle demo, V2 version.  This example shows the best state of

// the art as of v2 (using SafeHandle) for writing a class that wraps

// an OS handle.  This sample shows how to harden an existing class library

// to correctly free resources even in the presence of async exceptions.

// This tries to mimic a real library as close as possible, to show you

// precisely what design & implementation considerations may arise.

// Note the P/Invoke method signatures, and the Dispose & lack of any

// finalizer below.  Also, below I have a primitive fault injection

// harness for testing this code, to demonstrate that the handle leak

// has been fixed.

 

using System;

using System.Runtime.InteropServices;

using System.IO;

using System.ComponentModel;

using System.Security.Permissions;

using System.Security;

using System.Threading;

using Microsoft.Win32.SafeHandles;

using System.Runtime.ConstrainedExecution;

 

namespace SafeHandleDemoV2

{

    // If this class is ever public, ensure the callers have unmanaged

    // code permission.

    [SecurityPermission(SecurityAction.LinkDemand, UnmanagedCode=true)]

    internal class MySafeFileHandle : SafeHandleZeroOrMinusOneIsInvalid

    {

        // Create a SafeHandle, informing the base class that this SafeHandle

        // instance "owns" the handle, and therefore SafeHandle should call

        // our ReleaseHandle method when the SafeHandle is no longer in use.

        // This method is called only via P/Invoke.

        private MySafeFileHandle() : base(true)

        {

        }

 

        [ReliabilityContract(Consistency.WillNotCorruptState, Cer.MayFail)]

        override protected bool ReleaseHandle()

        {

            // Here, we must obey all rules for constrained execution regions.

            return NativeMethods.CloseHandle(handle);

            // If ReleaseHandle failed, it can be reported via the

            // "releaseHandleFailed" managed debugging assistant (MDA).  This

            // MDA is disabled by default, but can be enabled in a debugger

            // or during testing to diagnose handle corruption problems.

            // We do not throw an exception because most code could not recover

            // from the problem.

        }

    }

 

    // We've reviewed this class & take responsibility for security reviews, etc.

    [SuppressUnmanagedCodeSecurity()]

    internal static class NativeMethods

    {

        // Win32 constants for accessing files

        internal const int GENERIC_READ = unchecked((int)0x80000000);

 

        // Allocate a file object in the kernel, then return a handle to it.

        [DllImport("kernel32", CharSet=CharSet.Auto, SetLastError=true)]

        internal extern static MySafeFileHandle CreateFile(String fileName,

           int dwDesiredAccess, System.IO.FileShare dwShareMode,

           IntPtr securityAttrs_MustBeZero, System.IO.FileMode dwCreationDisposition,

           int dwFlagsAndAttributes, IntPtr hTemplateFile_MustBeZero);

 

        // Use the file handle

        [DllImport("kernel32", SetLastError=true)]

        internal extern static int ReadFile(MySafeFileHandle handle, byte[] bytes,

           int numBytesToRead, out int numBytesRead, IntPtr overlapped_MustBeZero);

 

        // Free the kernel's file object (close the file)

        [DllImport("kernel32", SetLastError=true)]

        [ReliabilityContract(Consistency.WillNotCorruptState, Cer.MayFail)]

        internal extern static bool CloseHandle(IntPtr handle);

    }

 

    // This is a sample class that accesses an OS resource and implements

    // IDisposable.  This is useful to show the types of transformation that

    // are required to make your resource-wrapping classes more resilient.

    // Note the Dispose & finalizer implementations.  Consider this a very

    // good simulation of System.IO.FileStream.

    public class MyFileReader : IDisposable

    {

        // _handle is set to null to indicate that we've disposed of this instance.

        private MySafeFileHandle _handle;

 

        public MyFileReader(String fileName)

        {

            // Security permission check

            String fullPath = Path.GetFullPath(fileName);

            new FileIOPermission(FileIOPermissionAccess.Read, fullPath).Demand();

 

            // Open the file, and save the file handle in _handle

            // Note that the most optimized code turns into two processor

            // instructions: 1) a call, and 2) moving the return value into

            // the _handle field.  With SafeHandle, the CLR's Platform Invoke

            // marshaling layer will store the handle into the SafeHandle

            // object for us in an atomic fashion.  We still have the problem

            // that the SafeHandle object may not be stored in _handle, but

            // the real OS handle value has been safely stored in a critical

            // finalizable object, ensuring we won't leak the handle, even if

            // we get an async exception here.

            MySafeFileHandle tmpHandle;

            tmpHandle = NativeMethods.CreateFile(fileName, NativeMethods.GENERIC_READ,

                FileShare.Read, IntPtr.Zero, FileMode.Open, 0, IntPtr.Zero);

            // An async exception here will cause us to run our finalizer with

            // a null _handle, but MySafeFileHandle's ReleaseHandle code will

            // run to free the handle. 

            // This call to Sleep will help trigger a race when run from our

            // fault injection harness (below in Main).  This race won’t cause

            // a handle leak because the handle is already stored in a

            // SafeHandle instance.  Critical finalization then guarantees

            // that we can free the handle, even during a rude AppDomain unload.

            // I'll leave in this Sleep for comparison with the v1.1 version,

            // which would leak the handle under stress.

            Thread.Sleep(500);

            _handle = tmpHandle;  // Makes _handle point to a critical finalizable object.

 

            // Ensure we successfully opened the file.

            if (_handle.IsInvalid)

                throw new Win32Exception(Marshal.GetLastWin32Error(), fileName);

        }

 

        public void Dispose()  // Follow the Dispose pattern - public nonvirtual

        {

            Dispose(true);

            GC.SuppressFinalize(this);

        }

 

        // We don't need a finalizer.  ~MyFileReader has been removed.

        // The finalizer on SafeHandle will clean up the MySafeFileHandle

        // instance, if it hasn't already been disposed.

 

        // There may be a need for a subclass to introduce a finalizer,

        // so we will still properly implement the Dispose pattern, with a

        // protected Dispose(bool).

        protected virtual void Dispose(bool disposing)

        {

            // Note there are three interesting states here:

            // 1) CreateFile failed, _handle contains an invalid handle

            // 2) We called Dispose already, _handle is closed.

            // 3) _handle is null, due to an async exception before

            //    calling CreateFile.  The finalizer runs if the .ctor fails!

            if (_handle != null && !_handle.IsInvalid)

            {

                // Free the handle

                _handle.Dispose();

            }

            // SafeHandle records the fact that we've called Dispose.

        }

 

        public byte[] ReadContents(int length)

        {

            if (_handle.IsInvalid)  // Disposed?

                throw new ObjectDisposedException("FileReader is closed");

 

            // This is just sample code & won't work for all files.

            byte[] bytes = new byte[length];

            int numRead = 0;

            int r = NativeMethods.ReadFile(_handle, bytes, length, out numRead, IntPtr.Zero);

            // Since we removed MyFileReader's finalizer, we no longer need to

            // call GC.KeepAlive here.  P/Invoke will keep the SafeHandle

            // instance alive for the duration of the call.

            if (r == 0)

                throw new Win32Exception(Marshal.GetLastWin32Error());

            if (numRead < length) {

                byte[] newBytes = new byte[numRead];

                Array.Copy(bytes, newBytes, numRead);

                bytes = newBytes;

            }

            return bytes;

        }

    }

 

    static class Program

    {

        // For building a fault injection tester

        private static bool _printToConsole = false;

        private static bool _workerStarted = false;

 

        private static void Usage() {

            Console.WriteLine("Usage:");

            Console.WriteLine("HexViewer <fileName> [-fault]");

            Console.WriteLine(" -fault   Run the hex viewer over & over in a loop, injecting faults.");

        }

 

        private static void ViewInHex(Object fileName)

        {

            _workerStarted = true;

            byte[] bytes;

            using (MyFileReader reader = new MyFileReader((String)fileName)) {

                bytes = reader.ReadContents(20);

            }  // Using block calls Dispose() for us here

 

            if (_printToConsole) {

                int printNBytes = Math.Min(20, bytes.Length);

                Console.WriteLine("First {0} bytes of {1} in hex", printNBytes, fileName);

                for (int i = 0; i < printNBytes; i++)

                    Console.Write("{0:x} ", bytes[i]);

                Console.WriteLine();

            }

        }

 

        static void Main(string[] args)

        {

            if (args.Length == 0 || args.Length > 2 ||

                args[0] == "-?" || args[0] == "/?") {

                Usage();

                return;

            }

 

            String fileName = args[0];

            bool injectFaultMode = args.Length > 1;

            if (!injectFaultMode) {

                _printToConsole = true;

                ViewInHex(fileName);

            }

            else {

                Console.WriteLine("Injecting faults - watch handle count in perfmon (press Ctrl-C when done)");

                int numIterations = 0;

                while (true) {

                    _workerStarted = false;

                    Thread t = new Thread(new ParameterizedThreadStart(ViewInHex));

                    t.Start(fileName);

                    Thread.Sleep(1);

                    while (!_workerStarted) {

                        Thread.Sleep(0);

                    }

                    t.Abort();  // Normal apps shouldn't do this, of course.

                    numIterations++;

                    if (numIterations % 10 == 0)

                        GC.Collect();

                    if (numIterations % 10000 == 0)

                        Console.WriteLine(numIterations);

                }

            }

        }

    }

}

 

 

 

Page 1 of 1 (4 items)