If broken it is, fix it you should

Using the powers of the debugger to solve the problems of the world - and a bag of chips    by Tess Ferrandez, ASP.NET Escalation Engineer (Microsoft)

ASP.NET Crash: System.Security.Cryptography.CryptographicException

ASP.NET Crash: System.Security.Cryptography.CryptographicException

  • Comments 14

This has been a busy month for blogging for me, I'm up to a whopping 8 posts this month including this one which is the most I have written in any given month (since Feb 2006)...

We have seen a few cases lately where ASP.NET apps die due to an unhandled CryptographicException when finalizing a SafeHandle. Here is the explanation of why this happens and what you can do to avoid it.

Problem description

Intermittently ASP.NET will crash with the following entries in the appication eventlog

Event Type:	Error
Event Source:	ASP.NET 2.0.50727.0
Event Category:	None
Event ID:	         1334
Date:		2007-10-31
Time:		08:10:03
User:		N/A
Computer:	         PRATHER
Description:
An unhandled exception occurred and the process was terminated.

Application ID: /LM/w3svc/1/ROOT/TestCrypto

Process ID: 4296

Exception: System.Security.Cryptography.CryptographicException

Message: Keyset does not exist


StackTrace:    at System.Security.Cryptography.CryptographicException.ThrowCryptogaphicException(Int32 hr)
   at System.Security.Cryptography.SafeProvHandle._FreeCSP(IntPtr pProvCtx)
   at System.Security.Cryptography.SafeProvHandle.ReleaseHandle()
   at System.Runtime.InteropServices.SafeHandle.InternalFinalize()
   at System.Runtime.InteropServices.SafeHandle.Dispose(Boolean disposing)
   at System.Runtime.InteropServices.SafeHandle.Finalize()

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

 

followed by this event

Event Type:	Error
Event Source:	.NET Runtime 2.0 Error Reporting
Event Category:	None
Event ID:	         5000
Date:		2007-10-31
Time:		08:10:03
User:		N/A
Computer:	         PRATHER
Description:
EventType clr20r3, P1 w3wp.exe, P2 6.0.3790.3959, P3 45d6968e, P4 mscorlib, P5 2.0.0.0, P6 46693664, P7 4d78, P8 6, P9 udta330idobh2roz2ayvlcelag5agtls, P10 NIL.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

accompanied by this event in the system log...

Event Type:	Warning
Event Source:	W3SVC
Event Category:	None
Event ID:	         1009
Date:		2007-10-31
Time:		08:10:22
User:		N/A
Computer:	         PRATHER
Description:
A process serving application pool 'DefaultAppPool' terminated unexpectedly. The process id was '4296'. The process exit code was '0xe0434f4d'.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

 

 

So we know that we have an unhandled CryptographicException in SafeProvHandle._FreeCSP while finalizing a SafeHandle, but we don't know why.  On a related note, the reason the unhandled exception on the finalizer causes a process crash, and how to debug unhandled exceptions in general is described here.

It should also be noted that the callstacks for the same issue can be cryptographic exceptions in any of SafeProvHandle._FreeCSP, SafeHashHandle._FreeHash or SafeHandle._FreeHKey depending on the situation.

Cause:

A problem like this is pretty hard to track down because the correlation beteween cause and effect is not very obvious. 

This problem only occurrs if you create RSACryptoServiceProviders while impersonated and dont clean up the keys of the RSACryptoProvider using the clear method while impersonated.

            CspParameters cspParam = new CspParameters();
            cspParam.Flags = CspProviderFlags.UseMachineKeyStore;
            RSACryptoServiceProvider pair = new
            RSACryptoServiceProvider(cspParam);
            string keyInfo = pair.ToXmlString(false);
	    //... encrypt or decrypt data

 

What actually happens is this

  1. Impersonate user (either manually or through impersonate=true in web.config)
  2. Create a new RSACryptoServiceProvider with an Ephemeral Key like UseMachineKeyStore
  3. Undo impersonation (i.e. finish the request) without cleaning up the key
  4. The Key gets cleaned up when the finalizer runs under the process account

The problem here is that since the key is created while impersonated so it is created under the impersonated users profile.  When the finalizer runs it does not run under the impersonated user so the key does not exist and you get an exception like "keyset does not exist", and since there is noone there to handle it the process exits out.  Alternatively you could get an exception to the effect that the finalizer does not have permissions to delete a key or similar.

If you use a named keypair instead of the ephemeral keys, i.e. create a new CspParameter with an arbitrary KeyContainerName the problem will not occurr since the framework will not attempt to delete this when disposing of the RSACryptoServiceProvider.

 

To see the problem in action, put the above code in an aspx page with impersonate=true or surrounded by

            System.Security.Principal.WindowsImpersonationContext impersonationContext;
            impersonationContext = ((System.Security.Principal.WindowsIdentity)User.Identity).Impersonate();
		...
            impersonationContext.Undo();

 

 

And call GC.Collect(), GC.WaitForPendingFinalizers(), GC.Collect() in another aspx page.

Solution:

The solution to the problem is given in this blog post, essentially it comes down to

  1. Impersonate user
  2. Create a new RSACryptoServiceProvider (pair)
  3. Encrypt or Decrypt your data
  4. Call clear to release the keys (pair.Clear())
  5. Undo impersonation

Whenever we are dealing with objects that hold on to native resources we should follow the finalization/disposal rules in order to avoid higher than necessary memory usage and resource depletion, but in this case, not following the rules of disposing finalizable objects directly after use (by calling Close, Dispose or Clear, whichever the dispose method happens to be) causes even worse problems than just resource depletion...

Laters,
Tess 





  • AJAX JSON in ASP.NET Ajax: Part 3. Server side deserialization and elaboration of JSON data returned...

  • Further to this exception.

    We got the same exception when calling in .net managed dll from classic ASP

    It  appears w3wp was crashing when the Cryptography key is garbage collected.

    (As cryptography keys are created in MachineKeys when using a managed component called from a classic ASP page)

    So when GC kicked in , it couldn't

    remove the keyset.

    Number of exceptions of this type: 1

    Exception MethodTable: 791846e8

    Exception object: 062a98cc

    Exception type: System.Security.Cryptography.CryptographicException

    Message: Keyset does not exist

    Checking C:\Documents and Settings\All Users\Application Data\Microsoft\Crypto\RSA\MachineKeys was not having appropaite rights

    (marking Administrator & System : Full control resolve the issue)

    Thanks

    Jas

  • I have to say that I just love it when people start chipping in with their own experiences and solutions since it means that my original goal of this blog (helping people with their asp.net issues) has gone one step further with people helping each other.

    Awesome:)  thanks Jas

  • I have hit this error around a WCF call.  The odd thing is that it makes no reference to internal certificates.  It uses the following:

    <security mode="TransportWithMessageCredential">

    <transport clientCredentialType="None" proxyCredentialType="None" realm=""/>

    <message clientCredentialType="UserName" algorithmSuite="Default"/>

    </security>

  • I also observe the issue in C# prototype code similar to below. We would see the exception when exiting Main().

    In my case, I am testing PKCS#8 and X.509 serialization (custom AsnKeyBuikder and AsnKeyParser classes). I'm treating the keys as temporal, so PersistKeyInCsp = false.

    It would have taken quite some time to track this down. Thanks for the blog Tess...

    Jeff

    Jeffrey Walton

    Pasadena, MD

    * CreateDsaKeys() creates a DSA key pair, writes to disk, and then returns.

    * SignDsaMessage() retrieves the private key, signs a message, then writes the message and the signature on the message to disk.

    * VerifyDsaMessage() retrieves the public key, message and signature. It then verifies the message.

    internal static void Main(string[] args)

    {

     CreateDsaKeys();

     SignDsaMessage();

     VerifyDsaMessage();

    }

  • The loss of the user's principal could also happen when Finalize code is trying to clean up resources or make calls to objects which demand some permissions. (for Windows development). For the same reason as finalizer could have released Principal when Finalize code runs.

    So in that case you could use something like this in the Finilize method

    AppDomain.CurrentDomain.SetPrincipalPolicy(PrincipalPolicy.WindowsPrincipal)

    before you make a call to an object which requests such permissions.

  • I'm hitting this in SharePoint2007 when using Active Directory Federation Services.

    Somehow I don't think I'm going to be able to fix the code... Hopefully it's just a configuration issue...

  • I am seeing the very first error with ADFS and it is causing me no end of grief.  It typically starts with me getting the following error

    Server Error in '/adfs' Application.

    --------------------------------------------------------------------------------

    IAsyncResult returned from Begin method is null.

    Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

    Exception Details: System.InvalidOperationException: IAsyncResult returned from Begin method is null.

    Source Error:

    [No relevant source lines]

    Source File: c:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\adfs\b5e5f3d0\948ef83d\App_Web_i-rkvfv5.0.cs    Line: 0

    Stack Trace:

    [InvalidOperationException: IAsyncResult returned from Begin method is null.]

      System.Web.UI.PageAsyncInfo.CallHandlersPossiblyUnderLock(Boolean onPageThread) +385

      System.Web.UI.PageAsyncInfo.CallHandlersCancellableCallback(Object state) +67

      System.Web.HttpContext.InvokeCancellableCallback(WaitCallback callback, Object state) +74

      System.Web.UI.PageAsyncInfo.CallHandlers(Boolean onPageThread) +68

    [HttpException (0x80004005): Exception of type 'System.Web.HttpException' was thrown.]

      System.Web.HttpAsyncResult.End() +3378903

      System.Web.UI.Page.AsyncPageEndProcessRequest(IAsyncResult result) +21

      ASP.ls_clientlogon_aspx.EndProcessRequest(IAsyncResult ar) in c:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\adfs\b5e5f3d0\948ef83d\App_Web_i-rkvfv5.0.cs:0

      System.Web.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +183

      System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +161

    Then if I restart the World Wide Web Publishing Service, I get the System.Security.Cryptography.CryptographicException error.

  • I have had this problem with a Web Service that uses Bouncy Castle for DSA cryptography.

    The w3wp.exe process executing the asp.net program crashed after a System.Security.Cryptography.CryptographicException . Bouncy Castle being provided as MSIL-binary I could not apply your article's correction to the source.

    IN MY CASE THE SOLUTION WAS :

    In IIS, set the user for DefaultAppPool to be SYSTEM. Do this in IIS > (machine) > Application Pools > DefaultAppPool > Properties . Choose identity then set the user to "local system".

    Thus, even after "impersonnation" is finished, the garbage collector still has the necessary credentials (?) to destroy the problematic object.

    This can be a security problem but it was the only workaround I could find...

    Many thanks for this blog article !!

    JB

  • Thank you somuch for sharing this info. It was great article. I solved error by just adding pair.Clear();

  • Thanks so much for the information. I had this problem on one of my wesites and couldn't figure out what was causing the issue. Turning impersonation off did the trick for me. We are now using a different approach to the tasks that require impersonation.

    Thanks again

  • I realize its been a year since the last comment was posted here, but please allow me to offer a hearty "thanks" to the author. This article described and explained precisely a situation we were encountering with intermittent CryptographicException problems. We found the offending code, and switched to the named key container in the CspParameter object for the solution. No more stray CryptographicExceptions!

  • David,  I am very happy to hear that even a year after the post it still helps people:)

  • Another three years and still a helpful post! Spent the the day trying to track down an intermittent CryptographicException, this post was a lifesaver =)

Page 1 of 1 (14 items)
Leave a Comment
  • Please add 4 and 4 and type the answer here:
  • Post