Decrypt my World : Interop

The use of MSXML is not supported in .NET applications

alejacma — Mon, 08 Sep 2008 14:33:00 GMT

Hi all,

The other day I was working with a customer of mine on a .NET application which was suffering from a memory leak. The weird thing was that the application could run just fine for 10 days in a row before the memory began growing and growing. What was going on? Why was the Garbage Collector suddenly deciding to stop working? Additionally, this issue was happening with both .NET 1.1 and 2.0. Weird.

So we got a dump of a .NET 1.1 version of the application when the issue began. We used Adplus to take a hang dump we could analize (How to use ADPlus to troubleshoot "hangs" and "crashes").

We loaded SOS debugger extension (the one that comes with .NET Framework 1.1):

0:000>.load sos

And we went to check all objects currently on the heap:

0:000> !dumpheap -stat
Loaded Son of Strike data table version 5 from "C:\WINNT\Microsoft.NET\Framework\v1.1.4322\mscorwks.dll"
Loading the heap objects into our cache.
total 1,958,563 objects
Statistics:
        MT      Count     TotalSize Class Name
...

Almost 2 million objects! That is a lot. Are any of them ready for finalization?

0:000> !finalizequeue
Loaded Son of Strike data table version 5 from "C:\WINNT\Microsoft.NET\Framework\v1.1.4322\mscorwks.dll"
SyncBlock to be cleaned up: 6371
----------------------------------
MTA interfaces to be released: 0
Total STA interfaces to be released: 0
----------------------------------
generation 0 has 215 finalizable objects (0x17daef74->0x17daf2d0)
generation 1 has 0 finalizable objects (0x17daef74->0x17daef74)
generation 2 has 15,317 finalizable objects (0x17da0020->0x17daef74)
Ready for finalization 0 objects (0x17daf2d0->0x17daf2d0) - Freachable queue
All Finalizable Objects Statistics:
        MT      Count     TotalSize Class Name
...

So apparently there are no objects ready to be finalized, but we have 6371 SyncBlocks ready to be cleaned up. Why? What are those SyncBlocks?

0:000> !finalizequeue -detail
To be cleaned Com Data
0x4edc1b8  ComPlusWrapper
0x4edc590  ComPlusWrapper
...
0x4e227b8  ComPlusWrapper
SyncBlock to be cleaned up: 6371
----------------------------------
MTA interfaces to be released: 0
Total STA interfaces to be released: 0
...

Ok, so those SyncBlocks are related to ComPlusWrappers which need to be cleaned up. It seems the application is referencing some COM objects which are not needed anymore.

And what is the Finalizer thread doing that is not cleaning up those objects? Is it too busy?

0:000> !threads
ThreadCount: 23
UnstartedThread: 0
BackgroundThread: 17
PendingThread: 0
DeadThread: 0
                                  PreEmptive   GC Alloc                     Lock     
        ID  ThreadOBJ       State     GC       Context           Domain     Count APT Exception
  0  0x7a4 0x0013e1b8      0xa020 Enabled  0x00000000:0x00000000 0x00138fa0     0 MTA
  2  0x484 0x0014a508      0xb220 Enabled  0x00000000:0x00000000 0x00138fa0     0 MTA (Finalizer)
...
0:000> ~2s
0:002> kL100
ChildEBP RetAddr  
02a5f980 77f8822a NTDLL!ZwWaitForSingleObject+0xb
02a5f9f4 77f8819b NTDLL!RtlpWaitForCriticalSection+0x9e
02a5f9fc 72e55ff1 NTDLL!RtlEnterCriticalSection+0x46
02a5fa08 72e56010 msxml3!X_CRITICAL_SECTION::Enter+0xc
02a5fa10 72e5b735 msxml3!CSMutex::Enter+0xd
02a5fa20 72e5b6fd msxml3!RemovePointerFromCache+0x2e
02a5fa2c 72e55cfe msxml3!GCObject::~GCObject+0x15
02a5fa38 72e55c21 msxml3!_array<_reference >::`vector deleting destructor'+0xd
02a5fa44 72e5b5b6 msxml3!Base::weakRelease+0x19
02a5fa54 72e5b565 msxml3!Base::_release+0x95
02a5fa5c 72e5685d msxml3!DefaultWriter::Release+0xd
02a5fa74 72e60805 msxml3!assign+0x26
02a5fa8c 72e5ff5f msxml3!NodeData::setText+0x1e
02a5fa98 72e5b5a5 msxml3!Node::finalize+0x73
02a5faa8 72e60b7d msxml3!Base::_release+0x76
02a5fac0 72e6098b msxml3!Node::setNodeParent+0x7f
02a5fae0 72e5ff9a msxml3!Node::_remove+0x90
02a5faec 72e5b5a5 msxml3!Node::finalize+0x65
02a5fafc 72e60ae4 msxml3!Base::_release+0x76
02a5fb14 72e5c36e msxml3!Node::_release+0x77
02a5fb2c 72e5c1a7 msxml3!Node::_release+0x8d
02a5fb40 72e5b54e msxml3!Node::Release+0x26
02a5fb4c 72e70e12 msxml3!release+0x17
02a5fb58 72e70ddf msxml3!DOMNodeList::~DOMNodeList+0x16
02a5fb64 72e56962 msxml3!DOMNamedNodeMapList::`vector deleting destructor'+0x16
02a5fba4 72e5b8e3 msxml3!__comexport::Release+0x62
02a5fbac 79221be1 msxml3!_dispatchEx::Release+0x10
02a5fbdc 79232e77 mscorwks!SafeRelease+0x56
02a5fbf8 79232ea8 mscorwks!IUnkEntry::Free+0x7c
02a5fc04 79232f86 mscorwks!ComPlusWrapper::ReleaseAllInterfaces+0xa
02a5fe3c 79232e0d mscorwks!ComPlusWrapper::ReleaseAllInterfacesCallBack+0x135
02a5fe48 7923303c mscorwks!ComPlusWrapper::Cleanup+0x16
02a5fe58 79233302 mscorwks!ComPlusContextCleanupGroup::CleanUpWrappers+0x19
02a5fe64 792335bc mscorwks!ComPlusApartmentCleanupGroup::ReleaseCleanupGroup+0xe
02a5fe74 792325bb mscorwks!ComPlusApartmentCleanupGroup::ReleaseCleanupGroupCallback+0x51
02a5fe98 792324dc mscorwks!ComPlusApartmentCleanupGroup::CleanUpWrappers+0xa8
02a5fed0 791fea3d mscorwks!ComPlusWrapperCleanupList::CleanUpWrappers+0x84
02a5fedc 791fe9cd mscorwks!SyncBlockCache::CleanupSyncBlocks+0x85
02a5fee0 791bbd18 mscorwks!Thread::DoExtraWorkForFinalizer+0x23
02a5ff20 791cf03c mscorwks!GCHeap::FinalizerThreadStart+0xbb
02a5ffb4 7c57b3bc mscorwks!Thread::intermediateThreadProc+0x44
02a5ffec 00000000 KERNEL32!BaseThreadStart+0x52

Interesting. The Finalizer thread is trying to free an MSXML3 object, it entered a critical section and it's waiting for it to be released. We took a few more dumps and the Finalizer thread always stayed the same. It didn't move a bit. It's hung in there. So no wonder Garbage Collector can't free memory.

Why is Finalizer thread hung on a critical section when freeing MSXML objects?

Well, check this:

The use of MSXML is not supported in .NET applications

MSXML uses threading models and garbage-collection mechanisms that are not compatible with the .NET Framework.

Summing up, we fail to garbage-collect msxml objects because of these incompatibilities. This issue affects all versions of the Framework and MSXML. The only alternative we have is to use .Net's System.Xml namespace to work with XML as suggested by previous article.

I hope this helps.

Cheers,

Alex (Alejandro Campos Magencio)

How to modify an Interop assembly to change the return type of a method (VB.NET)

alejacma — Mon, 04 Aug 2008 09:52:00 GMT

Hi all,

In some situations we may need to reference a COM dll in our Visual Studio project in order to use a specific COM object within our .NET application, and thanks to COM Interop we can do it. When we add a reference to a COM dll in our project, a COM Interop assembly gets generated so we can use COM objects in the dll almost like any other .NET object. And I say almost because i.e. .NET needs to translate COM types to .NET types and it doesn't always do the conversion the way we like or need.

Let's see this with the following real example:

We are writing an Exit Module for our Certification Authority. We are trying to obtain the raw binary data for a certificate issued by our CA. We are calling CCertServerExit.GetCertificateProperty("RawCertificate", PROPTYPE_BINARY) (ICertServerExit::GetCertificateProperty Method) to achieve that. But GetCertificateProperty is returning a String and not an array of bytes, because .NET interop layer is converting a VARIANT of type BSTR (vt = VT_BSTR) to a .NET String. Because of this return type, .NET is trimming the last byte of the binary certificate when the number of bytes is odd. .NET strings cannot represent BSTRs that contain an odd number of bytes.

Fortunately we can modify the Interop assembly and choose a different return type for that method. This modification will allow us to get all the bytes in the certificate. These are the steps to do this:

1) Extract IL from Interop.CertClientLib.dll interop assembly generated by VS IDE when we added CertCli COM reference to our VS project:

ildasm.exe Interop.CertClientLib.dll /out:temp.il

2) Delete Interop.CertClientLib.dll file.

3) Delete CertCli COM reference from our project.

4) Modify temp.il and change the declaration of GetCertificateProperty in base and derived classes to take an IntPtr for the last output parameter instead of a VARIANT *:

instance void 
GetCertificateProperty([in] string  marshal( bstr) strPropertyName,
                       [in] int32 PropertyType,
                       [out] native int pvarPropertyValue) runtime managed internalcall

5) Generate a new Interop.CertClientLib.dll which includes these modifications:

ilasm.exe /DLL temp.il /res:temp.res /out=Interop.CertClientLib.dll

6) Modify our project's references to use this new DLL.

7) Change our .NET code to use the IntPtr calling convention. Manage Variant objects from IntPtr and free the memory after its use. Sample:

<DllImport("Oleaut32.dll", PreserveSig:=False)> _
Private Shared Sub VariantClear(ByVal var As IntPtr)
End Sub

'Exit Module Notify callback
Public Sub Notify(ByVal ExitEvent As Integer, ByVal Context As Integer) Implements CERTEXITLib.ICertExit.Notify

    Trace.WriteLine(">> Notify")

    Dim oCSE As New CERTCLIENTLib.CCertServerExit
    oCSE.SetContext(Context)

    Trace.WriteLine(String.Format("ExitEvent = {0}", ExitEvent))
    If ExitEvent = 1 Then 'Issue

        ' Allocate memory to hold VARIANT
        '
        Dim variantObjectPtr As IntPtr
        variantObjectPtr = Marshal.AllocHGlobal(2048)

        ' Get VARIANT containing certificate bytes
        '
        oCSE.GetCertificateProperty("RawCertificate", DataTypeEnum.Binary, variantObjectPtr)

        ' Read ANSI BSTR information from the VARIANT as we know RawCertificate property 
        ' is ANSI BSTR. Please note that the below code is written based on how the 
        ' VARIANT structure looks like in C/C++
        '
        Dim bstrPtr As IntPtr
        bstrPtr = Marshal.ReadIntPtr(variantObjectPtr, 8)

        Dim bstrLen As Integer
        bstrLen = Marshal.ReadInt32(bstrPtr, -4)

        Dim certBytes(bstrLen - 1) As Byte
        Marshal.Copy(bstrPtr, certBytes, 0, bstrLen)

        Trace.WriteLine(String.Format("Certificate length = {0}", certBytes.Length))

        ' Clear the VARIANT which will free the memory allocated for the VARIANT members
        '
        VariantClear(variantObjectPtr)

        ' Get certificate
        '
        Dim x509 As New X509Certificate(certBytes)

        ' Store the certificate in the certificate store
        '
        Dim store As New StoreClass
        Dim cert As New CertificateClass
        Dim data As String = Convert.ToBase64String(x509.GetRawCertData)

        cert.Import(data)
        store.Open(CAPICOM_STORE_LOCATION.CAPICOM_LOCAL_MACHINE_STORE, _
                    "My", _
                    CAPICOM_STORE_OPEN_MODE.CAPICOM_STORE_OPEN_READ_WRITE)
        store.Add(cert)

        ' Get VARIANT containing Serial Number
        '
        oCSE.GetCertificateProperty("SerialNumber", DataTypeEnum.String, variantObjectPtr)

        ' Construct Variant .NET object from IntPtr
        '
        Dim variantObject As Object
        variantObject = Marshal.GetObjectForNativeVariant(variantObjectPtr)

        ' Get Serial Number
        '
        Dim serialNo As String = variantObject

        Trace.WriteLine(String.Format("Serial Number = {0}", serialNo))

        ' Free memory
        '
        VariantClear(variantObjectPtr)

        ' Get VARIANT containing Distinguished Name
        '
        oCSE.GetCertificateProperty("DistinguishedName", DataTypeEnum.String, variantObjectPtr)

        ' Construct Variant .NET object from IntPtr
        '
        variantObject = Marshal.GetObjectForNativeVariant(variantObjectPtr)

        ' Get Serial Number
        '
        Dim distinguishedName As String = variantObject

        Trace.WriteLine(String.Format("DistinguishedName = {0}", distinguishedName))

        ' Free memory
        '
        VariantClear(variantObjectPtr)

        ' Get VARIANT containing Not Before date
        '
        oCSE.GetCertificateProperty("NotBefore", DataTypeEnum.Date, variantObjectPtr)

        ' Construct Variant .NET object from IntPtr
        '
        variantObject = Marshal.GetObjectForNativeVariant(variantObjectPtr)

        ' Get Not Before date
        '
        Dim validFrom As Date = variantObject

        Trace.WriteLine(String.Format("Not Before = {0}", validFrom.ToString))

        ' Free memory
        '
        VariantClear(variantObjectPtr)

        ' Get VARIANT containing Not After date
        '
        oCSE.GetCertificateProperty("NotAfter", DataTypeEnum.Date, variantObjectPtr)

        ' Construct Variant .NET object from IntPtr
        '
        variantObject = Marshal.GetObjectForNativeVariant(variantObjectPtr)

        ' Get Not After date
        '
        Dim validTo As Date = variantObject

        Trace.WriteLine(String.Format("Not After = {0}", validTo.ToString))

        ' Free memory
        '
        VariantClear(variantObjectPtr)

        ' Free the memory block allocated to hold VARIANT 
        '
        Marshal.FreeHGlobal(variantObjectPtr)

    End If

    Trace.WriteLine("<< Notify")

End Sub

Note: Becareful when translating the sample to C#: "Dim certBytes(bstrLen - 1) As Byte" will translate to "byte[] rawCert = new byte[bstrLen];".

I've also seen this kind of issues when referencing CAPICOM.dll (Interop.CAPICOM.dll interop assembly). These errors may appear when we are i.e. decrypting a string:

System.Runtime.InteropServices.COMException: ASN1 unexpected end of data.
System.Runtime.InteropServices.COMException: ASN1 bad tag value met.

For instance, CAPICOM.Utilities.ByteArrayToBinaryString returns a BSTR that represents the binary data being passed as an input parameter. The interop layer assigns the BSTR to a .NET string which removes the last byte and makes the data even. By using an IntPtr as return type we can access the complete binary data as we've already seen. In this case we'll replace "string marhal(bstr)" to "native int" just before each ByteArrayToBinaryString occurrence in the IL file.

I hope this helps.

Cheers,

Alex (Alejandro Campos Magencio)

Credits: Sample code has been provided by my colleague Prabagar Ramadasse. Thank you Prabagar!