Larry Osterman's WebLog

Confessions of an Old Fogey
Blog - Title

Hey, why am I leaking all my BSTR's?

Hey, why am I leaking all my BSTR's?

Rate This
  • Comments 12

IMHO, every developer should have a recent copy of the debugging tools for windows package installed on their machine (it's updated regularly, so check to see if there's a newer version).

One of the most useful leak tracking tools around is a wonderfully cool tool that's included in this package, UMDH.  UMDH allows you to take a snapshot of the heaps in a process, and perform a diff of the heap over time - basically you run it once to take a snapshot, then run it a second time after running a particular test and it allows you to compare the differences in the heaps.

This tool can be unbelievably useful when debugging services, especially shared services.  The nice thing about it is that it provides a snapshot of the heap usage, there are often times when that's the only way to determine the cause of a memory leak.

As a simple example of this the Exchange 5.5 IMAP server cached user logons.  It did this for performance reasons, it could take up to five seconds for a call to LogonUser to complete, and that affected our ability to service large numbers of clients - all of the server threads ended up being blocked waiting on the domain controllers to respond.  So we put in a logon cache.  The cache took the users credentials, performed a LogonUser with those credentials, and put the results into a heap.  On subsequent logons, the cache took the users credentials, looked them up in the heap, and if they were found, it just reused the token from the cache (and no, it didn't do the lookup in clear text, I'm not that stupid).  Unfortunately, when I first wrote the cache implementation, I had an uninitialized variable in the hash function used to lookup the user in the cache, and as a result, every user logon occupied a different slot in the hash table.  As a result, when run over time, I had a hideous memory leak (hundreds of megabytes of VM).  But, since the cache was purged on exit, the built-in leak tracking logic in the Exchange store didn't detect any memory leaks. 

We didn't have UMDH at the time, but UMDH would have been a perfect solution to the problem.

I recently went on a tear trying to find memory leaks in some of the new functionality we've added to the Windows Audio Service, and used UMDH to try to catch them.

I found a bunch of the leaks, and fixed them, but one of the leaks I just couldn't figure out showed up every time we allocated a BSTR object.

It drove me up the wall trying to figure out how we were leaking BSTR objects, nothing I did found the silly things.  A bunch of the leaks were in objects allocated with CComBSTR, which really surprised me, since I couldn't see how on earth they would leak memory.

And then someone pointed me to this KB article (KB139071).  KB1239071 describes the OLE caching of BSTR objects.  It also turns out that this behavior is described right on the MSDN page for the string manipulation functions, proving once again that I should have looked at the documentation :).

Basically, OLE caches all BSTR objects allocated in a process to allow it to pool together strings.  As a result, these strings are effectively leaked “on purposeâ€.  The KB article indicates that the cache is cleared when the OLEAUT32.DLL's DLL_PROCESS_DETACH logic is run, which is good to know, but didn't help me to debug my BSTR leak - I could still be leaking BSTRs.

Fortunately, there's a way of disabling the BSTR caching, simply set the OANOCACHE environment variable to 1 before launching your application.  If your application is a service, then you need to set OANOCACHE as a system environment variable (the bottom set of environment variables) and reboot.

I did this and all of my memory leaks mysteriously vanished.  And there was much rejoicing.

 

  • Your apostrophes have been mangled again.
  • Actually WinDbg hasn't been updated in a few months now. Any idea if a new release is being postponed or what?
  • I was editing it when I saw your comment Carlos, thanks.

    Skywing, I know it's updated about quarterly, so... But I'm not sure what their release schedule is.

    There's nothing "postponed" as far as I know.
  • The UMDH/GFLAGS/DHCMP trio are so handy that I wrote a nice wrapper around them (before the UMDH and DHCMP functionality was merged). The first time we used it was to find a memory leak that turned out to be caused by one of our objects not calling DeleteCriticalSection.

    UMDH is great for those types of leaks.
  • UMDH has one big problem: it doesn't handle FPO optimized call stacks (technically, it's not UMDH's fault since the stack traces are collected by ntdll, UMDH just dumps them).

    Before XP SP2 this meant that any allocations using CRT or COM allocators(new/malloc/SysAllocString/CoTaskMemAlloc etc) would have incomplete call stacks which are totally useless for debugging.

    Fortunately, XP SP2 is recompiled without FPO so UMDH stack traces are now much more accurate.
  • In some of the projects I've worked on, the CComBSTR class has been responsible for at least 50% of all memory leaks.

    This is because unlike CComPtr, CComBSTR::operator& doesn't assert when the internal BSTR pointer is not NULL. So people would write code like this:

    CComBSTR bstr;
    pNode1->get_text (&bstr);
    ...
    pNode2->get_text (&bstr);

    and leak memory. I heard that this will finally be fixed in Whidbey.

    In the meantime, you can fix this yourself by adding ATLASSERT(m_bstr == NULL); to CComBSTR::operator&. Just like CComPtr::operator&, this can generate some false positives, but they can easily be worked around.
  • OOoh, good one Pavel, I hadn't even thought of that (although I've seen the problem in my code).
  • OOoh, good one Pavel, I hadn't even thought of that (although I've seen the problem in my code).

    My personal favorite is:

    CComPtr<IUnknown> foo;

    foo.CoCreateInstance(CLSID_Foo);

    foo.CoCreateInstance(CLSID_Bar);

    THis leaks a reference to the Foo object.
  • I prefer to use leakdiag coz it supports allocators other than the heap. And no gflags needed.
  • Larry: Do you have any documentation on how caching logon credentials should be done in a secure manner?
  • The debugging tools for Windows&amp;nbsp;comes with a great facility, umdh.exe, that makes tracking memory...
  • PingBack from http://www.keyongtech.com/2317476-bstr-memory-leak

Page 1 of 1 (12 items)