Welcome to MSDN Blogs Sign in | Join | Help

How does Raymond get rid of his excess pennies?

Commenter Boris mentions that he uses NJ Transit to get rid of his excess pennies. But what do you do if your area isn't served by NJ Transit?

I use the self-checkout line at the grocery store. The machine has a slot for accepting coins, and you can drop pennies in there until your arm falls off. I don't do this when the grocery store is crowded, since this holds up the line.

(Yes, banks also have change-counting machines, but using the machine is overkill when you have only thirty pennies to get rid of.)

Posted by oldnewthing | 20 Comments
Filed under:

Caches are nice, but they confuse memory leak detection tools

Knowledge Base article 139071 has the technically correct but easily misinterpreted title FIX: OLE Automation BSTR caching will cause memory leak sources in Windows 2000. The title is misleading because it makes you think that Oh, this is a fix for a memory leak in OLE Automation, but that's not what it is.

The BSTR is the string type used by OLE Automation, and since strings are used a lot, OLE Automation maintains a cache of recently-freed strings which it can re-use when somebody allocates a new one. Caches are nice (though you need to make sure you have a good replacement policy), but they confuse memory leak detection tools, because the memory leak detection tool will not be able to match up the allocator with the deallocator. What the memory leak detection tool sees is not the creation and freeing of strings but rather the allocation and deallocation of memory. And if there is a string cache (say, of just one entry, for simplicity), what the memory leak detection tool sees is only a part of the real story.

  • Program (line 1): Creates string 1.
  • String manager: Allocates memory block A for string 1.
  • Program (line 2): Frees string 1.
  • String manager: Puts memory block A into cache.
  • Program (line 3): Creates string 2.
  • String manager: Re-uses memory block A for string 2.
  • Program (line 4): Creates string 3.
  • String manager: Allocates memory block B for string 3.
  • Program (line 5): Frees string 3.
  • String manager: Puts memory block B into cache.
  • Program (line 6): Frees string 2.
  • String manager: Deallocates memory block A since there is no room in the cache.

Your program sees only the lines marked Program:, and the memory leak detection tool sees only the underlined part. As a result, the memory leak detection tool sees a warped view of the program's string usage:

  • Line 1 of your program allocates memory block A.
  • Line 4 of your program allocates memory block B.
  • Line 6 of your program deallocates memory block A.

Notice that the memory leak detection tool thinks that line 6 freed the memory allocated by line 1, even though the two lines of the program are unrelated. Line 6 is freeing string 2, and line 1 is creating string 1!

Notice also that the memory leak detection tool will report a memory leak, because it sees that you allocated two memory blocks but deallocated only one of them. The memory leak detection tool will say, "Memory allocated at line 4 is never freed." And you stare at line 4 of your program and insist that the memory leak detection tool is on crack because there, you freed it right at the very next line! You chalk this up as "Stupid memory leak detection tool, it has all these useless false positives."

Even worse: Suppose somebody deletes line 6 of your program, thereby introducing a genuine memory leak. Now the memory leak detection tool will report two leaks:

  • Memory allocated at line 1 is never freed.
  • Memory allocated at line 4 is never freed.

You already marked the second report as bogus during your last round of investigation. Now you look at the first report, and decide that it too is bogus; I mean look, we free the string right there at line 2!

Result: A memory leak is introduced, the memory leak detection tool finds it, but you discard it as another bug in the memory leak detection tool.

When you're doing memory leak detection, it helps to disable your caches. That way, the high-level object creation and destruction performed in your program maps more directly to the low-level memory allocation and deallocation functions tracked by the memory leak detection tool. In our example, if there were no cache, then every Create string would map directly to an Allocate memory call, and every Free string would map directly to a Deallocate memory call.

What KB article 139071 is trying to say is FIX: OLE Automation BSTR cache cannot be disabled in Windows 2000. Windows XP already contains support for the OANOCACHE environment variable, which disables the BSTR cache so you can investigate those BSTR leaks more effectively. The hotfix adds support for OANOCACHE to Windows 2000.

Bonus chatter: Why do we have BSTR anyway? Why not just use null-terminated strings everywhere?

The BSTR data type was introduced by Visual Basic. They couldn't use null-terminated strings because Basic permits nulls to be embedded in strings. Whereas Win32 is based on the K&R C  way of doing things, OLE automation is based on the Basic way of doing things.

Posted by oldnewthing | 15 Comments
Filed under:

I want to take all your chocolate milk

My older niece visited me at work one day, and I got her a carton of chocolate milk, which she very much enjoyed. Some days later, she told me, "I want to go to your work."

"Why?" I asked.

"I want to take all your chocolate milk."

Missing from the story is that upon returning home after that first visit, she told everybody about her awesome visit with her uncle, and that he even got her a chocolate milk from the refrigerator. "And the chocolate milk is free, you can just take it!"

Her uncle (not me, a different uncle) told her, "Then you should go there with a knapsack and take all the chocolate milk."

That uncle is clearly a troublemaker. I'll have to keep an eye on him.

Posted by oldnewthing | 16 Comments
Filed under:

When you want to copy a file into a folder, make sure you have a folder

This story is inspired by an actual customer problem.

The program LitWare.exe is used for TPS management, and when you want to create a new TPS report, you have to pick a cover sheet. The program shows you the cover sheets that have been defined, which it loads from the C:\TPS Cover Sheets directory.

The customer found that on one of the machines, no of the cover sheets were showing up, even though the standard system setup copies a sample cover sheet into the C:\TPS Cover Sheets directory. The error message they got was Cannot load cover sheets. The directory name is invalid.

The customer did some troubleshooting and determined that "The cover sheet directory is missing, and we have a file instead."

C:\>dir
 Volume in drive C is INITECH
 Volume Serial Number is BAAD-F00D

 Directory of C:\

09/18/2006  02:43 PM                24 autoexec.bat
09/18/2006  02:43 PM                10 config.sys
03/18/2009  10:30 AM    <DIR>          Program Files
11/21/2008  01:04 PM             1,677 TPS Cover Sheets
02/20/2008  10:39 AM    <DIR>          Users
05/29/2009  02:23 PM    <DIR>          Windows
               2 File(s)          1,711 bytes
               3 Dir(s)  229,031,751,680 bytes free

One of my colleagues employed psychic powers to determine that at the time the customer tried to install the sample cover sheet on the machine, the C:\TPS Cover Sheets directory did not yet exist, and that the batch file they used to set up a new computer just does a copy \\server\TPSConfig\Sample.tps "C:\TPS Cover Sheets", which results in a file being created with the name C:\TPS Cover Sheets.

The customer was surprised by this conclusion. "I would think that copy will fail if the C:\TPS Cover Sheets directory doesn't exist, but this might be our problem. We'll look into it." (I guess this customer never used the copy command to copy a file to a new name.)

If the destination of a copy command exists and is a directory, then the source files are copied into that directory. If the destination of a copy command does not exists or if it exists and is a file, then the destination is treated as a file name for the destination. (If there is more than one source file, then they are concatenated as if they were text files.)

The customer went back and checked the scripts, and the line they used was almost exactly what my colleague predicted:

copy "\\INITECH\Defaults\Sample cover sheet.tps" "C:\TPS Cover Sheets" /Y

If the C:\TPS Cover Sheets directory hasn't been created yet, then that would explain the behavior they're seeing: The copy command sees that the destination doesn't exist and assumes you are doing a file-to-file copy (as opposed to a file-to-directory copy). In this case, the problem was that copying a sample cover sheet was a step they added to their setup scripts, but they added it before the step that creates the cover sheet directory. Reordering the two steps fixed the problem.

Posted by oldnewthing | 16 Comments
Filed under:

The magic of chocolate milk

While enjoying a meal with my nieces (at the time, ages 3 and 5), I diluted my chocolate milk to cut the sweetness. The nieces then demanded that I dilute their chocolate milk as well, because as far as they could determine, it was a magical way to create more chocolate milk.

Posted by oldnewthing | 15 Comments
Filed under:

How do I get the command line of another process?

Win32 doesn't expose a process's command line to other processes. From Win32's point of view, the command line is just a conveniently initialized parameter to the process's startup code, some data copied from the launching process to the new process and forgotten. We'll get back to the Win32 point of view a little later.

If you look around in WMI, you'll find a Win32_Process object, and lo and behold, it has a CommandLine property. Let's check it out, using the standard WMI application:

strComputer = "."
Set objWMIService = GetObject("winmgmts:\\" & strComputer & "\root\cimv2")
Set colItems = objWMIService.ExecQuery("Select * from Win32_Process")
For Each objItem in colItems
     Wscript.Echo objItem.Name
     Wscript.Echo objItem.CommandLine
Next

I fully anticipate that half of my readers will stop right there. "Thanks for the script. Bye!" And they won't bother reading the analysis. "Because analysis is boring, and it'll just tell me stuff I don't want to hear. The analysis is going to tell me why this won't work, or why it's a bad idea, and that just cramps my style."

Remember that from Win32's point of view, the command line is just a string that is copied into the address space of the new process. How the launching process and the new process interpret this string is governed not by rules but by convention.

What's more, since the string is merely a "preinitialized variable", a process could in principle (and many do in practice, although usually inadvertently) write to the memory that holds the command line, in which case, if you go snooping around for it, you'll see the modified command line. There is no secret hiding place where the kernel keeps the "real original command line," any more than there is a secret hiding place where the C compiler keeps the "real original parameters to a function."

This is just another manifestation of the principle of not keeping track of information you don't need.

What does this mean for people who disregard this principle and go after the command line of another process? You have to understand what you are getting is non-authoritative information. In fact, it's worse. It's information the application itself may have changed in order to try to fool you, so don't use it to make important decisions.

Posted by oldnewthing | 28 Comments
Filed under:

When computer programmers dabble in making change

My colleague who dabbled in economics when deciding how many lunch vouchers to buy had a number of other money-related quirks.

One of the ones that I remember is that when paying for a purchase, my colleague would double the balance and give the cashier that much money. For example, if the total was $5.20, my colleague would hand over $10.40.

Why?

Just to see if the cashier reacted when pressing the Enter code appeared to have no effect.

Total is $5.20.

Cash tendered is $10.40.

Change is $5.20.

Most of the time, the cashier wouldn't pay any attention. Heck, the cashier wouldn't even question why my colleague handed over such a strange amount of money.

Sometimes my colleague would mix it up and instead add $6.66 to the total. For example, if the total was $5.20, my colleague would hand over $11.86, just to see the cashier's reaction when the cash register indicated that the change due was $6.66.

And then one day, magic happened: The total was $6.66. Without skipping a beat, my colleague handed over $13.32.

Posted by oldnewthing | 51 Comments
Filed under:

Can I talk to that William fellow? He was so helpful

Today we're going to take a little trip in the wayback machine with the help of my colleague Seth Manheim, who was there when this happened.

Set the date to November 22, 1989, twenty years ago and one day. Bill Gates is being taken on a guided tour of the product support department's new office building, and during his visit, he asks one of the people manning the phones, "Mind if I take this call?"

Bill puts on a headset, sits down, and answers the phone. "Hello, this is Microsoft Product Support, William speaking. How can I help you?"

Bill talks with the customer, collects the details of the problem, searches in the product support Knowledge Base, sifts through the search results, finds the solution, and patiently walks the customer through fixing the problem.

The customer is thrilled that William was able to fix the problem so quickly, and with such a pleasant attitude. Bill wraps up the call. "And thank you for using Microsoft products."

At no point did Bill identify himself as anything other than William. The customer had no idea that the product support engineer who took the call was none other than Bill Gates.

But the story doesn't end there.

Even though this story took place while most of the support staff were on their lunch break, news travels quickly, and soon everybody in the department knows about The time Bill took a product support call.

Some time later, the same customer calls back with a follow-up question.

Hi, I called you folks with a problem with XYZ, and I talked with a nice man named William who straightened it all out. But I have another question. Can I speak with William?

"Okay, let me see if William is available." The product support engineer brings up the customer's service record and looks at the name of the support engineer who handled the earlier call: billg.

"Yeah, um, I'm sorry, but William is not available right now. His friends call him Bill, by the way. The person who helped you last time? That was Bill Gates."

Oh my God.

While I'm tinkering with the wayback machine, I may as well point you to a story from a few years ago with a similar (but less dramatic) punch line.

Posted by oldnewthing | 24 Comments
Filed under:

The difference between assignment and attachment with ATL smart pointers

Last time, I presented a puzzle regarding a memory leak. Here's the relevant code fragment:

CComPtr<IStream> pMemoryStream;
CComPtr<IXmlReader> pReader;
UINT nDepth = 0;

//Open read-only input stream
pMemoryStream = ::SHCreateMemStream(utf8Xml, cbUtf8Xml);

The problem here is assigning the return value of SHCreateMemStream to a smart pointer instead of attaching it.

The SHCreateMemStream function creates a memory stream and returns a pointer to it. That pointer has a reference count of one, in accordance with COM rules that a function which produces a reference calls AddRef, and the responsibility is placed upon the recipient to call Release. The assignment operator for CComPtr<T> is a copy operation: It AddRefs the pointer and saves it. You're still on the hook for the reference count of the original pointer.

ATLINLINE ATLAPI_(IUnknown*) AtlComPtrAssign(IUnknown** pp, IUnknown* lp)
{
        if (lp != NULL)
                lp->AddRef();
        if (*pp)
                (*pp)->Release();
        *pp = lp;
        return lp;
}

template <class T>
class CComPtr
{
public:
        ...

        T* operator=(T* lp)
        {
                return (T*)AtlComPtrAssign((IUnknown**)&p, lp);
        }

Observe that assigning a T* to a CComPtr<T> AddRefs the incoming pointer and Releases the old pointer (if any). When the CComPtr<T> is destructed, it will release the pointer, undoing the AddRef that was performed by the assignment operator. In other words, assignment followed by destruction has a net effect of zero on the pointer you assigned. The operation behaves like a copy.

Another way of putting a pointer into a CComPtr<T> is with the Attach operator. This is a transfer operation:

        void Attach(T* p2)
        {
                if (p)
                        p->Release();
                p = p2;
        }

Observe that there is no AddRef here. When the CComPtr<T> is destructed, it will perform the Release, which doesn't undo any operation performed by the Attach. Instead, it releases the reference count held by the original pointer you attached.

Let's put this in a table, since people seem to like tables:

Operation Behavior Semantics
Attach() Takes ownership Transfer semantics
operator=() Creates a new reference Copy semantics

You use the Attach method when you want to assume responsibility for releasing the pointer (ownership transfer). You use the assignment operator when you want the original pointer to continue to be responsible for its own release (no ownership transfer).

There is also a Detach method which is the opposite of Attach: Detaching a pointer from the CComPtr<T> means "I am taking over responsibility for releasing this pointer." The CComPtr<T> gives you its pointer and then forgets about it; you're now on your own.

The memory leak in the code fragment above occurs because the assignment operator has copy semantics, but we wanted transfer semantics, since we want the smart pointer to take the responsibility for releasing the pointer when it is destructed.

pMemoryStream.Attach(::SHCreateMemStream(utf8Xml, cbUtf8Xml));

The CComPtr<T>::operator=(T*) method is definitely one of the more dangerous methods in the CComPtr<T> repertoire, because it's so easy to assign a pointer to a smart pointer without giving it a moment's thought. (Another dangerous method is the T** CComPtr<T>::operator&(), but at least that has an assertion to try to catch the bad usages. Even nastier is the secret QI'ing assignment operator.) I have to say that there is merit to Ben Hutchings' recommendation simply not to allow a simple pointer to be assigned to a smart pointer, precisely because the semantics are easily misunderstood. (The boost library, for example, follows Ben's recommendation.)

Here's another exercise based on what you've learned:

Application Verifier told us that we have a memory leak, and we traced it back to the function GetTextAsInteger.

BSTR GetInnerText(IXMLDOMNode *node)
{
    BSTR bstrText = NULL;
    node->get_text(&bstrText);
    return bstrText;
}

DWORD GetTextAsInteger(IXMLDOMNode *node)
{
    DWORD value = 0;

    CComVariant innerText = GetInnerText(node);
    hr = VariantChangeType(&innerText, &innerText, 0, VT_UI4);
    if (SUCCEEDED(hr))
    {
        value = V_UI4(&innerText);
    }

    return value;
}

Obviously, the problem is that we passed the same input and output pointers to VariantChangeType, causing the output integer to overwrite the input BSTR, resulting in the leak of the BSTR. But when we fixed the function, we still got the leak:

DWORD GetTextAsInteger(IXMLDOMNode *node)
{
    DWORD value = 0;

    CComVariant innerText = GetInnerText(node);
    CComVariant textAsValue;
    hr = VariantChangeType(&innerText, &textAsValue, 0, VT_UI4);
    if (SUCCEEDED(hr))
    {
        value = V_UI4(&textAsValue);
    }

    return value;
}

Is there a leak in the VariantChangeType function itself?

Hint: It is in fact explicitly documented that the output parameter to VariantChangeType can be equal to the input parameter, which results in an in-place conversion. There was nothing wrong with the original call to VariantChangeType.

Posted by oldnewthing | 15 Comments
Filed under:

We're using a smart pointer, so we can't possibly be the source of the leak

A customer reported that there was a leak in the shell, and they included the output from Application Verifier as proof. And yup, the memory that was leaked was in fact allocated by the shell:

VERIFIER STOP 00000900 : pid 0x3A4: A heap allocation was leaked.

        497D0FC0 : Address of the leaked allocation.
        002DB580 : Adress to the allocation stack trace.
        0D65CFE8 : Address of the owner dll name.
        6F560000 : Base of the owner dll.

1: kd> du 0D65CFE8
0d65cfe8  "SHLWAPI.dll"

1: kd> !heap -p -a 497D0FC0
...
    ntdll!RtlpAllocateHeap+0x0003f236
    ntdll!RtlAllocateHeap+0x0000014f
    Kernel32!LocalAlloc+0x0000007c
    shlwapi!CreateMemStreamEx+0x00000043
    shlwapi!CreateMemStream+0x00000012
    <Unloaded_xyz.dll>+0x000642de
    <Unloaded_xyz.dll>+0x0005e2af
    <Unloaded_xyz.dll>+0x0002d49a
    <Unloaded_xyz.dll>+0x0002a0fd
    <Unloaded_xyz.dll>+0x000289cb
    <Unloaded_xyz.dll>+0x0002a25c
    <Unloaded_xyz.dll>+0x00027225
    <Unloaded_xyz.dll>+0x0002252b
    <Unloaded_xyz.dll>+0x00025394
    <Unloaded_xyz.dll>+0x0004d70f
    Kernel32!BaseThreadInitThunk+0x0000000d
    ntdll!RtlUserThreadStart+0x0000001d

1: kd> dps 002DB580
shlwapi!CreateMemStreamEx+0x43
shlwapi!CreateMemStream+0x12
<Unloaded_xyz.dll>+0x642de
<Unloaded_xyz.dll>+0x5e2af
<Unloaded_xyz.dll>+0x2d49a
<Unloaded_xyz.dll>+0x2a0fd
<Unloaded_xyz.dll>+0x289cb
<Unloaded_xyz.dll>+0x2a25c
<Unloaded_xyz.dll>+0x27225
<Unloaded_xyz.dll>+0x2252b
<Unloaded_xyz.dll>+0x25394
<Unloaded_xyz.dll>+0x4d70f
Kernel32!BaseThreadInitThunk+0xd
ntdll!RtlUserThreadStart+0x1d

On the other hand, SHCreateMemStream is an object creation function, so it's natural that the function allocate some memory. The responsibility for freeing the memory belongs to the caller.

We suggested that the customer appears to have leaked the interface pointer. Perhaps there's a hole where they called AddRef and managed to avoid the matching Release.

"Oh no," the customer replied, "that's not possible. We call this function in only one place, and we use a smart pointer, so a leak is impossible." The customer was kind enough to include a code snippet and even highlighted the lines that proved they weren't leaking.

CComPtr<IStream> pMemoryStream;
CComPtr<IXmlReader> pReader;
UINT nDepth = 0;

//Open read-only input stream
pMemoryStream = ::SHCreateMemStream(utf8Xml, cbUtf8Xml);

The exercise for today is to identify the irony in the highlighted lines.

Hint. Answers (and more discussion) tomorrow.

Posted by oldnewthing | 29 Comments
Filed under:

News flash: Healthy people live longer

Researchers have determined that people in good physical condition live longer. Who'd'a thunk it?

Posted by oldnewthing | 15 Comments
Filed under:

How do I move the Windows.edb and other search index files?

Nothing profound today, just a little tip.

My customer is looking out for a way to change the location of the windows.edb file to another (larger) drive.

From the Indexing Options Control Panel, click Advanced, and then under Index location, click Select new.

Posted by oldnewthing | 7 Comments
Filed under:

We found the author of Notepad, sorry you didn't go to the award ceremony

I've received independent confirmations as to the authorship of Notepad, so I'm inclined to believe it. Sorry you didn't get to go to the award ceremony.

The original author of Notepad also served as the development manager for Windows 95. His job was to herd the cats that made up the programmers who worked on Windows 95, a job which you can imagine falls into the "not easy" category.

After Windows 95, he retired from the software industry and became a high school science teacher. At a social event some years later, I met him again and asked about the transition from software development manager to high school science teacher.

His response: "You'd be surprised how many of the skills transfer."

Posted by oldnewthing | 34 Comments
Filed under:

How to tell when your patent has been approved

There are a variety of ways of finding out when your patent is granted, but the quickest mechanism is to check your mailbox. But the thing to look for is not what you might think.

Even before you receive word from your company's patent department, you will start receiving junk mail delivered to your home address from companies that sell patent-related novelties, pointless trinkets like pencils and mugs with your patent number on it. Nevermind that you can probably order personalized pencils for much less than what the patent-chasers were offering.

Posted by oldnewthing | 10 Comments
Filed under:

How to pretend that you attended my talk at UIUC Reflections|Projections 2009

Step 1: Buy a 1.55-ounce Hershey's Milk Chocolate Bar from a convenience store, supermarket, or (if truly desperate) online.

Step 2: Print out this candy bar wrapper.

Step 3: Trim wrapper on registration marks and wrap around candy bar.

Step 4: Stay up late the night before you plan on watching the video by partying with Ryan North and teaching him how to play beer pong.

Step 5: Force yourself to wake up the next morning and watch the recorded video of my talk while trying desperately to stay awake. The candy bar might help.

Note: Although most steps are optional, they are essential if you want an accurate simulation.

Posted by oldnewthing | 12 Comments
Filed under:
More Posts Next page »
 
Page view tracker