• #### Suggestion Box 3

Post suggestions for future topics here instead of posting off-topic comments. Note that the suggestion box is emptied and read periodically so don't be surprised if your suggestion vanishes. (Note also that I am under no obligation to accept any suggestion.)

Topics I are more inclined to cover:

• Windows history (particularly the Windows 95 era).
• Windows user interface programming in Win32, and shell programming in particular.
• General programming topics (selectively).
• Issues of general interest.
• My personal hobbies.

Topics I am not inclined to cover:

• The blog software itself. You can visit the Community Server home page and cruise their support forums.
• Internet Explorer. You can try the IE folks.
• Visual Studio. You can try one of the Visual Studio blogs.
• Managed code. This is not a .NET blog. I do not work on .NET technologies. As far as .NET is concerned, I'm just another programmer like you. Occasionally I touch a .NET-related topic, but I do not bring any expertise to the subject.
• Non-software Microsoft topics, such as product support policies, marketing tactics, jobs and careers, legal issues.
• Microsoft software that isn't Windows. (Exchange, Office, ...)
• Windows topics outside user interface programming. (Plug and Play, Terminal Services, Windows Messenger, Outlook Express, SQL, IIS, remoting, SOA...)
• User interface programming in anything other than Win32. (Because I know nothing about it.)
• Debugging a specific problem. (Not of general interest.)
• Predictions for the future. (What's the title of this blog again?)
• Participation in Internet memes.

You can also send feedback on Microsoft products directly to Microsoft. All the feedback gets read, even the death threats.

Suggestions should be between two and four sentences in length. As you can see, there are hundreds of them already, so you have three seconds to get your point across. Please also search the blog first because your suggestion may have already been covered. And remember, questions aren't suggestions.

Note the enormous topic backlog. Consequently, the suggestion box has been closed temporarily and will reopen once the existing backlog has cleared, which I estimate will happen sometime in early 2010. If your suggestion is that important, I'm sure you'll remember it when the suggestion box reopens.

• #### Capturing the current directory from a batch file

Sometimes people go to great lengths to get information which is available in a much simpler way. We saw it a few days ago when we found a 200+-line C# program that could be replaced with a 90-byte batch file. Here's another example of a rather roundabout way of capturing the current directory from a batch file.

The easy way is to use the %CD% pseudo-variable. It expands to the current working directory.

set OLDDIR=%CD%
.. do stuff ..
chdir /d %OLDDIR% &rem restore current directory


(Of course, directory save/restore could more easily have been done with pushd/popd, but that's not the point here.)

The %CD% trick is handy even from the command line. For example, I often find myself in a directory where there's a file that I want to operate on but... oh, I need to chdir to some other directory in order to perform that operation.

set _=%CD%\curfile.txt
cd ... some other directory ...
somecommand args %_% args


(I like to use %_% as my scratch environment variable.)

Type SET /? to see the other pseudo-variables provided by the command processor.

• #### What does the COM Surrogate do and why does it always stop working?

The dllhost.exe process goes by the name COM Surrogate and the only time you're likely even to notice its existence is when it crashes and you get the message COM Surrogate has stopped working. What is this COM Surrogate and why does it keep crashing?

The COM Surrogate is a fancy name for Sacrificial process for a COM object that is run outside of the process that requested it. Explorer uses the COM Surrogate when extracting thumbnails, for example. If you go to a folder with thumbnails enabled, Explorer will fire off a COM Surrogate and use it to compute the thumbnails for the documents in the folder. It does this because Explorer has learned not to trust thumbnail extractors; they have a poor track record for stability. Explorer has decided to absorb the performance penalty in exchange for the improved reliability resulting in moving these dodgy bits of code out of the main Explorer process. When the thumbnail extractor crashes, the crash destroys the COM Surrogate process instead of Explorer.

In other words, the COM Surrogate is the I don't feel good about this code, so I'm going to ask COM to host it in another process. That way, if it crashes, it's the COM Surrogate sacrificial process that crashes instead of me process. And when it crashes, it just means that Explorer's worst fears were realized.

In practice, if you get these types of crashes when browsing folders containing video or media files, the problem is most likely a flaky codec.

Now that you know what the COM Surrogate does, you can answer this question from a customer:

I'm trying to delete a file, but I'm told that "The action can't be completed because the file is open in COM Surrogate." What is going on?
• #### What's the deal with the System Volume Information folder?

In the root of every drive is a folder called "System Volume Information". If your drive is NTFS, the permissions on the folder are set so not even administrators can get in there. What's the big secret?

The folder contains information that casual interference could cause problems with proper system functioning. Here are some of the things kept in that folder. (This list is not comprehensive.)

• System Restore points. You can disable System Restore from the "System" control panel.
• Content Indexing Service databases for fast file searches. This is also the source of the cidaemon.exe process: That is the content indexer itself, busy scanning your files and building its database so you can search for them quickly. (If you created a lot of data in a short time, the content indexer service gets all excited trying to index it.)
• Information used by the Volume Snapshot Service (also known as "Volume Shadow Copy") so you can back up files on a live system.
• Longhorn systems keep WinFS databases here.
• #### How does the calculator percent key work?

The Windows calculator percent sign works the same way as those cheap pocket calculators (which are often called four-function calculators even though they have around six function nowadays). What you first have to understand is that the percent key on those pocket calculators was not designed for mathematicians and engineers. It was designed for your everyday person doing some simple calculations. Therefore, the behavior of the key to you, an engineer, seems bizarrely counter-intuitive and even buggy. But to an everyday person, it makes perfect sense. Or at least that's the theory.

Let's look at it from the point of view of that everyday person. Suppose you want to compute how much a $72 sweater will cost after including 5% tax.¹ Pull out your handy pocket calculator² (or fire up Calc if you don't have a pocket calculator) and type 72 + 5% = The result is 75.6, or$75.60, which is the correct answer, because 5% of 72 is 3.6. Add that to 72 and you get 75.6.

Similarly, suppose that sweater was on sale at 20% off. What is the sale price?

72 − 20% =

The result is 57.6 or \$57.60. This is the correct answer, because 20% of 72 is 14.4. Subtract that from 72 and you get 57.6.

You can chain these percentage operations, too. For example, how much will you have to pay for that 20%-off sweater after adding 5% tax?

72 − 20% + 5% =

The result is 60.48. A mathematician or engineer would have calculated the same result via the equivalent computation:

72 × 0.80 × 1.05 =

Okay, now that we see how the calculator product designer intended the percent key to be used, let's look at what the calculator engineer it has to do in order to match the specification. When the user enters A + B % =, the result should be A × (1 + B/100) or A + (A × B/100) after you distribute the multiplication over the addition. Similarly, when the user enters A − B % =, the result should be A × (1 − B/100) or A − (A × B/100).

Aha, the calculator engineer says, we can achieve this result by defining the percent key as follows:

When the user enters a value, an operator, a second value, and then the percent key, the first two values are multiplied and the product divided by 100, and that result replaces the second value in the ongoing computation.

Let's walk through that algorithm with our first example.

You typeRemarks
72First value is 72
5Second value is 5
%72 × 5 ÷ 100 = 3.6
3.6 becomes the new second value
=72 + 3.6 = 75.6, the final result

If you watch the display as you go through this exercise, you will even see the number 3.6 appear in the display once you press the % key. The percentage is calculated and replaces the original value in the ongoing computation.

This algorithm also works for the chained percentages.

You typeRemarks
72First value is 72
Operation is subtraction
20Second value is 20
%72 × 20 ÷ 100 = 14.4
14.4 becomes the new second value
+72 − 14.4 = 57.6, intermediate result
57.6 is the new first value
5Second value is 5
%57.6 × 5 ÷ 100 = 2.88
2.88 becomes the new second value
=57.6 + 2.88 = 60.48, the final result

This even works for multiplication and division, but there is much less call for multiplying or dividing a number by a percentage of itself.

500 × 5 % =

The result of this is 12,500 because you are multiplying 500 by 5% of 500 (which is 25). The result of 500 × 25 is 12,500. You aren't computing five percent of 500. You're multiplying 500 by 5% of 500. (It appears that the authors of this Knowledge Base article didn't consult with the calculator engineer before writing up their analysis. The percent key is behaving as designed. The problem is that the percent key is not designed for engineers.)

What if you want to compute 5% of 500? Just pick a dummy operation and view the result when you press the percent key.

500 + 5 %

When you hit the percent key, the answer appears: 25. You could've used the minus key, multiplication key, or division key instead of the addition key. It doesn't matter since all you care about is the percentage, not the combined operation. Once you hit the % key, you get your answer, and then you can hit Clear to start a new calculation.

Footnotes

¹In the United States, quoted prices typically do not include applicable taxes.

²In my limited experiments, it appears that no two manufacturers of pocket calculators handle the percent key in exactly the same way. Casio appears to handle it in a manner closest to the engineering way. TI is closer to the layman algorithm. And when you get into cases like 1 ÷ 2 %, calculators start wandering all over the map. Should the answer be 50, since 1/2 is equal to 50%? Or should it be 0.005 since that is the numeric value of 0.5%? Should that answer appear immediately or should it wait for you to hit the equals sign? I don't know what the intuitive result should be either.

• #### Why does Windows not recognize my USB device as the same device if I plug it into a different port?

You may have noticed that if you take a USB device and plug it into your computer, Windows recognizes it and configures it. Then if you unplug it and replug it into a different USB port, Windows gets a bout of amnesia and thinks that it's a completely different device instead of using the settings that applied when you plugged it in last time. Why is that?

The USB device people explained that this happens when the device lacks a USB serial number.

Serial numbers are optional on USB devices. If the device has one, then Windows recognizes the device no matter which USB port you plug it into. But if it doesn't have a serial number, then Windows treats each appearance on a different USB port as if it were a new device.

(I remember that one major manufacturer of USB devices didn't quite understand how serial numbers worked. They gave all of their devices serial numbers, that's great, but they all got the same serial number. Exciting things happened if you plugged two of their devices into a computer at the same time.)

But why does Windows treat it as a different device if it lacks a serial number and shows up on a different port? Why can't it just say, "Oh, there you are, over there on another port."

Because that creates random behavior once you plug in two such devices. Depending on the order in which the devices get enumerated by Plug and Play, the two sets of settings would get assigned seemingly randomly at each boot. Today the settings match up one way, but tomorrow when the devices are enumerated in the other order, the settings are swapped. (You get similarly baffling behavior if you plug in the devices in different order.)

In other words: Things suck because (1) things were already in bad shape—this would not have been a problem if the device had a proper serial number—and (2) once you're in this bad state, the alternative sucks more. The USB stack is just trying to make the best of a bad situation without making it any worse.

• #### Everybody thinks about garbage collection the wrong way

Welcome to CLR Week 2010. This year, CLR Week is going to be more philosophical than usual.

When you ask somebody what garbage collection is, the answer you get is probably going to be something along the lines of "Garbage collection is when the operating environment automatically reclaims memory that is no longer being used by the program. It does this by tracing memory starting from roots to identify which objects are accessible."

This description confuses the mechanism with the goal. It's like saying the job of a firefighter is "driving a red truck and spraying water." That's a description of what a firefighter does, but it misses the point of the job (namely, putting out fires and, more generally, fire safety).

Garbage collection is simulating a computer with an infinite amount of memory. The rest is mechanism. And naturally, the mechanism is "reclaiming memory that the program wouldn't notice went missing." It's one giant application of the as-if rule.¹

Now, with this view of the true definition of garbage collection, one result immediately follows:

If the amount of RAM available to the runtime is greater than the amount of memory required by a program, then a memory manager which employs the null garbage collector (which never collects anything) is a valid memory manager.

This is true because the memory manager can just allocate more RAM whenever the program needs it, and by assumption, this allocation will always succeed. A computer with more RAM than the memory requirements of a program has effectively infinite RAM, and therefore no simulation is needed.

Sure, the statement may be obvious, but it's also useful, because the null garbage collector is both very easy to analyze yet very different from garbage collectors you're more accustomed to seeing. You can therefore use it to produce results like this:

A correctly-written program cannot assume that finalizers will ever run at any point prior to program termination.

The proof of this is simple: Run the program on a machine with more RAM than the amount of memory required by program. Under these circumstances, the null garbage collector is a valid garbage collector, and the null garbage collector never runs finalizers since it never collects anything.

Garbage collection simulates infinite memory, but there are things you can do even if you have infinite memory that have visible effects on other programs (and possibly even on your program). If you open a file in exclusive mode, then the file will not be accessible to other programs (or even to other parts of your own program) until you close it. A connection that you open to a SQL server consumes resources in the server until you close it. Have too many of these connections outstanding, and you may run into a connection limit which blocks further connections. If you don't explicitly close these resources, then when your program is run on a machine with "infinite" memory, those resources will accumulate and never be released.

What this means for you: Your programs cannot rely on finalizers keeping things tidy. Finalizers are a safety net, not a primary means for resource reclamation. When you are finished with a resource, you need to release it by calling Close or Disconnect or whatever cleanup method is available on the object. (The IDisposable interface codifies this convention.)

Furthermore, it turns out that not only can a correctly-written program not assume that finalizers will run during the execution of a program, it cannot even assume that finalizers will run when the program terminates: Although the .NET Framework will try to run them all, a bad finalizer will cause the .NET Framework to give up and abandon running finalizers. This can happen through no fault of your own: There might be a handle to a network resource that the finalizer is trying to release, but network connectivity problems result in the operation taking longer than two seconds, at which point the .NET Framework will just terminate the process. Therefore, the above result can be strengthened in the specific case of the .NET Framework:

A correctly-written program cannot assume that finalizers will ever run.

Armed with this knowledge, you can solve this customer's problem. (Confusing terminology is preserved from the original.)

I have a class that uses Xml­Document. After the class is out of scope, I want to delete the file, but I get the exception System.IO.Exception: The process cannot access the file 'C:\path\to\file.xml' because it is being used by another process. Once the progam exits, then the lock goes away. Is there any way to avoid locking the file?

This follow-up might or might not help:

A colleague suggested setting the Xml­Document variables to null when we're done with them, but shouldn't leaving the class scope have the same behavior?

Bonus chatter: Finalizers are weird, since they operate "behind the GC." There are also lots of classes which operate "at the GC level", such as Weak­Reference GC­Handle and of course System.GC itself. Using these classes properly requires understanding how they interact with the GC. We'll see more on this later.

Unrelated reading: Precedence vs. Associativity Vs. Order.

Footnote

¹ Note that by definition, the simulation extends only to garbage-collected resources. If your program allocates external resources those external resources continue to remain subject to whatever rules apply to them.

• #### Does Windows have a limit of 2000 threads per process?

Often I see people asking why they can't create more than around 2000 threads in a process. The reason is not that there is any particular limit inherent in Windows. Rather, the programmer failed to take into account the amount of address space each thread uses.

A thread consists of some memory in kernel mode (kernel stacks and object management), some memory in user mode (the thread environment block, thread-local storage, that sort of thing), plus its stack. (Or stacks if you're on an Itanium system.)

Usually, the limiting factor is the stack size.

#include <stdio.h>
#include <windows.h>

{
Sleep(INFINITE);
return 0;
}

int __cdecl main(int argc, const char* argv[])
{
int i;
for (i = 0; i < 100000; i++) {
DWORD id;
if (!h) break;
CloseHandle(h);
}
return 0;
}


This program will typically print a value around 2000 for the number of threads.

Why does it give up at around 2000?

Because the default stack size assigned by the linker is 1MB, and 2000 stacks times 1MB per stack equals around 2GB, which is how much address space is available to user-mode programs.

You can try to squeeze more threads into your process by reducing your stack size, which can be done either by tweaking linker options or manually overriding the stack size passed to the CreateThread functions as described in MSDN.

  HANDLE h = CreateThread(NULL, 4096, ThreadProc, NULL,
STACK_SIZE_PARAM_IS_A_RESERVATION, &id);


With this change, I was able to squeak in around 13000 threads. While that's certainly better than 2000, it's short of the naive expectation of 500,000 threads. (A thread is using 4KB of stack in 2GB address space.) But you're forgetting the other overhead. Address space allocation granularity is 64KB, so each thread's stack occupies 64KB of address space even if only 4KB of it is used. Plus of course you don't have free reign over all 2GB of the address space; there are system DLLs and other things occupying it.

But the real question that is raised whenever somebody asks, "What's the maximum number of threads that a process can create?" is "Why are you creating so many threads that this even becomes an issue?"

The "one thread per client" model is well-known not to scale beyond a dozen clients or so. If you're going to be handling more than that many clients simultaneously, you should move to a model where instead of dedicating a thread to a client, you instead allocate an object. (Someday I'll muse on the duality between threads and objects.) Windows provides I/O completion ports and a thread pool to help you convert from a thread-based model to a work-item-based model.

Note that fibers do not help much here, because a fiber has a stack, and it is the address space required by the stack that is the limiting factor nearly all of the time.

• #### On 64-bit Windows, 32-bit programs run in an emulation layer, and if you don't like that, then don't use the emulator

On 64-bit Windows, 32-bit programs run in an emulation layer. This emulation layer simulates the x86 architecture, virtualizing the CPU, the file system, the registry, the environment variables, the system information functions, all that stuff. If a 32-bit program tries to look at the system, it will see a 32-bit system. For example, if the program calls the GetSystemInfo function to see what processor is running, it will be told that it's running on a 32-bit processor, with a 32-bit address space, in a world with a 32-bit sky and 32-bit birds in the 32-bit trees.

And that's the point of the emulation: To keep the 32-bit program happy by simulating a 32-bit execution environment.

Commenter Koro is writing an installer in the form of a 32-bit program that detects that it's running on a 64-bit system and wants to copy files (and presumably set registry entries and do other installery things) into the 64-bit directories, but the emulation layer redirects the operations into the 32-bit locations. The question is "What is the way of finding the x64 Program Files directory from a 32-bit application?"

The answer is "It is better to work with the system than against it." If you're a 32-bit program, then you're going to be fighting against the emulator each time you try to interact with the outside world. Instead, just recompile your installer as a 64-bit program. Have the 32-bit installer detect that it's running on a 64-bit system and launch the 64-bit installer instead. The 64-bit installer will not run in the 32-bit emulation layer, so when it tries to copy a file or update a registry key, it will see the real 64-bit file system and the real 64-bit registry.

• #### When should your destructor be virtual?

When should your C++ object's destructor be virtual?

First of all, what does it mean to have a virtual destructor?

Well, what does it mean to have a virtual method?

If a method is virtual, then calling the method on an object always invokes the method as implemented by the most heavily derived class. If the method is not virtual, then the implementation corresponding to the compile-time type of the object pointer.

For example, consider this:

class Sample {
public:
void f();
virtual void vf();
};

class Derived : public Sample {
public:
void f();
void vf();
}

void function()
{
Derived d;
Sample* p = &d;
p->f();
p->vf();
}


The call to p->f() will result in a call to Sample::f because p is a pointer to a Sample. The actual object is of type Derived, but the pointer is merely a pointer to a Sample. The pointer type is used because f is not virtual.

On the other hand, the call to The call to p->vf() will result in a call to Derived::vf, the most heavily derived type, because vf is virtual.

Okay, you knew that.

Virtual destructors work exactly the same way. It's just that you rarely invoke the destructor explicitly. Rather, it's invoked when an automatic object goes out of scope or when you delete the object.

void function()
{
Sample* p = new Derived;
delete p;
}


Since Sample does not have a virtual destructor, the delete p invokes the destructor of the class of the pointer (Sample::~Sample()), rather than the destructor of the most derived type (Derived::~Derived()). And as you can see, this is the wrong thing to do in the above scenario.

Armed with this information, you can now answer the question.

A class must have a virtual destructor if it meets both of the following criteria:

• You do a delete p.
• It is possible that p actually points to a derived class.

Some people say that you need a virtual destructor if and only if you have a virtual method. This is wrong in both directions.

Example of a case where a class has no virtual methods but still needs a virtual destructor:

class Sample { };
class Derived : public Sample
{
CComPtr<IStream> m_p;
public:
Derived() { CreateStreamOnHGlobal(NULL, TRUE, &m_p); }
};

Sample *p = new Derived;
delete p;


The delete p will invoke Sample::~Sample instead of Derived::~Derived, resulting in a leak of the stream m_p.

And here's an example of a case where a class has virtual methods but does not require a virtual destructor.

class Sample { public: virtual void vf(); }
class Derived : public Sample { public: virtual void vf(); }

Derived *p = new Derived;
delete p;


Since the object deletion occurs from the pointer type that matches the type of the actual object, the correct destructor will be invoked. This pattern happens often in COM objects, which expose several virtual methods corresponding to its interfaces, but where the object itself is destroyed by its own implementation and not from a base calss pointer. (Notice that no COM interfaces contain virtual destructors.)

The problem with knowing when to make your destructor virtual or not is that you have to know how people will be using your class. If C++ had a "sealed" keyword, then the rule would be simpler: If you do a "delete p" where p is a pointer to an unsealed class, then that class needs have a virtual destructor. (The imaginary "sealed" keyword makes it explicit when a class can act as the base class for another class.)

Page 1 of 416 (4,152 items) 12345»