December, 2003

  • The Old New Thing

    What is the command line length limit?


    It depends on whom you ask.

    The maximum command line length for the CreateProcess function is 32767 characters. This limitation comes from the UNICODE_STRING structure.

    CreateProcess is the core function for creating processes, so if you are talking directly to Win32, then that's the only limit you have to worry about. But if you are reaching CreateProcess by some other means, then the path you travel through may have other limits.

    If you are using the CMD.EXE command processor, then you are also subject to the 8192 character command line length limit imposed by CMD.EXE.

    If you are using the ShellExecute/Ex function, then you become subject to the INTERNET_MAX_URL_LENGTH (around 2048) command line length limit imposed by the ShellExecute/Ex functions. (If you are running on Windows 95, then the limit is only MAX_PATH.)

    While I'm here, I may as well mention another limit: The maximum size of your environment is 32767 characters. The size of the environment includes the all the variable names plus all the values.

    Okay, but what if you want to pass more than 32767 characters of information to a process? You'll have to find something other than the command line. We'll discuss some options tomorrow.

  • The Old New Thing

    What's the difference between CreateMenu and CreatePopupMenu?

    CreateMenu creates a horizontal menu bar, suitable for attaching to a top-level window. This is the sort of menu that says "File, Edit", and so on.

    CreatePopupMenu creates a vertical popup menu, suitable for use as a submenu of another menu (either a horizontal menu bar or another popup menu) or as the root of a context menu.

    If you get the two confused, you can get strange menu behavior. Windows on rare occasions detects that you confused the two and converts as appropriate, but I wouldn't count on Windows successfully reading your mind.

    There is no way to take a menu and ask it whether it is horizontal or vertical. You just have to know.

    Answers to other questions about menus:

    When a window is destroyed, its menu is also destroyed. When a menu is destroyed, the entire menu tree is destroyed. (All its submenus are destroyed, all the submenu's submenus, etc.) And when you destroy a menu, it had better not be the submenu of some other menu. That other menu would have an invalid menu as a submenu!

    If you remove a submenu from its parent, then you become responsible for destroying it, since it no longer gets destroyed automatically when the parent is destroyed.

    It is legal for a menu to be a submenu of multiple parent menus. Be extra careful when you do this, however, because if one of the parents is destroyed, it will destroy the submenu with it, leaving the other parent with an invalid submenu.

    And finally: The menu nesting limit is currently 25 on Windows XP. That may change in the future, of course. (As with window nesting, Windows 95 let you go ahead and nest menus all you wanted. In fact, you could go really evil and create an infinite loop of menus. You crashed pretty quickly thereafter, of course...)

  • The Old New Thing

    What is the window nesting limit?


    In the old days, Windows didn't bother enforcing a nesting limit because, well, if you want to nest windows 200 deep, that's your decision. Many window operations are recursive, but since everything happened on the application's stack, it was your own responsibility to make your stack big enough so it didn't overflow.

    But Windows NT moved the window manager off the application stack (first into a separate process, then into kernel mode). So now the OS needs to watch out for stack overflow attacks from people creating too many nested windows.

    The window nesting limit was set to 100 for the early days of Windows NT. For Windows XP, it dropped to 50 because increased stack usage in some internal functions caused us to overflow at around 75. Dropping to 50 created some breathing room.

    Disclaimer: I was not personally involved in this issue. I'm just reporting what I was able to figure out from reading checkin logs.

  • The Old New Thing

    When programs grovel into undocumented structures...


    Three examples off the top of my head of the consequences of grovelling into and relying on undocumented structures.

    Defragmenting things that can't be defragmented
    In Windows 2000, there are several categories of things that cannot be defragmented. Directories, exclusively-opened files, the MFT, the pagefile... That didn't stop a certain software company from doing it anyway in their defragmenting software. They went into kernel mode, reverse-engineered NTFS's data structures, and modified them on the fly. Yee-haw cowboy! And then when the NTFS folks added support for defragmenting the MFT to Windows XP, these programs went in, modified NTFS's data structures (which changed in the meanwhile), and corrupted your disk.

    Of course there was no mention of this illicit behavior in the documentation. So when the background defragmenter corrupted their disks, Microsoft got the blame.

    Parsing the Explorer view data structures
    A certain software company decided that they wanted to alter the behavior of the Explorer window from a shell extension. Since there is no way to do this (a shell extension is not supposed to mess with the view; the view belongs to the user), they decided to do it themselves anyway.

    From the shell extension, they used an undocumented window message to get a pointer to one of the internal Explorer structures. Then they walked the structure until they found something they recognized. Then they knew, "The thing immediately after the thing that I recognize is the thing that I want."

    Well, the thing that they recognize and the thing that they want happened to be base classes of a multiply-derived class. If you have a class with multiple base classes, there is no guarantee from the compiler which order the base classes will be ordered. It so happened that they appeared in the order X,Y,Z in all the versions of Windows this software company tested against.

    Except Windows 2000.

    In Windows 2000, the compiler decided that the order should be X,Z,Y. So now they grovelled in, saw the "X" and said "Aha, the next thing must be a Y" but instead they got a Z. And then they crashed your system some time later.

    So I had to create a "fake X,Y" so when the program went looking for X (so it could grab Y), it found the fake one first.

    This took the good part of a week to figure out.

    Reaching up the stack
    A certain software company decided that it was too hard to take the coordinates of the NM_DBLCLK notification and hit-test it against the treeview to see what was double-clicked. So instead, they take the address of the NMHDR structure passed to the notification, add 60 to it, and dereference a DWORD at that address. If it's zero, they do one thing, and if it's nonzero they do some other thing.

    It so happens that the NMHDR is allocated on the stack, so this program is reaching up into the stack and grabbing the value of some local variable (which happens to be two frames up the stack!) and using it to control their logic.

    For Windows 2000, we upgraded the compiler to a version which did a better job of reordering and re-using local variables, and now the program couldn't find the local variable it wanted and stopped working.

    I got tagged to investigate and fix this. I had to create a special NMHDR structure that "looked like" the stack the program wanted to see and pass that special "fake stack".

    I think this one took me two days to figure out.

    I hope you understand why I tend to go ballistic when people recommend relying on undocumented behavior. These weren't hobbyists in their garage seeing what they could do. These were major companies writing commercial software.

    When you upgrade to the next version of Windows and you experience (a) disk corruption, (b) sporadic Explore crashes, or (c) sporadic loss of functionality in your favorite program, do you blame the program or do you blame Windows?

    If you say, "I blame the program," the first problem is of course figuring out which program. In cases (a) and (b), the offending program isn't obvious.

  • The Old New Thing

    Why not just block the apps that rely on undocumented behavior?

    Because every app that gets blocked is another reason for people not to upgrade to the next version of Windows. Look at all these programs that would have stopped working when you upgraded from Windows 3.0 to Windows 3.1.
    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Compatibility

    Actually, this list is only partial. Many times, the compatibility fix is made inside the core component for all programs rather than targetting a specific program, as this list does.

    (The Windows 2000-to-Windows XP list is stored in your C:\WINDOWS\AppPatch directory, in a binary format to permit rapid scanning. Sorry, you won't be able to browse it easily. I think the Application Compatibility Toolkit includes a viewer, but I may be mistaken.)

    Would you have bought Windows XP if you knew that all these programs were incompatible?

    It takes only one incompatible program to sour an upgrade.

    Suppose you're the IT manager of some company. Your company uses Program X for its word processor and you find that Program X is incompatible with Windows XP for whatever reason. Would you upgrade?

    Of course not! Your business would grind to a halt.

    "Why not call Company X and ask them for an upgrade?"

    Sure, you could do that, and the answer might be, "Oh, you're using Version 1.0 of Program X. You need to upgrade to Version 2.0 for $150 per copy." Congratulations, the cost of upgrading to Windows XP just tripled.

    And that's if you're lucky and Company X is still in business.

    I recall a survey taken a few years ago by our Setup/Upgrade team of corporations using Windows. Pretty much every single one has at least one "deal-breaker" program, a program which Windows absolutely must support or they won't upgrade. In a high percentage of the cases, the program in question was developed by their in-house programming staff, and it's written in Visual Basic (sometimes even 16-bit Visual Basic), and the person who wrote it doesn't work there any more. In some cases, they don't even have the source code any more.

    And it's not just corporate customers. This affects consumers too.

    For Windows 95, my application compatibility work focused on games. Games are the most important factor behind consumer technology. The video card that comes with a typical computer has gotten better over time because games demand it. (Outlook certainly doesn't care that your card can do 20 bajillion triangles a second.) And if your game doesn't run on the newest version of Windows, you aren't going to upgrade.

    Anyway, game vendors are very much like those major corporations. I made phone call after phone call to the game vendors trying to help them get their game to run under Windows 95. To a one, they didn't care. A game has a shelf life of a few months, and then it's gone. Why would they bother to issue a patch for their program to run under Windows 95? They already got their money. They're not going to make any more off that game; its three months are over. The vendors would slipstream patches and lose track of how many versions of their program were out there and how many of them had a particular problem. Sometimes they wouldn't even have the source code any more.

    They simply didn't care that their program didn't run on Windows 95. (My favorite was the one that tried to walk me through creating a DOS boot disk.)

    Oh, and that Application Compatibility Toolkit I mentioned above. It's a great tool for developers, too. One of the components is the Verifier: If you run your program under the verifier, it will monitor hundreds of API calls and break into the debugger when you do something wrong. (Like close a handle twice or allocate memory with GlobalAlloc but free it with LocalAlloc.)

    The new application compatibility architecture in Windows XP carries with it one major benefit (from an OS development perspective): See all those DLLs in your C:\WINDOWS\AppPatch directory? That's where many of the the compatibility changes live now. The compatibility workarounds no longer sully the core OS files. (Not all classes of compatibility workarounds can be offloaded to a compatibility DLL, but it's a big help.)
  • The Old New Thing

    What order do programs in the startup group execute?


    The programs in the startup group run in an unspecified order. Some people think they execute in the order they were created. Then you upgraded from Windows 98 to Windows 2000 and found that didn't work any more. Other people think they execute in alphabetical order. Then you installed a Windows XP multilingual user interface language pack and found that didn't work any more either.

    If you want to control the order that programs in the startup group are run, write a batch file that runs them in the order you want and put a shortcut to the batch file in your startup group.
  • The Old New Thing

    What's with those blank taskbar buttons that go away when I click on them?


    Sometimes you'll find a blank taskbar button that goes away when you click on it. What's the deal with that?

    There are some basic rules on which windows go into the taskbar. In short:

    • If the WS_EX_APPWINDOW extended style is set, then it will show (when visible).
    • If the window is a top-level unowned window, then it will show (when visible).
    • Otherwise it doesn't show.

    (Though the ITaskbarList interface muddies this up a bit.)

    When a taskbar-eligible window becomes visible, the taskbar creates a button for it. When a taskbar-eligible window becomes hidden, the taskbar removes the button.

    The blank buttons appear when a window changes between taskbar-eligible and taskbar-ineligible while it is visible. Follow:

    • Window is taskbar-eligible.
    • Window becomes visible ? taskbar button created.
    • Window goes taskbar-ineligible.
    • Window becomes hidden ? since the window is not taskbar-eligible at this point, the taskbar ignores it.

    Result: A taskbar button that hangs around with no window attached to it.

    This is why the documentation also advises, "If you want to dynamically change a window's style to one that doesn't support visible taskbar buttons, you must hide the window first (by calling ShowWindow with SW_HIDE), change the window style, and then show the window."

    Bonus question: Why doesn't the taskbar pay attention to all windows as they come and go?

    Answer: Because that would be expensive. The filtering out of windows that aren't taskbar-eligible happens inside USER32 and it then notifies the taskbar (or anybody else who has installed a WH_SHELL hook) via one of the HSHELL_* notifications only if a taskbar-eligibie window has changed state. That way, the taskbar code doesn't get paged in when there's nothing for it to to.
  • The Old New Thing

    Why you should never suspend a thread


    It's almost as bad as terminating a thread.

    Instead of just answering a question, I'm going to ask you the questions and see if you can come up with the answers.

    Consider the following program, in (gasp) C#:

    using System.Threading;
    using SC = System.Console;
    class Program {
      public static void Main() {
        Thread t = new Thread(new ThreadStart(Program.worker));
        SC.WriteLine("Press Enter to suspend");
        SC.WriteLine("Press Enter to resume");
      static void worker() {
        for (;;) SC.Write("{0}\r", System.DateTime.Now);

    When you run this program and hit Enter to suspend, the program hangs. But if you change the worker function to just "for(;;) {}" the program runs fine. Let's see if we can figure out why.

    The worker thread spends nearly all its time calling System.Console.WriteLine, so when you call Thread.Suspend(), the worker thread is almost certainly inside the System.Console.WriteLine code.

    Q: Is the System.Console.WriteLine method threadsafe?

    Okay, I'll answer this one: Yes. I didn't even have to look at any documentation to figure this out. This program calls it from two different threads without any synchronization, so it had better be threadsafe or we would be in a lot of trouble already even before we get around to suspending the thread.

    Q: How does one typically make an object threadsafe?

    Q: What is the result of suspending a thread in the middle of a threadsafe operation?

    Q: What happens if - subsequently - you try to access that same object (in this case, the console) from another thread?

    These results are not specific to C#. The same logic applies to Win32 or any other threading model. In Win32, the process heap is a threadsafe object, and since it's hard to do very much in Win32 at all without accessing the heap, suspending a thread in Win32 has a very high chance of deadlocking your process.

    So why is there even a SuspendThread function in the first place?

    Debuggers use it to freeze all the threads in a process while you are debugging it. Debuggers can also use it to freeze all but one thread in a process, so you can focus on just one thread at a time. This doesn't create deadlocks in the debugger since the debugger is a separate process.
  • The Old New Thing

    How do I pass a lot of data to a process when it starts up?


    As we discussed yesterday, if you need to pass more than 32767 characters of information to a child process, you'll have to use something other than the command line.

    One method is to wait for the child process to go input idle, then FindWindow for some agreed-upon window and send it a WM_COPYDATA message. This method has a few problems:

    • You have to come up with some way of knowing that the child process has created its window so you can start looking for it. (WaitForInputIdle is helpful here.)
    • You have to make sure the window you found belongs to the child process and isn't just some other window which happens to have the same name by coincidence. Or, perhaps, not by coincidence: If there is more than once instance of the child process running, you will need to make sure you're talking to the right one. (GetWindowThreadProcessId is helpful here.)
    • You have to hope that nobody else manages to find the window and send it the WM_COPYDATA before you do. (If they do, then they have effectively taken over your child process.)
    • The child process needs to be alert for the possibility of a rogue process sending bogus WM_COPYDATA messages in an attempt to confuse it.

    The method I prefer is to use anonymous shared memory. The idea is to create a shared memory block and fill it with goodies. Mark the handle as inheritable, then spawn the child process, passing the numeric value of the handle on the command line. The child process parses the handle out of its command line and maps the shared memory block to see what's in it.

    Remarks about this method:

    • You need to be careful to validate the handle, in case somebody tries to be sneaky and pass you something bogus on your command line.
    • In order to mess with your command line, a rogue process needs PROCESS_VM_WRITE permission, and in order to mess with your handle table, it needs PROCESS_DUP_HANDLE permission. These are securable access masks, so proper choice of ACLs will protect you. (And the default ACLs are usually what you want anyway.)
    • There are no names that can be squatted or values that can be spoofed (assuming you've protected the process against PROCESS_VM_WRITE and PROCESS_DUP_HANDLE).
    • Since you're using a shared memory block, nothing actually is copied between the two processes; it is just remapped. This is more efficient for large blocks of data.

    Here's a sample program to illustrate the shared memory technique.

    #include <windows.h>
    #include <shlwapi.h>
    #include <strsafe.h>
    struct STARTUPPARAMS {
        int iMagic;     // just one thing

    In principle, the STARTUPPARAMS can be arbitrarily complicated, but for illustrative purposes, I'm just going to pass a single integer.

    STARTUPPARAMS *CreateStartupParams(HANDLE *phMapping)
        STARTUPPARAMS *psp = NULL;
        sa.nLength = sizeof(sa);
        sa.lpSecurityDescriptor = NULL;
        sa.bInheritHandle = TRUE;
        HANDLE hMapping = CreateFileMapping(INVALID_HANDLE_VALUE,
                    &sa, PAGE_READWRITE, 0,
                    sizeof(STARTUPPARAMS), NULL);
        if (hMapping) {
            psp = (STARTUPPARAMS *)
                        MapViewOfFile(hMapping, FILE_MAP_WRITE,
                            0, 0, 0);
            if (!psp) {
                hMapping = NULL;
        *phMapping = hMapping;
        return psp;

    The CreateStartupParams function creates a STARTUPPARAMS structure in an inheritable shared memory block. First, we fill out a SECURITY_ATTRIBUTES structure so we can mark the handle as inheritable by child processes. Setting the lpSecurityDescriptor to NULL means that we will use the default security descriptor, which is fine for us. We then create a shared memory object of the appropriate size, map it into memory, and return both the handle and the mapped address.

    STARTUPPARAMS *GetStartupParams(LPSTR pszCmdLine, HANDLE *phMapping)
        STARTUPPARAMS *psp = NULL;
        LONGLONG llHandle;
        if (StrToInt64ExA(pszCmdLine, STIF_DEFAULT, &llHandle)) {
            *phMapping = (HANDLE)(INT_PTR)llHandle;
            psp = (STARTUPPARAMS *)
                    MapViewOfFile(*phMapping, FILE_MAP_READ, 0, 0, 0);
            if (psp) {
                //  Now that we've mapped it, do some validation
                MEMORY_BASIC_INFORMATION mbi;
                if (VirtualQuery(psp, &mbi, sizeof(mbi)) >= sizeof(mbi) &&
                    mbi.State == MEM_COMMIT &&
                    mbi.BaseAddress == psp &&
                    mbi.RegionSize >= sizeof(STARTUPPARAMS)) {
                    // Success!
                } else {
                    // Memory block was invalid - toss it
                    psp = NULL;
        return psp;

    The GetStartupParams function is the counterpart to CreateStartupParams. It parses a handle from the command line and attempts to map a view. If the handle isn't a file mapping handle, the call to MapViewOfFile will fail, so we get that part of the parameter validation for free. We use VirtualQuery to validate the size of the memory block. (We can't use a strict equality test since the value we get back will be rounded up to the nearest page multiple.)

    void FreeStartupParams(STARTUPPARAMS *psp, HANDLE hMapping)

    After we're done with the startup parameters (either on the creation side or the consumption side), we need to free them to avoid a memory leak. That's what FreeStartupParams is for.

    void PassNumberViaSharedMemory(HANDLE hMapping)
        TCHAR szModule[MAX_PATH];
        TCHAR szCommand[MAX_PATH * 2];
        DWORD cch = GetModuleFileName(NULL, szModule, MAX_PATH);
        if (cch > 0 && cch < MAX_PATH &&
            SUCCEEDED(StringCchPrintf(szCommand, MAX_PATH * 2,
                      TEXT("\"%s\" %I64d"), szModule,
                      (INT64)(INT_PTR)hMapping))) {
            STARTUPINFO si = { sizeof(si) };
            if (CreateProcess(szModule, szCommand, NULL, NULL,
                              0, NULL, NULL, &si, &pi)) {

    Most of the work here is just building the command line. We run ourselves (using the GetModuleFileName(NULL) trick), passing the numerical value of the handle on the command line, and passing TRUE to CreateProcess to indicate that we want inheritable handles to be inherited. Note the extra quotation marks in case our program's name contains a space.

    int CALLBACK
    WinMain(HINSTANCE hinst, HINSTANCE hinstPrev,
            LPSTR pszCmdLine, int nShowCmd)
        HANDLE hMapping;
        STARTUPPARAMS *psp;
        if (pszCmdLine[0]) {
            psp = GetStartupParams(pszCmdLine, &hMapping);
            if (psp) {
                TCHAR sz[64];
                StringCchPrintf(sz, 64, TEXT("%d"), psp->iMagic);
                MessageBox(NULL, sz, TEXT("The Value"), MB_OK);
                FreeStartupParams(psp, hMapping);
        } else {
            psp = CreateStartupParams(&hMapping);
            if (psp) {
                psp->iMagic = 42;
                FreeStartupParams(psp, hMapping);
        return 0;

    At last we put it all together. If we have a command line parameter, then this means that we are the child process, so we convert it into a STARTUPPARAMS and display the number that was passed. If we don't have a command line parameter, then this means that we are the parent process, so we create a STARTUPPARAMS, stuff the magic number into it (42, of course), and pass it to the child process.

    So there you have it: Passing a "large" (well, okay, small in this example, but it could have been megabytes if you wanted) amount of data securely to a child process.

  • The Old New Thing

    Which message numbers belong to whom?


    Valid window messages break down into four categories.

    0 .. 0x3FF (WM_USER-1): System-defined messages.

    The meanings of these messages are defined by the operating system and cannot be changed. Do not invent new messages here. Since the meanings are defined by Windows, the operating system understands how to parse the WPARAM and LPARAM parameters and can marshal the messages between processes (or knows to refuse to do so).

    0x400 .. 0x7FFF (WM_USER .. WM_APP-1): Class-defined messages.

    The meanings of these messages is determined by the implementor of the window class. (Informally: By the person who calls RegisterClass for that window class.) For example, the WM_USER+1 message means TB_ENABLEBUTTON if the window is a toolbar control, but it means TTM_ACTIVATE if it is a tooltip control, and it means DM_SETDEFID if it is a dialog box. If you created your own control, it would mean something else completely different. Since anybody can create a message in this range, the operating system does not know what the parameters mean and cannot perform automatic marshalling.

    0x8000 .. 0xBFFF (WM_APP ... MAXINTATOM-1): Application-defined messages.

    The meanings of these messages is determined by the application that created the window. (Informally: By the person who calls CreateWindow.) This message region was created in Windows 95 to ensure that applications which subclass a window and generate custom messages will not interfere with new messages created by the window class in future versions. Again, since anybody can create a message in this range, the operating system does not know what the parameters mean and cannot perform automatic marshalling.

    0xC000 .. 0xFFFF (MAXINTATOM .. MAXWORD): Registered messages.

    The meanings of these messages is determined by the caller of RegisterWindowMessage. Note that the numerical value of registered messages can change from run to run, so you must use RegisterWindowMessage to obtain the message number. Once again, since anybody can create a message in this range, the operating system does not know what the parameters mean and cannot perform automatic marshalling.

Page 1 of 5 (45 items) 12345