January, 2009

  • The Old New Thing

    How do I write a program that can be run either as a console or a GUI application?

    • 29 Comments

    You can't, but you can try to fake it.

    Each PE application contains a field in its header that specifies which subsystem it was designed to run under. You can say IMAGE_SUBSYSTEM_WINDOWS_GUI to mark yourself as a Windows GUI application, or you can say IMAGE_SUBSYSTEM_WINDOWS_CUI to say that you are a console application. If you are GUI application, then the program will run without a console.

    The subsystem determines how the kernel prepares the execution environment for the program. If the program is marked as running in the console subsystem, then the kernel will connect the program's console to the console of its parent, creating a new console if the parent doesn't have a console. (This is an incomplete description, but the details aren't relevant to the discussion.) On the other hand, if the program is marked as running as a GUI application, then the kernel will run the program without any console at all.

    There are some people who want to write what I call an "opportunistic" console program. These are programs that will use the console of their parent if available, but do not want a console created for them if not. The kernel doesn't support this type of program, but that hasn't stopped some people from coming up with clever workarounds. Note that if such a program type were introduced, it would create problems with programs such as cmd.exe and Explorer which change their behavior depending on what subsystem a program belongs to. These programs would have to be modified to understand a new pseudo-subsystem called "both".

    I've also seen requests for what I call a "dynamic" console program. These are programs that want to decide at run time whether they want a console or not. For example, a program might want to run with a console only if a special command line switch is passed. To do this the kernel would have to have psychic powers: It would somehow have to know whether to hook up a console to your program or not (which happens before the program begins executing) based on something that happens in the future (when your program actually runs and parses its command line and decides whether it wants to run as a console or a GUI program). Again, people have come up with workarounds (see earlier link).

  • The Old New Thing

    If you didn't like the answer, asking the same question again is unlikely to help

    • 33 Comments

    I find it surprising how often this happens. A customer liaison will send a question to a mailing list like this:

    From: X
    To: Gizmo Discussion

    Hi, everybody.

    My customer is using the Gizmo Toolkit and wants to frob a gizmo without first registering as a frobber. They created the gizmo with CreateGizmo, passing all the default flags, and then they call AttachGizmo to attach the gizmo to a sprocket. When the sprocket detects that its host is decycling, it tries to frob the gizmo by calling FrobGizmo, but the call fails. They can't register the sprocket as a frobber because the sprocket doesn't have the right frob context. They tried setting the DefaultFrobContext registry key to Any but that didn't help. How can they frob the gizmo?

    Somebody from the Gizmo team will reply, "Sorry, but you have to register as a frobber before you can frob a gizmo. The DefaultFrobContext registry key doesn't help you here; it's for cogs, not sprockets. There is no analogous registry key for sprockets, sorry."

    That seems to be the end of it; there is no further response. Then about two weeks later, some other person will ask a suspiciously similar question.

    From: Y
    To: Gizmo Discussion

    Hello. I hope somebody can help us with this.

    My customer is using the Gizmo Toolkit and wants to frob a gizmo without first registering as a frobber. They created the gizmo with CreateGizmo, passing all the default flags, and then they call AttachGizmo to attach the gizmo to a sprocket. When the sprocket detects that its host is decycling, it tries to frob the gizmo by calling FrobGizmo, but the call fails. They can't register the sprocket as a frobber because the sprocket doesn't have the right frob context. They tried setting the DefaultFrobContext registry key to Any but that didn't help. How can they frob the gizmo?

    Hm, that question looks awfully familiar, let me look in the archives and... hey, it's a word-for-word copy of the same question somebody else asked two weeks ago!

    In situations like this, I tend to respond by attaching the original reply with the text, "I refer the honourable gentleman to the answer given some moments ago."

    Okay, maybe the customer was not happy with the answer and instead of elaborating on their situation (so somebody might be able to spot an alternate design that avoided this problem), they just reopened the case and got a different liaison the second time. Maybe person Y was simply duped into asking the same question.

    And then this happens.

    From: Y
    To: Windows Programming Discussion

    Greetings.

    My customer is using the Gizmo Toolkit and wants to frob a gizmo without first registering as a frobber. They created the gizmo with CreateGizmo, passing all the default flags, and then they call AttachGizmo to attach the gizmo to a sprocket. When the sprocket detects that its host is decycling, it tries to frob the gizmo by calling FrobGizmo, but the call fails. They can't register the sprocket as a frobber because the sprocket doesn't have the right frob context. They tried setting the DefaultFrobContext registry key to Any but that didn't help. How can they frob the gizmo?

    Okay, so much for the theory that person Y was a victim. Person Y is clearly fishing around, hoping that if you ask the right person, you'll get an answer you like. It's sort of like the teenager who asks his father, "Can I borrow your car?" When Dad says, "No," the teenager goes to his mother and asks, "Can I borrow Dad's car?" in the hopes that maybe Mom will give a more favorable answer than Dad.

    But if you're asking a question about the Gizmo Toolkit, it's going to get routed to the Gizmo team one way or another. And if you ask the same, identical question, you're going to get the same, identical answer, just with a greater degree of exasperation each time you ask it again.

    "Can I have a pony?"
    — No
    "Can I have a pony?"
    — No.
    "Can I have a pony?"
    — No.

    Let me draw you a picture.

  • The Old New Thing

    Kids love cake, but that doesn't make them good judges of cake

    • 19 Comments

    My friend who got married last year went to the Seattle Wedding Show (here are some pictures from the 2007 show courtesy of a vendor's blog) and, through a series of circumstances not relevant to the story, combined the visit with a brief stint of babysitting for her nieces, one a tomboy and the other a girly-girl. The children's father came to pick them up after a half hour, but that half hour at the wedding show was quite exciting for two little girls. It was hardly surprising that the "I want to be a princess when I grow up" girly-girl would be completely enthralled by the wedding show. What was unexpected was that the "kissing is yucky, let's go climb a tree" tomboy was also won over despite the high concentration of lacy dresses and frilly things.

    The magic ingredient is cake.

    Now you'd think kids would be experts at cake, and in fact the girls were quite enthusiastic cake-eaters. At the wedding show, there were about ten bakeries all showing off their creations, and naturally there were samples of the cakes available for tasting. Even the most hardened "I'm not cute" child, when faced with the opportunity, will stand sweetly with an adorable smile if it means that an adult will offer her a sample of cake.

    By my friend's reckoning, the girls sampled five different cakes in the span of ten minutes. Despite this broad basis for evaluation, when asked for their assessment of the relative merits of the various cakes, the only response was "It's yummy!"

    By the way, the wedding show also alerts you to all the things you never thought a wedding needed but which, it seems, have become an indispensable element without which your wedding will be a total disaster. Here's a sample from last year's show.

    • Chair covers. Not just seat covers, but chair covers. The great thing about chair covers is that they take a perfectly normal chair and turn it into something uncomfortable and unwieldy because you can't put your feet or your purse or anything else under the chair.
    • Fish in fishbowls as table centerpieces.
    • Flower preservation services.
    • Ice sculptures.
    • A stand-up comic.
    • A photo booth.
    • Sending CDs with the wedding invitations so you can include all the information on the CD instead of forcing your guest to OCR it into their calendar. (That's my only guess.)
    • Videography. (I have yet to meet anyone who watched a video of their wedding more than once. For many couples the count was zero.)

    To see what sort of exciting must-have features are on the agenda for this year, go to the Seattle Wedding show site, and click on Wedding Specialists. Or just buy a ticket to this year's show and find out in person. Though given the recent economic downturn, this year's show is much more budget-conscious.

  • The Old New Thing

    Why isn't the screen resolution a per-user setting?

    • 50 Comments

    Via the suggestion box, Dominic Self asks why screen resolution is a global setting rather than a per-user setting. Well, first of all, it's not even a global setting. It's a session setting.

    That it's not a global setting is not readily apparent most of the time since only Windows Terminal Server is set up to support multiple simultaneous interactive sessions. On Windows Terminal Server, you specify the properties of the virtual monitor you wish to connect with, including resolution, and the server accommodates your wishes. Well, up to a point. I mean if you ask for a 1,000,000×1,000,000 pixel screen, the server is probably going to say "As if!" (The Remote Desktop Connection feature found in, for example, Windows Vista Ultimate is basically the same thing, but on a smaller scale.)

    You can have ten different people logged on, each using a different video configuration. Some users might be running at lowly 640×480, others at 800×600 and still others at 1024×768. Some of them are running in 16-color mode, others 8-bit color, and some really adventuresome people with a lot of network bandwidth running at 24-bit color. You can even have the same user logged on more than once, with each session running at a different screen resolution.

    If the resolution settings really were per-user, you'd have some conflict resolution to deal with. If a user logs on and specifies a resolution different from the one stored in the user profile, does the new resolution overwrite the existing one? Or is the new resolution just a temporary resolution and the original one should stay? If a user tries to log on a second time with a conflicting resolution, is the second resolution ignored? Does the second resolution force the first session to change its resolution? Do you just get two sessions running with different resolutions? Which one gets saved as the user's preferred resolution?

    You also have to come up with a way to customize the resolution of the screen when nobody is logged on, a way to reconcile the effect of roaming profiles, what happens if you do a Run As on a user whose screen resolution conflicts with the user who opened the session.

    I'm not saying that these problems can't be solved. They probably can, given enough thought, but not all of the solutions will please everybody all the time, because no matter what you do, somebody will tell you that you're an idiot. And think of all the time and effort necessary to design how the feature should work, nail all the boundary conditions ("What happens if your per-user setting conflicts with an administrative policy?"), then code it up, write automated tests for it, run it through usability exercises ("Does the behavior match what users intuitively expect?" The answers may surprise you.) write up the documentation and help text, and continue maintaining the code, tests, and documentation for the feature's anticipated lifetime (which in this case is probably forever). Could all those resources have been spent on something that would have a greater total benefit to the customer base? (The answer to that is always "Yes"—everybody and her sister-in-law can find a way to finish the sentence, "I can't believe they wasted all that time on this stupid feature instead of fixing...")

    Remember, every feature starts with minus 100 points.

  • The Old New Thing

    Where does shell.windows.com get information about file extensions, and how do I get in on that action?

    • 46 Comments

    If you double-click a file for which there is no registered handler, Windows will offer to visit the Web service on shell.windows.com to locate a program that can open it. But where does this information come from, and how can you add your program to the database?

    Knowledge Base article Q929149, titled Windows File Association System On-Boarding Process, provides step-by-step instructions on how you can add your file extension.

    If you look at the existing entries on shell.windows.com, most of them have relatively straightforward and neutral descriptions. "This document is a PowerPoint presentation." "This document is a sound file." But there is at least one company that decided to use the file association service for a bit of grandstanding. "Invented by XYZ company and perfected over 15 years, ABC file format lets you capture information from any application, on any computer, and share it with anyone around the world."

    By the way, if the file association Web service offends you, you can disable it.

  • The Old New Thing

    Microspeak: Learnings

    • 22 Comments

    If things you teach are teachings, then things you learn must be learnings, right? Good Microspeak citations for this word are hard to find since the word is rarely used in a sentence; it's just a heading in a slide presentation. I found dozens of presentations that had a slide titled Learnings from XYZ, or, for those who want to sound really fancy, Key Learnings from XYZ, but very few actual sentences. Here are two:

    Alice will send an email to Bob with regards to any learnings from the XYZ program being incented to do ABC.
    What are our key learnings for this project?

    There's that word key again, along with a surprise appearance of incent.

    And that second citation barely counts since it's really just a prose version of the slide heading!

  • The Old New Thing

    Why doesn't Windows 95 format floppy disks smoothly?

    • 49 Comments

    Welcome, Slashdot readers. Remember, this Web site is for entertainment purposes only.

    Who spends all day formatting floppy disks? From the reaction of geekdom, it appears that there are lots of geeks who sit around formatting disks all day. (Psst, you can buy them pre-formatted.) But why did Windows 95 get all sluggish when you formatted a floppy disk?

    It's that pesky MS-DOS compatibility again.

    As we saw a while ago, MS-DOS acted as the 16-bit legacy device driver layer for Windows 95. Even though the operation was handled by the 32-bit file system, all I/O calls were routed through 16-bit code (if only briefly) so that 16-bit drivers, TSR, and the like would see what appeared to be "normal 16-bit behavior" and continue operating in the manner to which they had become accustomed.

    In the old 16-bit days, disk formatting was done via software interrupt 13h, and many programs took advantage of this by hooking the interrupt so they would know whenever a floppy disk was being formatted. Some TSRs did this, as did backup programs (including backup programs designed for Windows 3.0 which included 32-bit Windows 3.x drivers—VxDs they were called—to monitor the floppy drive). But that doesn't explain everything. After all, Windows 95 sent all disk I/O through the 16-bit vectors, not just floppy disk formatting. Why does floppy disk formatting take such a toll on the system?

    As I noted in the linked article, the 32-bit file system did a lot of fakery to make 16-bit code believe that MS-DOS was in charge, even though it wasn't. Anybody who's done TSR programming (wow, the phrase anybody who's done TSR programming used to cover a lot of people but nowadays describes a few dozen geezers, most of whom are trying very hard to forget those days) knows all about the INDOS flag. This was a flag that MS-DOS set when an MS-DOS I/O call was active. Since MS-DOS was not re-entrant, TSRs had to pay close attention to that flag to know whether it was safe to issue MS-DOS calls or not. This INDOS flag was the 16-bit manifestation of what the 32-bit kernel called simply The Critical Section, with the definite article, because the 32-bit kernel kept the two critical sections (the 32-bit one and the 16-bit one) in sync so that MS-DOS drivers and TSRs wouldn't themselves get re-entered. If one virtual machine claimed the critical section, another virtual machine that tried to claim it would wait until the first one released it. In that manner, the driver or TSR would not get re-entered.

    As I already noted, back in the 16-bit days, the actual work of formatting was done by the ROM BIOS, and for compatibility reasons, floppy disk formatting was still sent through software interrupt 13h on the 16-bit side so any TSRs or drivers could see what was going on. There are a lot of crazy ROM BIOSes out there, and when a floppy disk format request was issued, the 32-bit kernel would do a bunch of extra work to ensure that the ROM BIOS got the execution environment it wanted. For example, the hardware timer ports were temporarily unvirtualized so as not to mess up the sensitive timing loops that ROM BIOSes used for floppy disk formatting.

    Okay, let's add up the damage. When a floppy disk is formatting, the timer is unvirtualized so that the ROM BIOS timing loops will run accurately. Only the virtual machine that is formatting the floppy drive receives timer ticks; the others have to wait. No timer ticks means the scheduler doesn't get told when it's time to let another thread run. What's more, the critical section is held across this operation, which means that no other thread can issue I/O operations either. And on top of that, the floppy disk is a slow medium, so any operations that wait on the floppy disk will have to sit and wait for several seconds.

    Well, at least floppy disks are formatted a track at a time, so the system doesn't get locked out for the entire duration of the format operation. The ROM BIOS would be told to format a track, and when it was done, the timers would be returned to normal (allowing the scheduler to do a little bit of scheduling), the critical section would be released (so that any pent-up I/O gets a chance to run), but then the FORMAT.COM program would turn around and format the next track, and the system would go back into hang on, let's not disturb the ROM BIOS while it does its thing mode for another track.

    Now, as with the 32-bit file system, there was a 32-bit floppy driver that tried to catch the format operations on the back end, and if successful, it would take over the job of formatting one track from the ROM BIOS. It was a valiant effort, but it doesn't matter how high-performance your driver is; the speed of formatting a track is pretty much constrained by the mechanics of the floppy disk drive itself. (The person responsible for Windows 95's 32-bit floppy driver was no slouch. I'll try to remember to tell some more stories later.)

    Sure, if Windows 95 didn't have to be compatible with 16-bit device drivers, TSRs, and squirly ROM BIOSes, it could have gone straight to the 32-bit floppy driver to do the formatting without having to do all this timer and critical section nonsense. But it turns out we already had a product that said good-bye to compatibility with 16-bit device drivers, TSRs, 16-bit Windows programs that talked to custom 32-bit Windows 3.x drivers, and squirly ROM BIOSes. It was called Windows NT.

    If you want Windows NT you know where to find it.

  • The Old New Thing

    But then we ran into problems when we started posting 10,000 messages per second

    • 4 Comments

    Once upon a time, a long, long time ago, there was a research team inside Microsoft who was working on alternate models for handling input. I don't know what eventually came of that project, and I don't even remember the details of the meeting, but I do remember the punch line, so I'm just going to make up the rest.

    The research project broke up the duties of their system into a few components. The two that are important to the story are a driver component which received information from various hardware devices and transmitted that information via the PostMessage function to another component whose job it was to study those input messages and route them to the appropriate application. (In 16-bit Windows, the PostMessage function was specifically written so it could be called from device drivers during hardware interrupts.) Each time the driver received information from a hardware device, it posted a message to its helper program.

    Everything seemed to go reasonably smoothly. The device driver received a hardware event, it posted a message to the helper program, and the helper program retrieved the message and processed it. But once they cranked up the hardware devices to produce information at a higher rate (and therefore produced input with much finer resolution), the events started coming in faster and faster, and their design started to collapse under the pressure.

    The research team asked to meet with the user interface team to help work out their problems under load. They outlined their design and explained that it worked well at low data rates, "but then we ran onto problems when we started posting 10,000 messages per second."

    At that point, the heads of all the user interface people just sat there and boggled for a few seconds.

    "That's like saying your Toyota Camry has stability problems once you get over 500 miles per hour."

    If you're going to be pumping huge quantities of data through the message queue, creating a separate message for each one is crazy. Think about it: Suppose you're posting 10,000 messages per second. The thread whose job it is to process the messages gets pre-empted and doesn't run for 50 milliseconds. That's 500 messages behind schedule already. Now suppose it takes a dozen page faults (which take 8ms each, say). Now you're another 1000 messages behind. Windows NT sets an arbitrary limit of 10,000 unprocessed messages in a message queue so that a runaway program won't drain the desktop heap and roach everything. A few hiccups in your process will quickly send you over that limit.

    For this usage pattern, you want to switch from one event per message to a signal on the transition (or edge triggering).

    When the first event occurs, post a single message to the helper program saying there is work to do and set a flag saying helper window has been notified. Meanwhile, stash the information you would have included in the message into a privately-managed queue. If an event arrives when the helper window has been notified flag is set, then don't post a message; just append the work item to the queue. When the helper window receives the there is work to do message, it calls back into the driver to say Okay, give me some work to do. After it does the work, it calls into the driver to say Okay, what else do you want me to do? (Alternatively, you can have the helper window grab the entire work list at once.) When the helper window asks for work to do and comes back empty-handed, then clear the helper window has been notified flag so the next time an event occurs, a new message will be posted to kick-start the helper window.

    Commenter Hayden proposed a number of other mechanisms. The send a list of work items rather than just one technique works well if you know when the list of work items is complete and therefore is ready to send. The second technique is the one I described here; it works well if the producer doesn't really know when each chunk of incoming work is finished, or if the work that comes in is continuous. The third mechanism merely avoids the message queue altogether and uses a semaphore instead.

    The point is not to try to drive your Camry at 500 miles per hour. Find a way to get your work done while keeping the Camry well within its design parameters, or look for a different vehicle.

  • The Old New Thing

    The day shell.windows.com went down

    • 41 Comments

    When the file association Web service was first being developed, the programmer responsible for implementing the feature just scrounged around and found an old unused computer and set it up as a simple Web server under his desk, so there would be something to test the code against. That server happily churned away serving out file extension information, and when people asked for their program to be added, he would manually add it to the server. The server worked just fine, and like most things which work just fine, it was forgotten.

    And then remembered once things no longer worked just fine.

    I think it was during one of the beta cycles, or maybe it was RC1, when the quiet neglected computer went offline. I forget why, so let's pretend that the programmer unplugged it as part of an office redecoration project. Suddenly, the shell.windows.com service went down.

    What? The file association Web service went down?

    Everybody had forgotten that shell.windows.com was still running on a computer under that programmer's desk. It had done such a good job up until now that nobody gave it a second though.

    He plugged the computer back in and watched the server bang out requests like nobody's business.

    Wheels were quickly set into motion to transfer the file association Web service to a machine with a little bit more professional attention and maintenance.

  • The Old New Thing

    A process shutdown puzzle

    • 29 Comments

    In honor of National Puzzle Day, I leave you today with a puzzle based on an actual customer problem.

    Part One: The customer explains the problem.

    We have this DLL, and during its startup, it creates a thread with the following thread procedure:

    DWORD CALLBACK ThreadFunction(void *)
    {
      HANDLE HandleArray[2];
      HandleArray[0] = SetUpStuff();
      if (HandleArray[0]) {
        HandleArray[1] = ShutdownEvent;
        while (WaitForMultipleObjects(2, HandleArray,
                                 FALSE, INFINITE) == WAIT_OBJECT_0) {
          ProcessStuff();
        }
        CleanUpStuff(HandleArray[0]);
      }
      SetEvent(ThreadCompleteEvent);
      FreeLibraryAndExitThread(ThisLibrary, 0);
    }
    

    During process shutdown, the following function is called as part of DLL_PROCESS_DETACH handling:

    void StopWorkerThread()
    {
      // tell the thread to stop
      SetEvent(ShutdownEvent);
    
      // wait for it to stop
      WaitForSingleObject(ThreadCompleteEvent, INFINITE);
    
      // Clean up
      CloseHandle(ShutdownEvent);
      ShutdownEvent = NULL;
    
      CloseHandle(ThreadCompleteEvent);
      ThreadCompleteEvent = NULL;
    }
    

    The above function is hanging at the call to WaitForSingleObject. If we break in, we see that the thread that is supposed to be running the ThreadFunction is gone. I verified that the thread was successfully created, but by the time we get around to waiting for it, it's already gone.

    I checked, and nobody sets the ThreadCompleteEvent except the StopWorkerThread function. I stepped through SetUpStuff, and it succeeded. However, a breakpoint on CleanUpStuff was never hit. No exceptions were thrown either.

    I am completely stumped as to how this thread disappeared.

    You already know enough to explain how the thread disappeared.

    Part Two: After providing your explanation, the customer came up with this solution.

    Thank you for your explanation. We've made the following changes to fix the problem. Again, thank you for your help.

    DWORD CALLBACK ThreadFunction(void *)
    {
      HANDLE HandleArray[2];
      HandleArray[0] = SetUpStuff();
      if (HandleArray[0]) {
        HandleArray[1] = ShutdownEvent;
        while (WaitForMultipleObjects(2, HandleArray,
                                 FALSE, INFINITE) == WAIT_OBJECT_0) {
          ProcessStuff();
        }
        CleanUpStuff(HandleArray[0]);
      }
      // SetEvent(ThreadCompleteEvent);
      FreeLibraryAndExitThread(ThisLibrary, 0);
    }
    
    void StopWorkerThread()
    {
      // tell the thread to stop
      SetEvent(ShutdownEvent);
    
      // wait for the thread
      WaitForSingleObject(ThreadHandle, INFINITE);
    
      // Clean up
      CloseHandle(ShutdownEvent);
      ShutdownEvent = NULL;
    }
    

    Criticize this proposed solution.

    Part Three: Even though the proposed solution is flawed, explain why it doesn't cause a problem in practice. (I.e., explain why the customer is always lucky.)

Page 1 of 4 (34 items) 1234