January, 2010

  • The Old New Thing

    Why aren't compatibility workarounds disabled when a debugger is attached?


    Ken Hagan wonders why compatibility workarounds aren't simply disabled when a debugger is attached.

    As I noted earlier, many compatibility workarounds are actually quicker than the code that detects whether the workaround would be needed.

    BOOL IsZoomed(HWND hwnd)
      return GetWindowLong(hwnd, GWL_STYLE) & WS_MAXIMIZED;

    Now suppose you find a compatibility problem with some applications that expect the IsZoomed function to return exactly TRUE or FALSE. You then change the function to something like this:

    BOOL IsZoomed(HWND hwnd)
      return (GetWindowLong(hwnd, GWL_STYLE) & WS_MAXIMIZED) != 0;

    Now, we add code to enable the compatibility workaround only if the application is on the list of known applications which need this workaround:

    BOOL IsZoomed(HWND hwnd)
      if (GetWindowLong(hwnd, GWL_STYLE) & WS_MAXIMIZED) {
          if (IsApplicationCompatibilityWorkaroundRequired(ISZOOMED_TRUEFALSE)) {
              return TRUE;
          } else {
              return WS_MAXIMIZED;
      } else {
        return FALSE;

    What was a simple flag test now includes a check to see whether an application compatibility workaround is required. These checks are not cheap, because the compatibility infrastructure needs to look up the currently-running application in the compatibility database, check that the version of the application that is running is the one the compatibility workaround is needed for (which could involve reading the file version resource or looking for other identifying clues), and then returning either the compatible answer (TRUE) or the answer that resulted from the original simple one-line function.

    So not only is the function slower (having to do a compatibility check), it also looks really stupid.

    Oh wait, now we also have to stick in a debugger check:

    BOOL IsZoomed(HWND hwnd)
      if (GetWindowLong(hwnd, GWL_STYLE) & WS_MAXIMIZED) {
          if (!IsDebuggerPresent() &&
             IsApplicationCompatibilityWorkaroundRequired(ISZOOMED_TRUEFALSE)) {
              return TRUE;
          } else {
              return WS_MAXIMIZED;
      } else {
        return FALSE;

    And then people complain that Windows is slow and bloated: A simple one-line function ballooned into ten lines.

    Another reason why these compatibility workarounds are left intact when a debugger is running is that changing program behavior based on whether a debugger is attached would prevent application vendors from debugging one problem because all sorts of new problems suddenly got injected.

    Suppose you support Program X, and you get a report of a security vulnerability in your program. You run the program under the debugger, and when you run the alleged exploit code, you find that the program doesn't behave the same as it does when the debugger is not attached. Some compatibility workaround that was active when your program is run normally is being suppressed, and the change in behavior changes your program enough that the alleged security exploit doesn't behave quite the same.

    When run outside the debugger, the program crashes, but when run under the debugger, the program displays a strange error message but manages to keep from crashing. Congratulations, you introduced a Heisenbug.

    And then you say, "There's something wrong with the debugger. It must be a bug in Windows."

    Pre-emptive Yuhong Bao comment: The heap manager switches to an alternate algorithm if it detects a debugger, and the CloseHandle function raises an exception if running under the debugger.

  • The Old New Thing

    What is the hSection parameter to CreateDIBSection for?


    The CreateDibSection function creates a special type of bitmap known as a DIB section. We've worked with these guys before: The feature of DIB sections that is by far the most interesting is that the raw pixels in the bitmap are mapped into your process space as if they were normal memory, which you can read from and write to directly.

    But what is the deal with that funky hSection parameter? Although the ppvBits parameter receives "a pointer to the location of the DIB bit values," the documentation also says that if you pass NULL, then "an application cannot later obtain a handle to this memory." That second part makes no sense. Why would I want to obtain a handle to the memory if I passed NULL, since I told it I didn't have any memory to begin with? Why is it so important to call out that I can't retrieve NULL? The documentation appears to go out of its way to point out something that makes no sense!

    Let's look at that second part in a little more context. Here is the entire description for the hSection parameter:

    If hSection is NULL, the system allocates memory for the DIB. In this case, the CreateDIBSection function ignores the dwOffset parameter. An application cannot later obtain a handle to this memory. The dshSection member of the DIBSECTION structure filled in by calling the GetObject function will be NULL.

    The "this" in "this memory" is "the memory the system allocated," not "the memory the application passed in." Because, as you noticed, the application didn't pass in any memory! It's trying to tell you that when you tell the system to allocate memory for the DIB section, you don't get to peek back in and get the handle to the memory which the system allocated.

    Let's look at the bigger picture here. The memory for a DIB section needs to be mapped into the application's address space, and from the description, the application has the option of passing explicit storage in the form of a file mapping handle (and offset into that file mapping handle), or it can pass NULL and let GDI worry about where to store it. If you were writing the GDI code to manage the memory for DIB sections, how would you do it?

    How would you do it? is is another one of those problem-solving questions similar to What would the world be like if that were true? or Imagine if two programs did this? For one thing, How would you do it? lets you rule out designs that involve clairvoyance or psychic powers. (It's surprising how often people quietly assume that systems are built upon clairvoyance.) For another thing, it forces you to put on a different hat and view the problem from another point of view, one which may help you understand the system better.

    If you had to implement a system where memory could be managed either in the form of a file mapping handle or a mechanism of your choice, you probably would choose as your alternate mechanism... a file mapping handle. That way, there is only one code path for memory management instead of two.

    Suppose you had to install a door security system that both you and Bob could use. Bob insists that he use a traditional metal key to unlock the door, but says you can use any system you want. Would you design a combination system that could be unlocked either with a metal key or an electronic smart card? Or would you just use a traditional keyed lock with two sets of keys, giving one of Bob and keeping the other for yourself?

    When you create a DIB section and pass NULL for the hSection, GDI simply creates an internal hSection and uses that. The documentation is trying to say that the internal hSection is inaccessible to the application.

    Note: I don't know if this is actually what happens internally, but it's the simplest explanation that matches the known facts.

  • The Old New Thing

    Learning how to cheat at Candy Land


    My young niece received the game Candy Land and wants to play it several times a day. Naturally, I am frequently drafted as an opponent.

    I discovered that my niece cheats rampantly. Sometimes, she will advance three green squares instead of two. Or if a yellow card will take her to a licorice square (lose a turn), she will ignore it and go to the yellow square after that. But the best cheating takes place when she draws a pink location card which sends her backward.

    My niece is always careful to draw the card and turn it so only she can see what it is. If the card is an unfavorable one, she will hide it under her knee and tell me, "Hey, go play with the baby for a little bit." I naturally oblige, and when I return after a few seconds, her location card has magically changed into a two-green card or something else comparatively harmless.

    I found this change in personality surprising. In earlier encounters, she was happy to share in the winning. In fact, sometimes, while resetting a game after winning, she would say, "Okay, you win this next one." And she would let me win!

    Anyway, after one of the games of Candy Land which she won (actually, she wins every game), I declined to play another because she was cheating. She pled with me, "Okay, this time, no cheating." I agreed to play another game.

    In the game that followed, she cheated five times, even more than in the previous game.

    She wanted to play again. "I'm serious, I'm not cheating now."

    In the "I'm serious, no cheating" game, there was, I will concede, technically no cheating during game play.

    Instead, she stacked the deck.

    Perhaps I need to study up on how to cheat at Candy Land. I could mark the cards or go for one of the more advanced methods.

    Bonus chatter

  • The Old New Thing

    How to change the debugger attached to a process


    Suppose your application crashes and debugger X is automatically connected because that's how the system happened to be configured. But you would prefer to use debugger Y. After installing debugger Y, how do you switch the debugger from X to Y? If you try to connect debugger Y to the process, you get the error code STATUS_PORT_ALREADY_SET, because only one debugger can be connected to a process at a time. But if you disconnect the old debugger, the application will disappear with it. How do you escape from this Catch-22?

    Here's what you do.

    • Attach the ntsd debugger in non-invasive mode: use -pv instead of -p when specifying the process id.
    • The ntsd debugger will suspend all the threads in the process.
    • Now tell debugger X to resume the process and detach from it. (If debugger X is ntsd, then the command for this is qd.)
    • Next, tell debugger Y to attach to the process.
    • Finally, go to the ntsd debugger which you attached in non-invasive mode, and tell it to detach with the qd command.

    This trick works because the non-invasive mode of debugging doesn't actually connect to the process as a debugger. It merely suspends all the threads in the process and lets you snoop around its memory. As a result, when you disconnect the original debugger and tell it to resume execution of the application, the application doesn't actually resume because the non-invasive ntsd is keeping it suspended. You then can attach the new debugger to the process and resume your debugging.

    In other words, the non-invasive ntsd acts as a bridge, holding the process frozen while one debugger gets out and another one comes in.

  • The Old New Thing

    The wisdom of seventh graders: Success


    Seventh grade students (ages 12 to 13, roughly) were asked to write an essay on what success is and how you know when you've achieved it. The assignment was given under standardized test conditions: 90 minutes with nothing but pencil and paper, with an additional hour available upon request. (In practice, few students ask for the extra hour.)

    Remember, these are only the funny sentences/excerpts. Do not assume that all students write like this.

    On the nature of success

    • Success is not a very difficult thing to think about, but if you explain it, it can be hard to explain.
    • Stride to Success
    • Success is not the end, but the beginning.
    • As the say goes, the only time you will find sucess before work is in the dictionary.

    Recognizing success

    • You have to do something that is genuin and worth while.
    • Success means you have a jolly time.
    • It's a feel good feeling.
    • To me success is a goal a feeling of prowness. that tells me that this is a goal of your limits not an age limit.
    • Sucess is when you had the opportunity to do something greatful.
    • Success is shaking your butt in their face that you are better than them.
    • Another meaning for success is just winning. Winning is what most people think success is, but winning is only half the meaning of success. Winning is just winning. Yeah, but at least you can shake your butt in their face, and that's worth something.
    • Success is trying your best... I have a question. do teachers ever have success?
    • You know you have reached the point of succession when you do really well.
    • So many kinds of meanings can mean success but I think success mostly means to me a achieving.

    Steps which do not lead to success

    • If you slack you get failure. You end up poor, sad, terrible.
    • You can't just sit around watching television and eat Doritos.

    The fleeting nature of success

    • But even though money may make you happy it doesn't last forever.
    • After all, feeling like the king of the mountain doesn't come as soften as the frustration does.

    The recipe for success

    • After you graguat from callage you can say, Success has finaly hit me.
    • If you get good grades, you can go to a splendid colledge and then get a splendid sallary. Sounds splendid.
    • One way is to make a T chart. Put what your good at on the left and what you love on the left. Burn the right hand side.
    • My opinion of success is a cab driver.
    • If you do all of the things mentioned in this article, you'll have a very successful life.

    Personal stories of success

    • Whether your moment is singing on a stage to a huge audience, or handing out your first order of French fries at McDonald's.
    • Success can be as simple as winning a video game. You win easily and celebrate with a cookie. Do you shake your butt in their face?
    • I was the most happiest person ever!
    • Success is when you tell a casino that someone cheated at Black Jack.
    • Success is like when I was in 4th grade and I learned to play Smoke on the Water a little.
    • One thing that really makes a family is a pet.
    • outsmarting and generally twisting my opponents words against him feels like hitting the jackpot.
    • I was in such a dilemma!

    Future success

    • The fist and most important key to making a family is loving, beautiful, and wonderful wife.
    • I want to get a wife that is nice, hard working, and has simple pleasures.
    • My fantasies of success are having a great boyfriend, a job studying the migration paths of turtles in Australia, and eventually growing old and waltzing through an arch of Agust trees. But under no circumstances marry your boyfriend. That ruins it.
    • Over the past 12 years I've been alive, I've had many emotions experiences, and a few lovers. Boys write about their wives. Girls write about their boyfriends and lovers. Wait, lovers? How old are you again?
    • Success is feeling comfortable having Thanksgiving with your in-laws. I'm happy to declare that I have achieved success.

    Examples of other successful people

    • When Billy Armstrong made it to the moon, he had a great feeling of accomplishment.
    • The teacher went around asking what they wanted to be when they grew up. Most kids said normal things but one girl said she wanted to be a unicorn. The class laughed. Because she believed in her insane dream she is the creator of My Little Pony.
    • Mr. Bill Gates did not just read a get rich quick book and deside to crate Microsoft.

    Out of about 300 students, the number who cited Bill Gates as a successful person: 17.

  • The Old New Thing

    Can you get rotating an array to run faster than O(n²)?


    Some follow-up remarks to my old posting on rotating a two-dimensional array:

    Some people noticed that the article I linked to purporting to rotate the array actually transposes it. I was wondering how many people would pick up on that.

    I was surprised that people confused rotating an array (or matrix) with creating a rotation matrix. They are unrelated operations; the only thing they have in common are the letters r-o-t-a-t-i. A matrix is a representation of a linear transformation, and a rotation matrix is a linear transformation which rotates vectors. In other words, applying the rotation matrix to a vector produces a new vector which is a rotated version of the original vector. The linear transformation is a function of one parameter: It takes a vector and produces a new vector. A rotation matrix is a matrix which rotates other things. Whereas rotating an array is something you do to the array. The array is the thing being rotated, not the thing doing the rotating. It didn't even occur to me that people would confuse the two. It's the difference a phone dial and dialing a phone.

    Showing that you cannot rotate an array via matrix multiplication is straightforward. Suppose there were a matrix R which rotated an array (laid out in the form of a matrix) clockwise. The result of rotating the identity matrix would be a a matrix with 1's along the diagonal from upper right to lower left, let's call that matrix J. Then we have RI = J, and therefore R = J. Now apply R to both sides: RRI = RJ = I and therefore R² = I. But clearly rotating clockwise twice is not the identity for n ≥ 2. (Rotating clockwise twice is turning upside-down.)

    A more mechanical way to see this is to take the equation R = J and show that J does not perform the desired operation; just try it on the matrix with 1 in the upper left entry and 0's everywhere else.

    And since it's one of those geeky math pastimes to see how many differents proofs you can come up with for a single result, the third way to show that rotation cannot be effected by matrix multiplication is to observe that the transformation is not linear. (That's the magical algebra-theoretical way of showing it, which is either so obvious you can tell just by looking at it or so obscure it defies comprehension.) [The transformation viewed as a transformation on matrices rather than a transformation on column vectors is indeed linear, but the matrix for that would be an n² × n² matrix, and the operation wouldn't be matrix multiplication, so that doesn't help us here.]

    The last question raised by this exercise was whether you could do better than O(n²). Computer science students spend so much time trying to push the complexity of an algorithm down that they neglect to learn how to tell that you can't go any lower. In this case, you obviously can't do better than O(n²) because every single one of the n² entries in the array needs to move (except of course the center element if n is odd). If you did less than O(n²) of work, then for sufficiently large n, you will end up not moving some array elements, which would be a failure to complete the required operation.

    Bonus chatter: Mind you, you can do better than O(n²) if you change the rules of the problem. For example, if you allow pretending to move the elements, say by overloading the [] operator, then you can perform the rotation in O(1) time by just writing a wrapper:

    struct IArray
      virtual int& Element(int x, int y) = 0;
      virtual ~IArray() = 0;
    class RotatedArray : public IArray {
     RotatedArray(IArray *p) : m_p(p) { }
     ~RotatedArray() { delete m_p; }
     int& Element(int x, int y) {
      return m_p->Element(y, x);
     IArray *m_p;
    void RotateInPlace(IArray *& p, int N)
     p = new RotatedArray(p);

    This pseudo-rotates the elements by changing the accessor. Cute but doesn't actually address the original problem, which said that you were passed an array, not an interface that simulates an array.

  • The Old New Thing

    Thanks for coming together to enjoy the holiday together, now get off my lawn


    Due to a confluence of circumstances involving parents on an overseas vacation and other older relatives choosing not to attend (probably related to parents being out of town), I ended up being the oldest person at the extended family holiday dinner table.

    This was a first for me, and I resisted the urge to shout "Get off my lawn!" to the assembled masses or to bore younger relatives with rambling pointless stories from my youth.

    I guess this is the converse to a friend of mine who presided over a holiday family gathering at his house some years ago, and who realized that if seating at the dinner table were done strictly by age, he would have to sit at the children's table.

  • The Old New Thing

    Microspeak: Engagement


    Meetings are so passé. You no longer have a meeting with a customer; you have an engagement:

    I have a customer engagement tomorrow and they have a question surrounding Feature X.

    Note that this use of the phrase customer engagement is different from the process known as customer engagement. The process is an ongoing interaction, a long-term activity to build customer loyalty.

    The author of the above sentence is not using it in the process sense (because you don't have "a" customer engagement; rather, a meeting is one component of the overall process of customer engagement). Nope, the author is just trying to sound cool.

  • The Old New Thing

    How many sides are there to a snowflake? The answer may surprise your marketing department


    Joe McManis runs the Snowflake Fail blog, which chronicles the various places non-hexagonal snowflakes are found in nature marketing.

    Bonus snowflake failage:

  • The Old New Thing

    How about not granting debug privileges to users?


    Commenter Yuhong Bao suggests, "How about not granting debug privileges on the user? This will make bypassing the protection impossible."

    This is such a great idea that Windows has worked that way for years.

    Normal non-administrative users do not have debug privilege. They can only debug processes that they already have PROCESS_ALL_ACCESS to. In other words, non-administrative users can only pwn processes that they already pwn. No protection is being bypassed since you had full access in the first place.

    The SeDebugPrivilege allows you to debug any process, even those to which you do not have full access. This is clearly dangerous, which is why it's not granted to non-administrative users by default.

    Yuhong Bao also suggests, "How about separating the dangerous activities from the non-dangerous activities, or better, only allowing approved programs to do the dangerous activities?" (Where dangerous activities are defined as things that modify the program behavior.) I'm assuming this is discussing limiting the capabilities of SeDebugPrivilege, since in the absence of SeDebugPrivilege, the scope of your abilities is limited to things you already had the ability to do anyway; debugging didn't add anything new.

    But even if you limited SeDebugPrivilege to nondestructive actions, you can still lose the farm. This imaginary SeLimitedDebugPrivilege would still let you read a target process's memory, which means you can do things like steal passwords and snoop on the activities of other users.

    The last suggestion is to "only allow approved programs to do the dangerous activities." Again, I'm assuming this is discussing limiting the capabilities of SeDebugPrivilege, because without SeDebugPrivilege there is no new danger. But even in that limited context, what is an "approved program"? Approved by whom?

    Must the program be digitally signed by Microsoft? I suspect people who write debuggers which compete with, say, Microsoft Visual Studio, would be upset if they had to submit their debugger to Microsoft for approval. And what are the requirements for receiving this approval? Does the debugger have to pass some battery of tests like WHQL? There are already plenty of readers of this Web site who reject WHQL as useless. Would this "debugger certification" also be useless?

    Or maybe approval consists of merely being digitally signed at all? There are plenty of readers of this Web site who object to the high cost of obtaining a digital certificate (US$399; I don't think the $99 discounted version works for code signing.) And there are also plenty of readers who consider code signing to be payola and designed to maximize profit rather than effectiveness.

    Or do you mean that the program needs to be listed in some new registry key called something like Approved Debuggers? Then what's to stop a rogue program from just auto-approving itself by writing to the Approved Debuggers registry key on its own?

    But then again, all this is a pointless discussion once you realize that SeDebugPrivilege is granted by default only to administrators. And since administrators already pwn the machine, there's no protection that SeDebugPrivilege bypasses: You already bypassed it when you became an administrator.

Page 3 of 4 (32 items) 1234