• The Old New Thing

    Finding the constructor by scanning memory for the vtable


    In Looking for leaked objects by their vtable, we used the object's constructor to locate the vtable, and then scanned the heap for the vtable to find the leaked object. But you can run this technique in reverse, too.

    Suppose you found an object and you want to find its constructor. This is not a problem if you have the source code, but if you are doing some reverse-engineering for application compatibility purposes, you don't have the luxury of the application source code. You may have figured out that the application fails because the byte at offset 0x50 is zero, but on the previous version of Windows, it was nonzero. You want to find out who sets the byte at offset 0x50, so that you can see why it is setting it to zero instead of a nonzero value.

    If the object has a vtable, you can scan the code segments for a copy of the vtable. It will show up in an instruction like

    mov dword ptr [reg], vtable_address

    This is almost certainly the object's constructor, setting up the object vtable as part of construction. You can set a breakpoint here to break when the object is constructed, and then you can set a write breakpoint on offset 0x50 to see where its value is seto.

  • The Old New Thing

    Sure, we have RegisterWindowMessage and RegisterClipboardFormat, but where are DeregisterWindowMessage and DeregisterClipboardFormat?


    The Register­Window­Message function lets you create your own custom messages that are globally unique. But how do you free the message format when you're done, so that the number can be reused for another message? (Similarly, Register­Clipboard­Format and clipboard formats.)

    You don't. There is no Deregister­Window­Message function or Deregister­Clipboard­Format function. Once allocated, a registered window message and registered clipboard format hangs around until you log off.

    There is room for around 16,000 registered window messages and registered clipboard formats, and in practice exhaustion of these pools of numbers is not an issue. Even if every program registers 100 custom messages, you can run 160 unique programs before running into a problem. And most people don't even have 160 different programs installed in the first place. (And if you do, you almost certainly don't run all of them!) In practice, the number of registered window messages is well under 1000.

    A customer had a problem with exhaustion of registered window messages. "We are using a component that uses the Register­Window­Message function to register a large number of unique messages which are constantly changing. Since there is no way to unregister them, the registered window message table eventually fills up and things start failing. Should we use Global­Add­Atom and Global­Delete­Atom instead of Register­Window­Message? Or can we use Global­Delete­Atom to delete the message registered by Register­Window­Message?"

    No, you should not use Global­Add­Atom to create window messages. The atom that comes back from Global­Add­Atom comes from the global atom table, which is different from the registered window message table. The only way to get registered window messages is to call Register­Window­Message. Say you call Global­Add­Atom("X") and you get atom 49443 from the global atom table. Somebody else calls Register­Window­Message("Y") and they get registered window message number 49443. You then post message 49443 to a window, and it thinks that it is message Y, and bad things happen.

    And you definitely should not use Global­Delete­Atom in a misguided attempt to deregister a window message. You're going to end up deleting some unrelated atom, and things will start going downhill.

    What you need to do is fix the component so it does not register a lot of window messages with constantly-changing names. Instead, encode the uniqueness in some other way. For example, instead of registering a hundred messages of the form Contoso user N logged on, just register a single Contoso user logged on message and encode the user number in the wParam and lParam payloads. Most likely, one or the other parameter is already being used to carry nontrivial payload information, so you can just add the user number to that payload. (And this also means that your program won't have to keep a huge table of users and corresponding window messages.)

    Bonus chatter: It is the case that properties added to a window via Set­Prop use global atoms, as indicated by the documentation. This is an implementation detail that got exposed, so now it's contractual. And it was a bad idea, as I discussed earlier.

    Sometimes, people try to get clever and manually manage the atoms used for storing properties. They manually add the atom, then access the property by atom, then remove the properties, then delete the atom. This is a high-risk maneuver because there are so many things that can go wrong. For example, you might delete the atom prematurely (unaware that it was still being used by some other window), then the atom gets reused, and now you have a property conflict. Or you may have a bug that calls Global­Delete­Atom for an atom that was not obtained via Global­Add­Atom. (Maybe you got it via Global­Find­Atom or Enum­Props.)

    I've even seen code that does this:

    atom = GlobalAddAtom(name);
    // Some apps are delete-happy and run around deleting atoms they shouldn't.
    // If they happen to delete ours by accident, things go bad really fast.
    // Prevent this from happening by bumping the atom refcount a few extra
    // times so accidental deletes won't destroy it.

    So we've come full circle. There is a way to delete an unused atom, but people end up deleting them incorrectly, so this code tries to make the atom undeletable. Le Chatelier's Principle strikes again.

  • The Old New Thing

    Microspeak: Headcount, req, and related personnel terms


    For some reason, there are a lot of Microspeak terms related to personnel. (Maybe you folks can tell me how common these terms are outside Microsoft.)

    We start with a term that is not actually used much outside the personnel world: The Position Control Number, or PCN. The PCN represents a place where an employee could be hired. If somebody is actually hired for the position, then the PCN is filled; if not, then it is unfilled.

    The term you are likely to hear outside of the personnel world is headcount. (Pronounced as the two words head count, accent on the first word.) This is a filled PCN, and it is often abbreviated to just head.

    Another term you are likely to hear is a req, short for requisition, and pronounced like the word wreck. A req is a requisition to recruit; in other words, it is permission to look for somebody to fill a position.

    We have an open req to find somebody to frob the whatsit so it can futz the doodad.

    An open req is a req that has not yet been filled. This sounds redundant to me, because a req by definition is unfilled, isn't it?

    Yet another personnel term you may encounter is backfill. This refers to hiring someone to take over a position that has been vacated by somebody who left the team. You will sometimes hear the term used in a metaphorical context.

    Who is the backfill for Bob while he is on vacation?

    Bob has not actually left the team; the person merely wants to know who is covering Bob's responsibilities while he is on vacation.

    The last term I'm going to expose you to is the ROP, or Recruiting Only Position. A ROP is permission to interview someone for a position that doesn't exist yet. You open a ROP with a particular person in mind, and once obtained, you have permission to interview them. You can think of a ROP as unapproved headcount, since if you decide to hire the person, you still have to find a PCN to put them in. And if you decide not to hire the person after you interviewed them, you close the ROP.

    I have no idea how useful these terms are for people not in the personnel world, but I figured I'd write them down for my own benefit, so I have something to refer to when I run across them.

  • The Old New Thing

    Debugging walkthrough: Access violation on nonsense instruction, episode 2


    A colleague of mine asked for help debugging a strange failure. Execution halted on what appeared to be a nonsense instruction.

    eax=0079f850 ebx=00000000 ecx=00000113 edx=00000030 esi=33ee06ef edi=74b9b8ad
    eip=00c0ac74 esp=0079f82c ebp=0079f86c iopl=0         nv up ei pl zr na pe nc
    cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
    00c0ac74 0000            add     byte ptr [eax],al          ds:002b:0079f850=74

    If you've been debugging x86 code for a while, you immediately recognize this instruction as "executing a page of zeroes". If you haven't been debugging x86 code for a while, you can see this from the code bytes in the second column.

    So how did we end up at this nonsense instruction?

    The instruction is not near a page boundary, so we didn't fall through to it. We must have jumped to it or returned to it.

    Since debugging is an exercise in optimism, let's assume that we jumped to it via a call instruction, and the return address is still on the stack.

    0:000> dps esp l2
    0079f82c  74b9b8b1 user32!GetMessageW+0x4
    0079f830  008f108b CONTOSO!MessageLoop+0xe7
    0:000> u user32!GetMessageW l3
    74b9b8ad cc              int     3
    74b9b8ae ff558b          call    dword ptr [ebp-75h]
    74b9b8b1 ec              in      al,dx

    Well, that explains it. The code bytes for the Get­MessageW function were overwritten, causing us to execute garbage, and one of the garbage instructions was a call that took us to page of zeroes.

    But look more closely at the overwritten bytes.

    The first byte is cc, which is a breakpoint instruction. Hm...

    Since Windows functions begin with a MOV EDI, EDI instruction for hot patching purposes, the first two bytes are always 8b ff. If we unpatch the cc to 8b, we see that the rest of the code bytes are intact.

    74b9b8ad 8bff            mov     edi,edi
    74b9b8af 55              push    ebp
    74b9b8b0 8bec            mov     ebp,esp

    After a brief discussion, we were able to piece together what happened:

    Somebody was trying to debug the CONTOSO application, so they connected a user-mode debugger to the application. Meanwhile, they set a breakpoint on user32!GetMessageW from the kernel debugger. Setting a breakpoint in a debugger is typically performed by patching an int 3 at the point where you want the breakpoint. When the int 3 fires, the debugger regains control and says, "Oh, thanks for stopping. Let me unpatch all the int 3's I put in the program to put things back the way they were."

    When the breakpoint hit, it was caught by the user-mode debugger, but since the user-mode debugger didn't set that breakpoint, it interpreted the int 3 as a hard-coded breakpoint in the application. At this point, the developer saw a spurious breakpoint, didn't know what it meant, and simply resumed execution. This executed the second half of the MOV EDI, EDI instruction as the start of a new instruction, and havoc ensued.

    That developer then asked his friend what happened, and his friend asked me.

    TL;DR: Be careful if you have more than one debugger active. Breakpoints set by one debugger will not be recognized by the other. If the breakpoint instruction is caught by the wrong debugger, things will go downhill fast unless you take corrective action. (In this case, it would be restoring the original byte.)

  • The Old New Thing

    Dubious security vulnerability: Copying a program and running the copy


    This wasn't an actual security vulnerability report, but it was inspired by one. "If you take the program XYZ.EXE and you rename it or copy it to a new name that contains the letters XYX, then you can trigger a buffer overflow in the renamed/copied version of XYZ.EXE due to a bug in the way it parses its own file name in order to generate the names of its auxiliary files."

    While that's a bug, and thanks for pointing it out, it is not a security issue because there is no elevation of privilege. Sure, you could rename or copy the program and run it, but if you have permission to do that, you may as well do it the easy way: Instead of copying XYZ.EXE and running it, just copy pwnz0rd.exe and run it! Either way, it's just a case of you attacking yourself. You did not gain any privileges.

    Renaming or copying a file requires FILE_ADD_FILE permission in the destination directory, and if you have permission to add files to a directory, why stop at just adding files that are copies of existing files? You can add entirely new files!

    In other words, instead of copy XYZ.EXE XYX.EXE, just do copy pwnz0rd.exe XYX.EXE.

    This is a variation of the dubious vulnerability known as Code execution results in code execution.

    Now, this would be an actual vulnerability if you could somehow redirect attempts by other people to run XYZ.EXE from the original to your alternate XYX.EXE instead. But that would be attacking the redirection code, not attacking XYZ.EXE itself. Because if you can fool somebody into running XYX.EXE instead of XYZ.EXE, then you may as well fool them into running pwnz0rd.exe. It's not like the Create­Process function performs a hard drive scan looking for a program whose name is similar to the one you requested and running that other program instead.

  • The Old New Thing

    The more times you use the word "simply" in your instructions, the more I suspect you don't know what that word means


    I was helping somebody look up how to enable frobbing for widgets, and I found one set of instructions on a blog somewhere. To be honest, this happened long enough ago that I forgot what it was exactly, but here's something that captures the general spirit:

    First, check whether your widget supports frobbing. To do this, simply run this command

    magic ppg=q-40 id=voodoo xyzzy:42

    where voodoo is the voodoo code for your widget. It will say "frob supported" if your widget supports frobbing.

    If you don't know your widget's voodoo code, you can get a list of the voodoo codes and enchantment numbers for all the widgets connected to your computer by simply typing

    yoda PHASERS=warp10

    and then using the voodoo code in the first command line above.¹

    Once you have confirmed that your widget supports frobbing, you can enable it by simply editing the widget configuration file abc and adding frob="1" to the attributes of the appropriate entry. (If there is an existing frob="0", then simply change the 0 to a 1.)

    The changes will take effect at the next reboot. To make them take effect immediately, simply run the command

    episkey GANDALF.color=black DRADIS=pikachu

    My reaction was "Wow, this is really complicated. I have no idea how a normal human being is expected to know how to do this." And each time the next step in the process was revealed, my bewilderment increased.

    What struck me more was that the instructions used the word "simply" a lot. It became clear that the person writing the article was living in a world different from me. To me, the simple way to accomplish the task would have been if frobbing were enabled automatically if the hardware supported it. If there is some downside to frobbing, say, because it makes the widget run slower or use more power, then the simple way would have been to check a checkbox somewhere saying "Enable frobbing".

    But this person lived in a world where dropping to a command prompt, running a magic command, extracting the right voodoo code from the cryptic output, running a second magic command, then editing a configuration file, and then running a third magic command for the changes to take effect is a perfectly simple operation.

    I have to confess that I am guilty of this as well, where I dismiss various Win32 concepts as obvious, but my excuse is that my intended audience is developers who are already familiar with Win32, and for whom these sorts of things should be simple and obvious, because I'm trying to move past the basic concepts and discuss something more advanced.

    I do have entries with a non-technical audience in mind. Those entries are typically tagged Tips/Support and usually come out on Tuesdays. In those entries, I try to remember to dial things back. I suspect I don't always succeed.

    ¹ If there is more than one widget connected to your computer, then there will be more than one voodoo code. The instructions didn't say how to tell which voodoo code corresponds to which widget. Perhaps it was so simple it didn't need to be explained.

  • The Old New Thing

    Why does the mouse cursor jump a few pixels if you click on the very bottom row of pixels on the taskbar?


    ABCDSchuetze discovered that if you click on the very bottom row of pixels of the taskbar, the mouse cursor jumps up a few pixels. Why is that?

    In order to take advantage of Fitts's Law, the bottom-most row of pixels on a bottom-docked taskbar are clickable. Even though they are technically on the dead border of the window, the taskbar redirects the click to the button immediately above the border. But then you have this problem:

    • User clicks on the border pixel between the button and the edge of the screen.
    • The taskbar remaps the click to the button, thereby activating the button.
    • The button takes capture because that's what buttons do when you click on them. This allows you to drag off the button to cancel the click.
    • But wait: Since the mouse is on the border, it is already outside the button.
    • Result: The button cancels immediately.

    The short version: Clicking on the Fitts's edge causes the button to be pressed and then immediately canceled.

    The fix is to nudge the mouse back inside the button when the click begins.

  • The Old New Thing

    When asking about the capacity of a program, you also need to consider what happens when you decide to stop using the program


    An internal customer had a question about a tool, let's call it Program Q.

    As part of our gated checkin system, the system creates a new record in a Program Q table to record details of the checkin. (What tests were run, who did the code review, that sort of thing.) We are considering incorporating the Program Q record number as part of our version number: major.minor.service­pack.hot­fix.record­number. What is the maximum number of records per table supported by Program Q?

    Now, the easy way out is to just answer the question: The theoretical maximum number of records per table is 2³²−1. Even if your gated checkin system's throughput is one checkin per second, that gives you over a century of record numbers.

    But answering the question misses the big picture: The limiting factor is not the capacity of Program Q. The limiting factor is how long you plan to keep using Program Q! Before the century is up, you probably won't be using Program Q. What is your transition plan?

    In this case, it's probably not that complicated. Suppose that at the time Program Q is retired and replaced with Program R, the highest record number is 314159. You could just say that the version number in the binary is the Program R record number plus 400000.

    If you're clever, you can time the switch from Program Q to Program R to coincide with a change to the major.minor.service­pack.hot­fix, at which point you can reset the record­number to 1.

  • The Old New Thing

    Microspeak: Light up


    In Microspeak, a feature lights up if it becomes available in an application when run on an operating system that supports it.

    The idea is that you write your application to run on, say, Windows versions N and N + 1. There is a new feature in Windows version N + 1, and new functionality in the application when the code detects that the underlying Windows feature is availble.

    Here are a few citations:

    I have had some requests lately for details on the path to Windows 7 compatibility and how to light up key features.
    Top 7 Ways to light Up Your Apps on Windows Server 2008.

    The idea is that the program takes advantage of new Windows features when running on new versions of Windows. if run on older versions of Windows without those features, nothing happens.

    Inside the product group, discussion about "lighting up" often takes the form of deciding how much new hotness will be automatically applied to old applications and how much needs to be explicitly opted in. Applying new features to old applications makes the old applications and the new feature more valuable because the user sees the feature everywhere, as opposed to working only in new applications designed to take advantage of it. On the other hand, applying them automatically to old applications creates a compatibility risk, because the application may observe a behavior that hadn't occurred before, and it may begin behaving erratically as a result.

  • The Old New Thing

    Why was the replacement installer for recognized 16-bit installers itself a 32-bit program instead of a 64-bit program?


    Even though 64-bit Windows does not support 16-bit applications, there is a special case for 16-bit installers for 32-bit applications. Windows detects this scenario and substitutes a 32-bit replacement installer which replicates the actions of the 16-bit installer. Commenter Karellen is horrified at the fact that the replacement installer is a 32-bit program. "You're writing a program that will run exclusively on 64-bit systems. Why not built it to run natively on the OS it's designed for? Why is this apparently not the "obvious" Right Thing(tm) to do? What am I missing?"

    Recall that a science project is a programming project that is technically impressive but ultimately impractical. For example it might be a project that nobody would actually use, or it attempts to add a Gee-Whiz feature that nobody is really clamoring for.

    But at least a science project is trying to solve a problem. This proposal doesn't even solve any problems! Indeed, this proposal creates problems. One argument in favor of doing it this way is that it is satisfies some obsessive-compulsive requirement that a 64-bit operating system have no 32-bit components beyond the 32-bit emulation environment itself.

    Because! Because you're running a 64-bit system, and running apps native to that system is just more elegant.

    Okay, it's not obsessive-compulsive behavior. It's some sort of aesthetic ideal, postulated for its own sake, devoid of practical considerations.

    Remember the problem space. We have a bunch of 32-bit applications that use a 16-bit installer. Our goal is to get those applications installed on 64-bit Windows. By making the replacement installer a 32-bit program, you get the emulator to do all the dirty work for you. Things like registry redirection, file system redirection, and 32-bit application compatibility.

    Suppose the original installer database says

    • Copy X.DLL file into the %Program­Files%\AppName directory.
    • Copy Y.DLL into the %windir%\System32 directory.
    • If the current version of C:\Program Files\Common Files\Adobe\Acrobat\ActiveX\AcroPDF.dll is 7.5 or higher, then set this registry key.

    If you write the replacement installer as a 32-bit program, then other parts of the 32-bit emulation engine do the work for you.

    • The environment manager knows that 64-bit processes get the environment variable Program­Files pointing to C:\Program Files, whereas 32-bit processes get Program­Files pointing to C:\Program Files (x86).
    • The file system redirector knows that if a 32-bit process asks for %windir%\System32, it should really get %windir%\SysWOW64.
    • The registry redirector knows that if a 32-bit process tries to access certain parts of the registry, they should be sent to the Wow­64­32­Node instead.

    If you had written the replacement installer as a 64-bit program, you would have to replicate all of these rules and make sure your copy of the rules exactly matched the rules used by the real environment manager, file system redirector, and registry redirector.

    Now you have to keep two engines in sync: the 32-bit emulation engine and the 64-bit replacement installer for 32-bit applications. This introduces fragility, because any behavior change in the 32-bit emulation engine must be accompanied by a corresponding change in the 64-bit replacement installer.

    Suppose the application compatibility folks add a rule that says, "If a 32-bit installer tries to read the version string from C:\Program Files\Common Files\Adobe\Acrobat\ActiveX\AcroPDF.dll, return the version string from C:\Program Files (x86)\Common Files\Adobe\Acrobat\ActiveX\AcroPDF.dll instead." And suppose that rule is not copied to the 64-bit replacement installer. Congratulations, your 64-bit replacement installer will incorrectly install any program that changes behavior based on the currently-installed version of AcroPDF.

    I don't know for sure, but I wouldn't be surprised if some of these installers support plug-ins, so that the application developer can run custom code during installation. It is possible for 16-bit applications to load 32-bit DLLs via a technique known as generic thunking, and the 16-bit stub installer would use a generic thunk to call into the 32-bit DLL to do whatever custom action was required. On the other hand, 64-bit applications cannot load 32-bit DLLs, so if the 64-bit replacement installer encountered a 32-bit DLL plug-in, it would have to run a 32-bit helper application to load the plug-in and call into it. So you didn't escape having a 32-bit component after all.

    And the original obsessive-compulsive reason for requiring the replacement installer to be 64-bit was flawed anyway. This is a replacement installer for a 32-bit application. Therefore, the replacement installer is part of the 32-bit emulation environment, so it is allowed to be written as a 32-bit component.

    Let's look at the other arguments given for why the replacement installer for a 32-bit application should be written as a 64-bit application.

    Because complexity is what will be our undoing in the end, and reducing it wherever we can is always a win.

    As we saw above, writing the replacement installer as a 64-bit application introduces complexity. Writing it as a 32-bit application reduces complexity. So this statement itself argues for writing the replacement installer as a 32-bit application.

    Because we can't rewrite everything from scratch at once, but we can create clean new code one small piece at a time, preventing an increase to our technical debt where we have the opportunity to do so at negligible incremental cost to just piling on more cruft.

    As noted above, the incremental cost is hardly negligible. Indeed, writing the replacement installer as a 64-bit application is not merely more complex, it creates an ongoing support obligation, because any time there is a change to the 32-bit emulation environment, that change needs to be replicated in the 64-bit replacement installer. This is a huge source of technical debt: Fragile coupling between two seemingly-unrelated components.

    And writing the replacement installer as a 32-bit application does not create a future obligation to port it to 64 bits when support for 32-bit applications is dropped in some future version of Windows. Because when support for 32-bit applications disappears (as it already has on Server Core), there will be no need to port the replacement installer to 64-bit because there's no point writing an installer for a program that cannot run!

    Writing the replacement installer as a 32-bit program was the right call.

Page 1 of 96 (952 items) 12345»