• The Old New Thing

    CancelIoEx can cancel I/O on console input, which is kind of nice

    • 24 Comments

    Today's Little Program asks you to type something, but gives you only two seconds to do it. This is not interesting in and of itself, but it shows you how to cancel console I/O. There is no motivation for this exercise because Little Programs come with little to no motivation.

    Okay, fine, here's the motivation.

    We have a GUI application that has a debug console. When the user exits the application, we cannot shut down cleanly because the debug console is stuck on a read from stdin. We want to unstick the thread gently. We don't want to use Generate­Console­Ctrl­Event with CTRL_C_EVENT because that will send the event to all processes using the same console, but we don't want other processes to be affected.

    Okay, now our Little Program.

    #include <windows.h>
    #include <stdio.h> // horrors! mixing C and C++!
    
    DWORD CALLBACK ThreadProc(void *)
    {
     Sleep(2000);
     CancelIoEx(GetStdHandle(STD_INPUT_HANDLE), nullptr);
     return 0;
    }
    
    int __cdecl wmain(int, wchar_t **)
    {
     DWORD scratch;
     HANDLE h = CreateThread(nullptr, 0, ThreadProc,
                             nullptr, 0, &scratch);
     if (h) {
      printf("type something\n");
      char buffer[80];
      if (fgets(buffer, 80, stdin) != nullptr) {
       printf("you typed %s", buffer);
      } else if (feof(stdin)) {
       printf("end-of-file reached\n");
      } else if (ferror(stdin)) {
       printf("error occurred\n");
      }
     }
     return 0;
    }
    

    If you type something within two seconds, it is reported back to you, but if you take too long, then the Cancel­Io­Ex cancels the console read, and you get an error back.

    If you want to continue, you'll have to clearerr(stdin), but if you just want to unstick the code that is performing the read (so that you can get the program to exit cleanly), then leaving stdin in an error state is probably better.

    (If you had used Read­File instead of fgets, the read would have failed with error code ERROR_OPERATION_ABORTED, as documented by Cancel­Io­Ex.)

  • The Old New Thing

    Finding the constructor by scanning memory for the vtable

    • 5 Comments

    In Looking for leaked objects by their vtable, we used the object's constructor to locate the vtable, and then scanned the heap for the vtable to find the leaked object. But you can run this technique in reverse, too.

    Suppose you found an object and you want to find its constructor. This is not a problem if you have the source code, but if you are doing some reverse-engineering for application compatibility purposes, you don't have the luxury of the application source code. You may have figured out that the application fails because the byte at offset 0x50 is zero, but on the previous version of Windows, it was nonzero. You want to find out who sets the byte at offset 0x50, so that you can see why it is setting it to zero instead of a nonzero value.

    If the object has a vtable, you can scan the code segments for a copy of the vtable. It will show up in an instruction like

    mov dword ptr [reg], vtable_address
    

    This is almost certainly the object's constructor, setting up the object vtable as part of construction. You can set a breakpoint here to break when the object is constructed, and then you can set a write breakpoint on offset 0x50 to see where its value is seto.

  • The Old New Thing

    Sure, we have RegisterWindowMessage and RegisterClipboardFormat, but where are DeregisterWindowMessage and DeregisterClipboardFormat?

    • 23 Comments

    The Register­Window­Message function lets you create your own custom messages that are globally unique. But how do you free the message format when you're done, so that the number can be reused for another message? (Similarly, Register­Clipboard­Format and clipboard formats.)

    You don't. There is no Deregister­Window­Message function or Deregister­Clipboard­Format function. Once allocated, a registered window message and registered clipboard format hangs around until you log off.

    There is room for around 16,000 registered window messages and registered clipboard formats, and in practice exhaustion of these pools of numbers is not an issue. Even if every program registers 100 custom messages, you can run 160 unique programs before running into a problem. And most people don't even have 160 different programs installed in the first place. (And if you do, you almost certainly don't run all of them!) In practice, the number of registered window messages is well under 1000.

    A customer had a problem with exhaustion of registered window messages. "We are using a component that uses the Register­Window­Message function to register a large number of unique messages which are constantly changing. Since there is no way to unregister them, the registered window message table eventually fills up and things start failing. Should we use Global­Add­Atom and Global­Delete­Atom instead of Register­Window­Message? Or can we use Global­Delete­Atom to delete the message registered by Register­Window­Message?"

    No, you should not use Global­Add­Atom to create window messages. The atom that comes back from Global­Add­Atom comes from the global atom table, which is different from the registered window message table. The only way to get registered window messages is to call Register­Window­Message. Say you call Global­Add­Atom("X") and you get atom 49443 from the global atom table. Somebody else calls Register­Window­Message("Y") and they get registered window message number 49443. You then post message 49443 to a window, and it thinks that it is message Y, and bad things happen.

    And you definitely should not use Global­Delete­Atom in a misguided attempt to deregister a window message. You're going to end up deleting some unrelated atom, and things will start going downhill.

    What you need to do is fix the component so it does not register a lot of window messages with constantly-changing names. Instead, encode the uniqueness in some other way. For example, instead of registering a hundred messages of the form Contoso user N logged on, just register a single Contoso user logged on message and encode the user number in the wParam and lParam payloads. Most likely, one or the other parameter is already being used to carry nontrivial payload information, so you can just add the user number to that payload. (And this also means that your program won't have to keep a huge table of users and corresponding window messages.)

    Bonus chatter: It is the case that properties added to a window via Set­Prop use global atoms, as indicated by the documentation. This is an implementation detail that got exposed, so now it's contractual. And it was a bad idea, as I discussed earlier.

    Sometimes, people try to get clever and manually manage the atoms used for storing properties. They manually add the atom, then access the property by atom, then remove the properties, then delete the atom. This is a high-risk maneuver because there are so many things that can go wrong. For example, you might delete the atom prematurely (unaware that it was still being used by some other window), then the atom gets reused, and now you have a property conflict. Or you may have a bug that calls Global­Delete­Atom for an atom that was not obtained via Global­Add­Atom. (Maybe you got it via Global­Find­Atom or Enum­Props.)

    I've even seen code that does this:

    atom = GlobalAddAtom(name);
    
    // Some apps are delete-happy and run around deleting atoms they shouldn't.
    // If they happen to delete ours by accident, things go bad really fast.
    // Prevent this from happening by bumping the atom refcount a few extra
    // times so accidental deletes won't destroy it.
    GlobalAddAtom(name);
    GlobalAddAtom(name);
    

    So we've come full circle. There is a way to delete an unused atom, but people end up deleting them incorrectly, so this code tries to make the atom undeletable. Le Chatelier's Principle strikes again.

  • The Old New Thing

    Travel tip: Don't forget your car on the ferry

    • 26 Comments

    One of my colleagues lives on Bainbridge Island and has quite a long commute to work each day. From his house, he walks to the bus stop, then takes the bus to the Winslow ferry terminal, then takes the ferry to the Seattle ferry terminal, then takes the bus to Microsoft. And at the end of the day, he does the trip in reverse.

    One day, for whatever reason, he drove to work instead of taking the bus. He drove to the ferry terminal, took the ferry across, then drove to work. And at the end of the day, he drove to the ferry, but when the ferry arrived at its destination, he forgot that he had driven his car and walked off the boat to the bus.

    While on his way home on the bus, he got a phone call from his wife. "Did you forget your car on the ferry, dear?"

    Oops.

    Now, leaving your car on the ferry is a bad thing not just because your car is now an obstacle on the ferry deck which all the other drivers must maneuver around. When there is an abandoned car on the deck, one of the possibilities that must be investigated is that a passenger has fallen overboard.

    The crew took the ship offline, conducted a search of the vessel, and initiated a search-and-rescue operation along the ferry route, looking for a body floating in the water.

    My colleague had to sheepishly call the ferry authorities and say, "Hello, I believe you're looking for me."

    (Today is Transit Driver Appreciation Day, but I don't think your ferry captain will complain if you thank him/her, too.)

  • The Old New Thing

    Microspeak: Headcount, req, and related personnel terms

    • 16 Comments

    For some reason, there are a lot of Microspeak terms related to personnel. (Maybe you folks can tell me how common these terms are outside Microsoft.)

    We start with a term that is not actually used much outside the personnel world: The Position Control Number, or PCN. The PCN represents a place where an employee could be hired. If somebody is actually hired for the position, then the PCN is filled; if not, then it is unfilled.

    The term you are likely to hear outside of the personnel world is headcount. (Pronounced as the two words head count, accent on the first word.) This is a filled PCN, and it is often abbreviated to just head.

    Another term you are likely to hear is a req, short for requisition, and pronounced like the word wreck. A req is a requisition to recruit; in other words, it is permission to look for somebody to fill a position.

    We have an open req to find somebody to frob the whatsit so it can futz the doodad.

    An open req is a req that has not yet been filled. This sounds redundant to me, because a req by definition is unfilled, isn't it?

    Yet another personnel term you may encounter is backfill. This refers to hiring someone to take over a position that has been vacated by somebody who left the team. You will sometimes hear the term used in a metaphorical context.

    Who is the backfill for Bob while he is on vacation?

    Bob has not actually left the team; the person merely wants to know who is covering Bob's responsibilities while he is on vacation.

    The last term I'm going to expose you to is the ROP, or Recruiting Only Position. A ROP is permission to interview someone for a position that doesn't exist yet. You open a ROP with a particular person in mind, and once obtained, you have permission to interview them. You can think of a ROP as unapproved headcount, since if you decide to hire the person, you still have to find a PCN to put them in. And if you decide not to hire the person after you interviewed them, you close the ROP.

    I have no idea how useful these terms are for people not in the personnel world, but I figured I'd write them down for my own benefit, so I have something to refer to when I run across them.

  • The Old New Thing

    Does the CLR really call CoInitializeEx on the first call to unmanaged code, even if you don't deal with COM at all and are just calling native code via p/invoke?

    • 13 Comments

    Some time ago, I called out this part of the documentation regarding managed and unmanaged threading:

    On the first call to unmanaged code, the runtime calls Co­Initialize­Ex to initialize the COM apartment as either an MTA or an STA apartment. You can control the type of apartment created by setting the System.Threading.ApartmentState property on the thread to MTA, STA, or Unknown.

    Commenter T asks, "Does it do this even if you don't deal with COM at all and call native code through a P/Invoke?"

    Well, the documentation says it does, and we can confirm with an experiment:

    using System.Runtime.InteropServices;
    
    class Program
    {
     public static void Main()
     {
      var thread = new System.Threading.Thread(
        () => {
       System.Console.WriteLine("about to p/invoke");
       GetTickCount();
      });
      thread.Start();
      thread.Join();
     }
    
     [DllImport("kernel32.dll")]
     extern static uint GetTickCount();
    }
    

    Run this program with a breakpoint on Co­InitializeEx.

    First breakpoint is hit with this stack:

    rax=00007ffebc529b70 rbx=00000000007c6100 rcx=0000000000000000
    rdx=0000000000000000 rsi=0000000000000001 rdi=0000000000000002
    rip=00007ffebc529b70 rsp=000000000056f038 rbp=000000000056f0b0
     r8=0000000000000001  r9=0000000000000000 r10=0000000000000000
    r11=0000000000000037 r12=0000000000004000 r13=0000000000000001
    r14=0000000000000001 r15=0000000000000001
    
    combase!CoInitializeEx
    clr!Thread::SetApartment
    clr!SystemDomain::SetThreadAptState
    clr!SystemDomain::ExecuteMainMethod
    clr!ExecuteEXE
    clr!_CorExeMainInternal
    clr!CorExeMain
    mscoreei!CorExeMain
    MSCOREE!CorExeMain_Exported
    KERNEL32!BaseThreadInitThunk
    ntdll!RtlUserThreadStart
    

    This call is initializing the main thread of the process. The flags passed to this first call to Co­Initialize­Ex are 0, which means that the default threading model of COINIT_MULTI­THREADED is used.

    The next time the breakpoint hits is with this stack:

    rax=00000000ffffffff rbx=00000000007d1180 rcx=0000000000000000
    rdx=0000000000000000 rsi=0000000000000001 rdi=00000000007d1180
    rip=00007ffebc529b70 rsp=000000001a6af9a8 rbp=000000001a6afa20
     r8=000000001a6af948  r9=0000000000000000 r10=00000000007f0340
    r11=00000000007f0328 r12=0000000000004000 r13=0000000000000000
    r14=0000000000000000 r15=0000000000000000
    
    combase!CoInitializeEx
    clr!Thread::SetApartment
    clr!Thread::DoExtraWorkForFinalizer
    clr!WKS::GCHeap::FinalizerThreadWorker
    clr!ManagedThreadBase_DispatchInner
    clr!ManagedThreadBase_DispatchMiddle
    clr!ManagedThreadBase_DispatchOuter
    clr!WKS::GCHeap::FinalizerThreadStart
    clr!Thread::intermediateThreadProc
    KERNEL32!BaseThreadInitThunk
    ntdll!RtlUserThreadStart
    

    From the name Finalizer­Thread­Start, this is clearly the finalizer thread.¹

    Next.

    rax=00000000ffffffff rbx=000000000039eb20 rcx=0000000000000000
    rdx=0000000000000000 rsi=0000000000000001 rdi=0000000000000000
    rip=00007ffebc529b70 rsp=000000001a5af3d8 rbp=000000001a5af450
     r8=0000000000000000  r9=000000001a5af3f0 r10=0000000000000000
    r11=0000000000000286 r12=0000000000004000 r13=0000000000000000
    r14=0000000000000000 r15=0000000000000000
    
    combase!CoInitializeEx
    clr!Thread::SetApartment
    clr!Thread::PrepareApartmentAndContext
    clr!Thread::HasStarted
    clr!ThreadNative::KickOffThread
    clr!Thread::intermediateThreadProc
    KERNEL32!BaseThreadInitThunk
    ntdll!RtlUserThreadStart
    

    Okay, this looks like it's kicking off a new thread. I inferred this from the presence on the stack of the function which is deviously named Kick­Off­Thread.

    And the flags passed to this call to Co­Initialize­Ex are 0, which once again means that it defaults to MTA.

    There, we have confirmed experimentally that, at least in this case, the implementation matches the documentation.

    That the implementation behaves this way is not surprising. After all, the CLR does not have insight into the Get­Tick­Count function. It does not know a priori whether that function will create any COM objects. After all, we could have been p/invoking to SHGet­Desktop­Folder, which does use COM. Given that the CLR cannot tell whether a native function is going to use COM or not, it has to initialize COM just in case.

    ¹ Or somebody who is trying to mislead us into thinking that it is the finalizer thread. I tend to discount this theory because as a general rule, code is not intentionally written to be impossible to understand.

  • The Old New Thing

    Debugging walkthrough: Access violation on nonsense instruction, episode 2

    • 17 Comments

    A colleague of mine asked for help debugging a strange failure. Execution halted on what appeared to be a nonsense instruction.

    eax=0079f850 ebx=00000000 ecx=00000113 edx=00000030 esi=33ee06ef edi=74b9b8ad
    eip=00c0ac74 esp=0079f82c ebp=0079f86c iopl=0         nv up ei pl zr na pe nc
    cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
    00c0ac74 0000            add     byte ptr [eax],al          ds:002b:0079f850=74
    

    If you've been debugging x86 code for a while, you immediately recognize this instruction as "executing a page of zeroes". If you haven't been debugging x86 code for a while, you can see this from the code bytes in the second column.

    So how did we end up at this nonsense instruction?

    The instruction is not near a page boundary, so we didn't fall through to it. We must have jumped to it or returned to it.

    Since debugging is an exercise in optimism, let's assume that we jumped to it via a call instruction, and the return address is still on the stack.

    0:000> dps esp l2
    0079f82c  74b9b8b1 user32!GetMessageW+0x4
    0079f830  008f108b CONTOSO!MessageLoop+0xe7
    0:000> u user32!GetMessageW l3
    USER32!GetMessageW:
    74b9b8ad cc              int     3
    74b9b8ae ff558b          call    dword ptr [ebp-75h]
    74b9b8b1 ec              in      al,dx
    

    Well, that explains it. The code bytes for the Get­MessageW function were overwritten, causing us to execute garbage, and one of the garbage instructions was a call that took us to page of zeroes.

    But look more closely at the overwritten bytes.

    The first byte is cc, which is a breakpoint instruction. Hm...

    Since Windows functions begin with a MOV EDI, EDI instruction for hot patching purposes, the first two bytes are always 8b ff. If we unpatch the cc to 8b, we see that the rest of the code bytes are intact.

    USER32!GetMessageW:
    74b9b8ad 8bff            mov     edi,edi
    74b9b8af 55              push    ebp
    74b9b8b0 8bec            mov     ebp,esp
    

    After a brief discussion, we were able to piece together what happened:

    Somebody was trying to debug the CONTOSO application, so they connected a user-mode debugger to the application. Meanwhile, they set a breakpoint on user32!GetMessageW from the kernel debugger. Setting a breakpoint in a debugger is typically performed by patching an int 3 at the point where you want the breakpoint. When the int 3 fires, the debugger regains control and says, "Oh, thanks for stopping. Let me unpatch all the int 3's I put in the program to put things back the way they were."

    When the breakpoint hit, it was caught by the user-mode debugger, but since the user-mode debugger didn't set that breakpoint, it interpreted the int 3 as a hard-coded breakpoint in the application. At this point, the developer saw a spurious breakpoint, didn't know what it meant, and simply resumed execution. This executed the second half of the MOV EDI, EDI instruction as the start of a new instruction, and havoc ensued.

    That developer then asked his friend what happened, and his friend asked me.

    TL;DR: Be careful if you have more than one debugger active. Breakpoints set by one debugger will not be recognized by the other. If the breakpoint instruction is caught by the wrong debugger, things will go downhill fast unless you take corrective action. (In this case, it would be restoring the original byte.)

  • The Old New Thing

    Under what conditions will the IUnknown::AddRef method return 0?

    • 14 Comments

    A customer was debugging their application and discovered that for one of the objects they were using, the IUnknown::Add­Ref method returns 0. How is that possible? That would imply that the object's reference count was originally negative one?

    The return value from IUnknown::Add­Ref is the object reference count by convention, but

    This value is intended to be used only for test purposes.

    The return value is purely advisory and is not required to be accurate.

    For example, if the object is a proxy, it will most likely return the reference count of the local proxy rather than the raw reference count of the original object. Conversely, if you have an object with outstanding proxies, the IUnknown::Add­Ref will count only one reference per proxy, even if the proxies themselves have reference counts greater than one.

    The object the customer was using came from MSHTML.DLL, and it so happens that the implementation of IUnknown::Add­Ref used by that component always returns zero. It is technically within their rights to do so.

    I don't know for sure, but I suspect this is done on purpose to avoid applications relying on the exact reference count. Applications are known to do dubious things, such as call IUnknown:­Release in a loop until it says the reference count is zero. Making the objects return a value from IUnknown::Add­Ref that betrays no information about the object's true reference count may have been a defensive step to prevent applications from making any such dubious dependency.

    If you install the debugging version of MSHTML.DLL, then the IUnknown::Add­Ref method will return the reference count. Which makes sense in its own way because the value is intended to be used only when debugging.

  • The Old New Thing

    Dubious security vulnerability: Copying a program and running the copy

    • 30 Comments

    This wasn't an actual security vulnerability report, but it was inspired by one. "If you take the program XYZ.EXE and you rename it or copy it to a new name that contains the letters XYX, then you can trigger a buffer overflow in the renamed/copied version of XYZ.EXE due to a bug in the way it parses its own file name in order to generate the names of its auxiliary files."

    While that's a bug, and thanks for pointing it out, it is not a security issue because there is no elevation of privilege. Sure, you could rename or copy the program and run it, but if you have permission to do that, you may as well do it the easy way: Instead of copying XYZ.EXE and running it, just copy pwnz0rd.exe and run it! Either way, it's just a case of you attacking yourself. You did not gain any privileges.

    Renaming or copying a file requires FILE_ADD_FILE permission in the destination directory, and if you have permission to add files to a directory, why stop at just adding files that are copies of existing files? You can add entirely new files!

    In other words, instead of copy XYZ.EXE XYX.EXE, just do copy pwnz0rd.exe XYX.EXE.

    This is a variation of the dubious vulnerability known as Code execution results in code execution.

    Now, this would be an actual vulnerability if you could somehow redirect attempts by other people to run XYZ.EXE from the original to your alternate XYX.EXE instead. But that would be attacking the redirection code, not attacking XYZ.EXE itself. Because if you can fool somebody into running XYX.EXE instead of XYZ.EXE, then you may as well fool them into running pwnz0rd.exe. It's not like the Create­Process function performs a hard drive scan looking for a program whose name is similar to the one you requested and running that other program instead.

  • The Old New Thing

    The more times you use the word "simply" in your instructions, the more I suspect you don't know what that word means

    • 49 Comments

    I was helping somebody look up how to enable frobbing for widgets, and I found one set of instructions on a blog somewhere. To be honest, this happened long enough ago that I forgot what it was exactly, but here's something that captures the general spirit:

    First, check whether your widget supports frobbing. To do this, simply run this command

    magic ppg=q-40 id=voodoo xyzzy:42
    

    where voodoo is the voodoo code for your widget. It will say "frob supported" if your widget supports frobbing.

    If you don't know your widget's voodoo code, you can get a list of the voodoo codes and enchantment numbers for all the widgets connected to your computer by simply typing

    yoda PHASERS=warp10
    

    and then using the voodoo code in the first command line above.¹

    Once you have confirmed that your widget supports frobbing, you can enable it by simply editing the widget configuration file abc and adding frob="1" to the attributes of the appropriate entry. (If there is an existing frob="0", then simply change the 0 to a 1.)

    The changes will take effect at the next reboot. To make them take effect immediately, simply run the command

    episkey GANDALF.color=black DRADIS=pikachu
    

    My reaction was "Wow, this is really complicated. I have no idea how a normal human being is expected to know how to do this." And each time the next step in the process was revealed, my bewilderment increased.

    What struck me more was that the instructions used the word "simply" a lot. It became clear that the person writing the article was living in a world different from me. To me, the simple way to accomplish the task would have been if frobbing were enabled automatically if the hardware supported it. If there is some downside to frobbing, say, because it makes the widget run slower or use more power, then the simple way would have been to check a checkbox somewhere saying "Enable frobbing".

    But this person lived in a world where dropping to a command prompt, running a magic command, extracting the right voodoo code from the cryptic output, running a second magic command, then editing a configuration file, and then running a third magic command for the changes to take effect is a perfectly simple operation.

    I have to confess that I am guilty of this as well, where I dismiss various Win32 concepts as obvious, but my excuse is that my intended audience is developers who are already familiar with Win32, and for whom these sorts of things should be simple and obvious, because I'm trying to move past the basic concepts and discuss something more advanced.

    I do have entries with a non-technical audience in mind. Those entries are typically tagged Tips/Support and usually come out on Tuesdays. In those entries, I try to remember to dial things back. I suspect I don't always succeed.

    ¹ If there is more than one widget connected to your computer, then there will be more than one voodoo code. The instructions didn't say how to tell which voodoo code corresponds to which widget. Perhaps it was so simple it didn't need to be explained.

Page 3 of 447 (4,467 items) 12345»