October, 2013

  • The Old New Thing

    The guide to trading candy


    Important information to know this evening.

  • The Old New Thing

    If there is no 16-bit emulation layer in 64-bit Windows, how come certain 16-bit installers are allowed to run?


    Troy Martin is puzzled by the remark in this knowledge base article that says

    No 16-bit code can run, except for recognized InstallShield and Acme installers (these are hard-coded in Wow64 to allow them to work).

    I agree that the sentence is rather confusingly written. It says "No 16-bit code can run, except for this code that runs."

    But that's not really what's going on. No 16-bit code running at all.

    What the article is trying to say (as briefly as possible) is that Windows has specific knowledge about certain InstallShield and Acme installers that allows it to parse the installer's data files directly. That's what the article is trying to say with the word recognized: When you try to run a 16-bit program, the application compatibility layer looks at the program and says, "Gosh, I wonder if I recognize this program." And maybe the answer is, "Yes, it is InstallShield version 5.0, and I have been taught very specific information about the data files that are used by that version of InstallSheield to the point that I know how to install them without actually invoking InstallShield itself." In that case, instead of playing a sad sound, the kernel hands the request to the application compatibility engine with the instructions, "You take care of this."

    The application compatibility engine then substitutes a 32-bit custom installer that knows how to open, parse, and apply the InstallShield data files. Another way of looking at it is that somebody sat down and ported InstallShield to 32-bit Windows, so that when a user tries to run a 16-bit installer, the request is redirected to the 32-bit version.

    You can see that installer in your C:\Windows\SysWOW64\Install­Shield directory.

  • The Old New Thing

    Distinguishing between asking for help with a product and asking for help with a product's installation


    Internally at Microsoft, we have a programmer's tool which I will call Program Q. On the peer-to-peer mailing list for Program Q, somebody asked the following question:

    What's the best way to look at all tables created in the past week? I want to repeat this command across multiple table repositories.

    Somebody chimed in with the answer.

    You are looking for something like q find table -age lt 7d. If you want to repeat across multiple table repositories, you would say q -server xxx find table -age lt 7d, replacing xxx with the table repository server name.

    This seemed to work, but there was a follow-up question:

    What's the best way to get the repository server name for any given repository? For example, what are all the repository servers used by my test team? Is there a central database I can query?

    To get the repository servers used by your test team, you um ask your test team?

    The people on the Program Q peer-to-peer help mailing list do not know the names of your test team's repository servers. There is no central database of all Program Q servers.

    It's like asking, "What's the best way to get the name of somebody's SharePoint site?" Answer: You ask them, "Hey, what's the name of your SharePoint site?" SharePoint itself does not maintain a central database of all SharePoint sites, but your test team might have a policy that all new SharePoint sites should be registered in some central database. You'll need to ask them.

  • The Old New Thing

    Help me optimize this code which enumerates all possible GUIDs


    Hi, I'm looking for help optimizing this code. It creates a file containing every possible GUID.

    class Program {
     public static void Main(string[] args)
      using (var w = new System.IO.StreamWriter("all_guids.txt")) {
       for (long l1 = 0; l1 <= 0xFFFFFFFFL; l1++)
       for (long l2 = 0; l2 <= 0xFFFFFFFFL; l2++)
       for (long l3 = 0; l3 <= 0xFFFFFFFFL; l3++)
       for (long l4 = 0; l4 <= 0xFFFFFFFFL; l4++)
         new System.Guid(string.Format("{0:X8}{1:X8}{2:X8}{3:X8}",
                                       l1, l2, l3, l4)));

    I know this code will take a really long time to run, so any performance improvements would be appreciated.

    Okay, wait a second. You do realize that even if you had a 100GHz machine that could generate a new GUID every cycle, it will still take 2¹²⁸ ÷ 10¹¹ = 3 × 10²⁷ seconds or 10¹⁹ years to complete.

    Right, that's why I was looking for help making the code run faster.

    Look at this way: Suppose you could somehow get this algorithm to run a quintillion times faster, so it finishes in under a year. Your output file is going to be 2¹²⁸ × 16 = 2¹³² bytes in size. That's around 10²⁷ terabytes. One terabyte of SSD storage weighs around 100 grams. The mass of the earth is 10²⁴ kilograms. Therefore, before you run this program, you will need to acquire 100 earth-sized planets and convert them all to SSDs.

    What are you trying to do with all of these GUIDs?

    There's a Web site that uses GUIDs to identify resources. I want to enumerate all the GUIDs so I can request each one from the Web site to see if it contains anything interesting.

    Okay, so it doesn't matter how fast your GUID enumeration algorithm is, because that's not the rate-limiting factor in this entire exercise. The limiting factor is the speed of the Web server you are attacking. (And at this point, it's fair to say that what you're doing is attacking a Web server.) Even if the Web server can handle a billion hits per second, it will take 10²² years for it to respond to all your requests.

    Looking at it another way: Suppose you could enlist the resources of every computer on the Internet to send a million requests per second. It would still take over 10¹³ centuries for them to try every GUID.

    And the server administrators might suspect something is up after, oh, about seven centuries.

    This problem requires a sociological, not engineering, solution. You need to contact the site administrators, describe what information you would like, and see if you can come to some sort of data-sharing agreement.

    I've contacted the site owners, but they are not interested in helping me, so I have no choice but to try every GUID.

    Your computer is already plenty powerful enough to generate GUIDs as fast as the Web site can handle inbound requests. Maybe you can upgrade your cable modem to a higher level of service.

    Actually, what you need to do is upgrade the other site's hardware and Internet connection to a higher level of service. Because they are the bottleneck, not you. "Here's a half million dollars to get faster servers. Just a gift from me. No really, no strings attached. Have fun. Just get those faster servers online right away, okay?"

    I wish you good luck.

  • The Old New Thing

    The financial acumen of sea turtles


    I dreamed that I was attending some sort of "how to be awesome" seminar where the presenter said, among other things, that a sea turtle, when left to thrive undisturbed, amasses $1 million in personal wealth within one year.

  • The Old New Thing

    Using GetLogicalProcessorInformationEx to see the relationship between logical and physical processors


    Today's Little Program uses the Get­Logical­Processor­Information­Ex function to print the mapping of logical processors to physical processors, as well as the mapping of logical processors to packages. (A dual-core processor is a single package with two cores. If those cores are themselves dual-hyperthreaded, then you have four logical processors total.)

    #define STRICT
    #include <windows.h>
    #include <stdio.h>
    template<typename T>
    T *AdvanceBytes(T *p, SIZE_T cb)
     return reinterpret_cast<T*>(reinterpret_cast<BYTE *>(p) + cb);

    The Advance­Bytes helper function takes a typed pointer and adds a byte offset to it. This is just a typing-saver function.

    class EnumLogicalProcessorInformation
     EnumLogicalProcessorInformation(LOGICAL_PROCESSOR_RELATIONSHIP Relationship)
      : m_pinfoBase(nullptr), m_pinfoCurrent(nullptr), m_cbRemaining(0)
      DWORD cb = 0;
      if (GetLogicalProcessorInformationEx(Relationship,
                                           nullptr, &cb)) return;
      if (GetLastError() != ERROR_INSUFFICIENT_BUFFER) return;
      m_pinfoBase =
                                         (LocalAlloc(LMEM_FIXED, cb));
      if (!m_pinfoBase) return;
      if (!GetLogicalProcessorInformationEx(Relationship, 
                                            m_pinfoBase, &cb)) return;
      m_pinfoCurrent = m_pinfoBase;
      m_cbRemaining = cb;
     ~EnumLogicalProcessorInformation() { LocalFree(m_pinfoBase); }
     void MoveNext()
      if (m_pinfoCurrent) {
       m_cbRemaining -= m_pinfoCurrent->Size;
       if (m_cbRemaining) {
        m_pinfoCurrent = AdvanceBytes(m_pinfoCurrent,
       } else {
        m_pinfoCurrent = nullptr;
                                             { return m_pinfoCurrent; }
     DWORD m_cbRemaining;

    Enumerating logical processor information is complicated due to the variable-size structures, so I wrap it inside this helper enumerator class.

    Construct it with the relationship you are interested in, then use Current() to see the current item and Move­Next() to move to the next item. When there are no more items, Current() returns nullptr.

    The constructor does the standard two-step query we've seen before: Ask for the required buffer size, then allocate a buffer, then ask for the buffer to be filled in. There is a TOCTTOU race condition if a processor is added dynamically, but I'm going to ignore that case because this is a Little Program.

    Since the SYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX structure is variable-sized, walking the packed array is not a simple array indexing operation. Instead, you have to bump the pointer by the Size of the current element to find the next element.

    Next comes a helper function to print processor affinity bitmasks.

    void PrintMask(KAFFINITY Mask)
     printf(" [");
     for (int i = 0; i < sizeof(Mask) * 8; i++) {
      if (Mask & (static_cast<KAFFINITY>(1) << i)) {
       printf(" %d", i);
     printf(" ]");

    Nothing exciting there.

    Finally, we wrap it up inside a sample program that enumerates the cores and then, just for fun, enumerates the packages.

    int __cdecl main(int argc, char **argv)
     for (EnumLogicalProcessorInformation enumInfo(RelationProcessorCore);
          auto pinfo = enumInfo.Current(); enumInfo.MoveNext()) {
     for (EnumLogicalProcessorInformation enumInfo(RelationProcessorPackage);
          auto pinfo = enumInfo.Current(); enumInfo.MoveNext()) {
       for (UINT GroupIndex = 0; GroupIndex < pinfo->Processor.GroupCount; GroupIndex++) {
       printf(" ]\n");
     return 0;

    Enumerating processor cores produces a bunch of PROCESSOR_RELATIONSHIP structures, each with a single group that describes the logical processors assigned to the core.

    Enumerating processor packages produces a bunch of PROCESSOR_RELATIONSHIP structures, and each one contains as many groups as there are cores in the package.

    Bonus chatter: The CoreInfo utility from Sysinternals is a command-line tool that is a fancier version of this Little Program.

  • The Old New Thing

    My, those threads start up really fast nowadays


    Here's a little puzzle inspired by an actual bug:

    // global variable
    DWORD g_WorkerThreadId;
    bool IsRunningOnWorkerThread()
      return GetCurrentThreadId() == g_WorkerThreadId;
    bool LaunchWorkerThread()
     HANDLE hThread = CreateThread(nullptr, 0,
                                   nullptr, 0,
     if (hThread != nullptr) {
       return true;
     return false;
    DWORD CALLBACK WorkerThread(void *Proc)
      // Can this assertion ever fire?
      return 0;

    Can the assertion at the start of WorkerThread ever fire?

    Naturally, the answer is Yes, otherwise it wouldn't be a very interesting article.

    The assertion can fire if the worker thread starts running before the call the Create­Thread returns. In that case, the caller hasn't yet received the handle or ID of the newly-started thread. The new thread calls Is­Running­On­Worker­Thread, which returns false since g_Worker­Thread­Id hasn't been initialized yet.

    The actual bug was something along the lines of this:

    void DoSomething()
      if (IsRunningOnWorkerThread()) {
         .. do it one way ..
      } else {
         .. do it the other way ..
    void DoManyThings()
    DWORD CALLBACK WorkerThread(void *Proc)
      return 0;

    If the new thread started up so quickly that the original thread doesn't get a chance to receive the new thread ID and put it into g_Worker­Thread­ID, then the Do­Something function called from the worker thread will accidentally do things the not-on-the-worker-thread way, and then things start go go awry.

    One way to address is is to add suspenders to your belt:

    DWORD CALLBACK WorkerThread(void *Proc)
      g_WorkerThreadId = GetCurrentThreadId();

    By having both the original thread and the created thread set the g_WorkerThreadId variable, you cover both cases of the race. If the original thread runs faster, then the CreateThread function will set the g_WorkerThreadId variable to the ID of the worker thread, and the first line of Worker­Thread will be redundant. On the other hand, if the worker thread runs faster, then the assignment at the beginning of Worker­Thread sets the thread ID, and the assignment performed by the CreateThread function will be redundant.

  • The Old New Thing

    When should I use the FIND_FIRST_EX_LARGE_FETCH flag to FindFirstFileEx?


    Windows 7 introduces a new flag to the Find­First­File­Ex function called FIND_FIRST_EX_LARGE_FETCH. The documentation says that it "uses a larger buffer for directory queries, which can increase performance of the find operation." This is classic MSDN-style normative documentation: It provides "just the facts". Far be it for MSDN to tell you how to write your application; the job of function-level documentation is to document the function. If you want advice, go see a therapist.

    If the reason why you're calling Find­First­File­Ex is to enumerate through the entire directory and look at every entry, then a large buffer is a good thing because it reduces the number of round trips to the underlying medium. If the underlying medium is a network drive halfway around the world, the latency will be high, and reducing the number of calls reduces the overall cost of communication. Another case where you have high latency is if you are enumerating from an optical drive, since those tend to be slow to cough up data, and once you get the medium spinning, you want to get all the information you can before the drive spins the medium back down. On the other hand, if your underlying medium has low latency, then there isn't much benefit to using a large buffer, and it can be a detriment if the channel is low bandwidth, because transferring that large buffer will take a long time, which can result in long pauses on your UI thread.

    But what if you aren't enumerating with the purpose of reading the entire contents but rather are going to abandon the enumeration once you get the answer to your question? For example, maybe your function wants to enumerate the directory to see if it contains more than ten files. Once the tenth call to Find­Next­File succeeds, you're going to abandon the enumeration. In this case, a large buffer means that the underlying medium is going to do work that you will end up throwing away.

    Here's the above discussion summarized in a table, since people seem to like tables so much.

    Enumerating entire directory on UI thread No¹
    on background thread Yes
    Abandoning enumeration prematurely No

    ¹Actually, if you're on a UI thread, you should try to avoid any directory enumeration at all.

  • The Old New Thing

    It rather involved being on the other side of this airtight hatchway: Planting DLLs into directories on the PATH for applications whose current directory is always System32


    A bunch of security vulnerability reports came in from the same person, all of the form, "Program X is vulnerable to DLL planting if you create a DLL with the name Y in a directory on the PATH," with varying values of X and Y. In all the cases, Program X runs as SYSTEM with the System32 directory as its current directory, so a current directory attack does not apply here. The only remaining directory to attack is the PATH.

    But if you can attack the PATH for the SYSTEM user, then you are already on the other side of the airtight hatchway.

    The default PATH directories are all directories which are read-only to non-administrative users. In order to plant a DLL there, you must already be an administrator, at which point you are on the other side of the airtight hatchway. If you gain administrative access, then why bother planting DLLs in sneaky places? You already pwn the machine. Just do whatever you want! In other words, planting a DLL into secured directories can be carried out only by administrators, at which point it's just the administrator attacking his own computer. Not interesting.

    The next angle of investigation is whether somebody can sneak an insecure directory onto the global PATH. But modifying the global PATH requires administrative privileges, so if you have the privilege to add an insecure directory to the path, you are already an administrator, so this is another case of an administrator attacking his own computer. Still not interesting.

    Finally, the third option is that some application, as part of its installation, added an insecure directory to the global PATH. But in that case, you don't need to do any DLL planting at all. Just put a rogue executable on the PATH. For example, you might call it tpye.exe, so that somebody who typos the type command runs your rogue executable instead. In other words, in order for DLL planting into an insecure directory on the global PATH to work, you must already have a system whose integrity has already been violated, albeit inadvertently in this case. If you have an attack along these lines, then the security vulnerability is in the application whose installer added an insecure directory to the global PATH. This is another case of If you set up an insecure system, don't be surprised that there's a security vulnerability.

    Now, there is still a flaw in the application that gets tricked into loading the rogue DLL. It should be more specific about how it loads DLLs so it doesn't load the wrong one by accident. But there is no elevation of privilege, since only administrators can trick the application into loading the wrong DLL.

  • The Old New Thing

    The hierarchy of user education, as interpreted by a vice president


    One of my colleagues told me a story from his days on the Windows team. He had to give a presentation to his vice president on his feature, and he prepared his presentation and demo obsessively to make sure it went smoothly.

    At the meeting, there were three presentations on the schedule, all of them with features targeting IT professionals. The first presenter got up and walked through their feature. Upon reaching a particular tricky part of the feature, the vice president asked, "How are customers going to know how to set up those parameters?"

    The presenter explained, "We'll have a whitepaper explaining how to set them up."

    The vice president growled, "I ☁☆⚕☄ hate whitepapers."

    The second presenter's feature was also rather complicated, and the vice president asked the same question. "How are customers supposed to know how to set this up?"

    The second presenter explained, "We'll use a wizard to step them through it."

    The vice president snarled, "If there's anything I hate more than ☁☆⚕☄ whitepapers, it's ☁☆⚕☄ wizards."

    All through this, my colleague was waiting for his turn to present, and one of his nervous habits is to re-run the demo over and over while waiting, making sure it runs smoothly each time. His particular demo required a good amount of effort to reset after each go-round, and just as luck would have it, he was called to present while he was in the middle of resetting his demo. He rushed through the reset as he got up to do his presentation.

    My colleague ran through his presentation, but the demo fell apart because he didn't reset his scenario properly. In fact, he got the system into a bad state, because he had been messing with various settings not exposed to users directly. The vice president asked, "How are customers supposed to recover when they get into this state?"

    Having watched the first two presentations, my colleague had a good handle on the vice president's mood. He replied, "They're ☁☆⚕☄ed."

Page 1 of 3 (30 items) 123