• The Old New Thing

    Why was the replacement installer for recognized 16-bit installers itself a 32-bit program instead of a 64-bit program?

    • 34 Comments

    Even though 64-bit Windows does not support 16-bit applications, there is a special case for 16-bit installers for 32-bit applications. Windows detects this scenario and substitutes a 32-bit replacement installer which replicates the actions of the 16-bit installer. Commenter Karellen is horrified at the fact that the replacement installer is a 32-bit program. "You're writing a program that will run exclusively on 64-bit systems. Why not built it to run natively on the OS it's designed for? Why is this apparently not the "obvious" Right Thing(tm) to do? What am I missing?"

    Recall that a science project is a programming project that is technically impressive but ultimately impractical. For example it might be a project that nobody would actually use, or it attempts to add a Gee-Whiz feature that nobody is really clamoring for.

    But at least a science project is trying to solve a problem. This proposal doesn't even solve any problems! Indeed, this proposal creates problems. One argument in favor of doing it this way is that it is satisfies some obsessive-compulsive requirement that a 64-bit operating system have no 32-bit components beyond the 32-bit emulation environment itself.

    Because! Because you're running a 64-bit system, and running apps native to that system is just more elegant.

    Okay, it's not obsessive-compulsive behavior. It's some sort of aesthetic ideal, postulated for its own sake, devoid of practical considerations.

    Remember the problem space. We have a bunch of 32-bit applications that use a 16-bit installer. Our goal is to get those applications installed on 64-bit Windows. By making the replacement installer a 32-bit program, you get the emulator to do all the dirty work for you. Things like registry redirection, file system redirection, and 32-bit application compatibility.

    Suppose the original installer database says

    • Copy X.DLL file into the %Program­Files%\AppName directory.
    • Copy Y.DLL into the %windir%\System32 directory.
    • If the current version of C:\Program Files\Common Files\Adobe\Acrobat\ActiveX\AcroPDF.dll is 7.5 or higher, then set this registry key.

    If you write the replacement installer as a 32-bit program, then other parts of the 32-bit emulation engine do the work for you.

    • The environment manager knows that 64-bit processes get the environment variable Program­Files pointing to C:\Program Files, whereas 32-bit processes get Program­Files pointing to C:\Program Files (x86).
    • The file system redirector knows that if a 32-bit process asks for %windir%\System32, it should really get %windir%\SysWOW64.
    • The registry redirector knows that if a 32-bit process tries to access certain parts of the registry, they should be sent to the Wow­64­32­Node instead.

    If you had written the replacement installer as a 64-bit program, you would have to replicate all of these rules and make sure your copy of the rules exactly matched the rules used by the real environment manager, file system redirector, and registry redirector.

    Now you have to keep two engines in sync: the 32-bit emulation engine and the 64-bit replacement installer for 32-bit applications. This introduces fragility, because any behavior change in the 32-bit emulation engine must be accompanied by a corresponding change in the 64-bit replacement installer.

    Suppose the application compatibility folks add a rule that says, "If a 32-bit installer tries to read the version string from C:\Program Files\Common Files\Adobe\Acrobat\ActiveX\AcroPDF.dll, return the version string from C:\Program Files (x86)\Common Files\Adobe\Acrobat\ActiveX\AcroPDF.dll instead." And suppose that rule is not copied to the 64-bit replacement installer. Congratulations, your 64-bit replacement installer will incorrectly install any program that changes behavior based on the currently-installed version of AcroPDF.

    I don't know for sure, but I wouldn't be surprised if some of these installers support plug-ins, so that the application developer can run custom code during installation. It is possible for 16-bit applications to load 32-bit DLLs via a technique known as generic thunking, and the 16-bit stub installer would use a generic thunk to call into the 32-bit DLL to do whatever custom action was required. On the other hand, 64-bit applications cannot load 32-bit DLLs, so if the 64-bit replacement installer encountered a 32-bit DLL plug-in, it would have to run a 32-bit helper application to load the plug-in and call into it. So you didn't escape having a 32-bit component after all.

    And the original obsessive-compulsive reason for requiring the replacement installer to be 64-bit was flawed anyway. This is a replacement installer for a 32-bit application. Therefore, the replacement installer is part of the 32-bit emulation environment, so it is allowed to be written as a 32-bit component.

    Let's look at the other arguments given for why the replacement installer for a 32-bit application should be written as a 64-bit application.

    Because complexity is what will be our undoing in the end, and reducing it wherever we can is always a win.

    As we saw above, writing the replacement installer as a 64-bit application introduces complexity. Writing it as a 32-bit application reduces complexity. So this statement itself argues for writing the replacement installer as a 32-bit application.

    Because we can't rewrite everything from scratch at once, but we can create clean new code one small piece at a time, preventing an increase to our technical debt where we have the opportunity to do so at negligible incremental cost to just piling on more cruft.

    As noted above, the incremental cost is hardly negligible. Indeed, writing the replacement installer as a 64-bit application is not merely more complex, it creates an ongoing support obligation, because any time there is a change to the 32-bit emulation environment, that change needs to be replicated in the 64-bit replacement installer. This is a huge source of technical debt: Fragile coupling between two seemingly-unrelated components.

    And writing the replacement installer as a 32-bit application does not create a future obligation to port it to 64 bits when support for 32-bit applications is dropped in some future version of Windows. Because when support for 32-bit applications disappears (as it already has on Server Core), there will be no need to port the replacement installer to 64-bit because there's no point writing an installer for a program that cannot run!

    Writing the replacement installer as a 32-bit program was the right call.

  • The Old New Thing

    Staying cool is one of the skills of a corporate president

    • 12 Comments

    Some time ago, there was a mechanical problem with the heating/cooling system in our part of the building, and one of the senior managers in our group took the opportunity to tell a story of a one-on-one skip-level meeting he had with Steve Sinofsky.

    I'm sitting there in my office with Steve, and there was something wrong with the HVAC, because as the meeting progresses, it gets warmer and warmer, and eventually I'm sitting there sweating profusely, not exactly making the best impression on our group president. Steve, on the other hand, appears to be completely unaffected. It's sweltering in my office, but he's cool as a cucumber.

    It can't be more than five minutes after the meeting is over before a team of technicians swarms into my office to figure out why the heating system has gone berzerk.

    Steve must've whipped out his phone as soon as he left, called the Facilities desk, and said "Dude, there's something seriously wrong with the heating system over in room 1234. It's like an oven in there. You need to check it out." And since the request came from a corporate president, it got dispatched with high priority.

  • The Old New Thing

    Using thread pool cleanup groups to clean up many things at once

    • 10 Comments

    Today's Little Program demonstrates thread pool cleanup groups. When you associate a thread pool item with a cleanup group, you can perform bulk operations on the group. That can save you a lot of bookkeeping.

    Remember that Little Programs do little to no error checking.

    #include <windows.h>
    #include <stdio.h> // horrors! Mixing stdio and C++!
    
    VOID
    CALLBACK
    Callback(
        PTP_CALLBACK_INSTANCE Instance,
        PVOID                 /* Parameter */,
        PTP_TIMER             /* Timer */
        )
    {
        // Say what time the callback ran.
        printf("%p at %d\n", Instance, GetTickCount());
    }
    
    int
    __cdecl
    main(int, char**)
    {
        // Create an environment that we use for our timers.
        TP_CALLBACK_ENVIRON environ;
        InitializeThreadpoolEnvironment(&environ);
    
        // Create a thread pool cleanup group and associate it
        // with the environment.
        auto cleanupGroup = CreateThreadpoolCleanupGroup();
        SetThreadpoolCallbackCleanupGroup(&environ,
                                          cleanupGroup,
                                          nullptr);
    
        // Say what time we started
        printf("Start: %d\n", GetTickCount());
    
        // Ask for a one-second delay
        LARGE_INTEGER dueTime;
        dueTime.QuadPart = -10000LL * 1000; // one second
        FILETIME ftDue = { dueTime.LowPart, dueTime.HighPart };
    
        // Create ten timers to run after one second.
        for (int i = 0; i < 10; i++) {
            auto timer = CreateThreadpoolTimer(Callback,
                                               nullptr,
                                               &environ);
            SetThreadpoolTimer(timer, &ftDue, 0, 500);
        }
    
        // Wait a while - the timers will run.
        Sleep(1500);
    
        // Clean up the group.
        CloseThreadpoolCleanupGroupMembers(cleanupGroup,
                                           FALSE,
                                           nullptr);
    
        // Close the group.
        CloseThreadpoolCleanupGroup(cleanupGroup);
    }
    

    There is some trickiness in building the FILETIME structure to specify that we want to run after a one-second delay. First, the value is negative to indicate a relative timeout. Second, we cannot treat the FILETIME as an __int64, so we use a LARGE_INTEGER as an intermediary.

    When we create the ten timers, we associate them with the environment, which is in turn associated with the cleanup group. This puts all the timers into the cleanup group, which is a good thing, because we didn't save the timer handles!

    When it's time to clean up the timers, we use Close­Thread­pool­Cleanup­Group­Members, which does the work of closing each individual timer in the cleanup group. This saves us the trouble of having to remember all the timers ourselves and manually closing each one.

    For our next trick, comment out the Sleep(1500); and run the program again. This time, the timers don't run at all. That's because we closed them before they reached their due time. We let the cleanup group do the bookkeeping for us.

  • The Old New Thing

    Use GFlags to catch the silent killer (silent but deadly)

    • 22 Comments

    Suppose you have some process that is mysteriously dying and you can't figure out why. You think that some other process is doing a Terminate­Process but heck if you can figure out who that is.

    Starting in Windows 7 and Windows Server 2008 R2, the GFlags tool will let you catch these miscreants.

    On the Silent Process Exit tab, you enter the program you want to keep an extra eye on and check the box Enable Silent Process Exit Monitoring and select what you want to happen when one of these mysterious exits occurs. You can ask for an entry in the event log that identifies the killer, and you can ask for debugging minidumps to be created of both the killer and the victim.

  • The Old New Thing

    The SuspendThread function suspends a thread, but it does so asynchronously

    • 22 Comments

    Prologue: Why you should never suspend a thread.

    Okay, so a colleague decided to ignore that advice because he was running some experiments with thread safety and interlocked operations, and suspending a thread was a convenient way to open up race windows.

    While running these experiments, he observed some strange behavior.

    LONG lValue;
    
    DWORD CALLBACK IncrementerThread(void *)
    {
     while (1) {
      InterlockedIncrement(&lValue);
     }
     return 0;
    }
    
    // This is just a test app, so we will abort() if anything
    // happens we don't like.
    
    int __cdecl main(int, char **)
    {
     DWORD id;
     HANDLE thread = CreateThread(NULL, 0, IncrementerThread, NULL, 0, &id);
     if (thread == NULL) abort();
    
     while (1) {
      if (SuspendThread(thread) == (DWORD)-1) abort();
    
      if (InterlockedOr(&lValue, 0) != InterlockedOr(&lValue, 0)) {
       printf("Huh? The variable lValue was modified by a suspended thread?\n");
      }
    
      ResumeThread(thread);
     }
     return 0;
    }
    

    The strange thing is that the "Huh?" message was being printed. How can a suspended thread modify a variable? Is there some way that Interlocked­Increment can start incrementing a variable, then get suspended, and somehow finish the increment later?

    The answer is simpler than that. The Suspend­Thread function tells the scheduler to suspend the thread but does not wait for an acknowledgment from the scheduler that the suspension has actually occurred. This is sort of alluded to in the documentation for Suspend­Thread which says

    This function is primarily designed for use by debuggers. It is not intended to be used for thread synchronization

    You are not supposed to use Suspend­Thread to synchronize two threads because there is no actual synchronization guarantee. What is happening is that the Suspend­Thread signals the scheduler to suspend the thread and returns immediately. If the scheduler is busy doing something else, it may not be able to handle the suspend request immediately, so the thread being suspended gets to run on borrowed time until the scheduler gets around to processing the suspend request, at which point it actually gets suspended.

    If you want to make sure the thread really is suspended, you need to perform a synchronous operation that is dependent on the fact that the thread is suspended. This forces the suspend request to be processed since it is a prerequisite for your operation, and since your operation is synchronous, you know that by the time it returns, the suspend has definitely occurred.

    The traditional way of doing this is to call Get­Thread­Context, since this requires the kernel to read from the context of the suspended thread, which has as a prerequisite that the context be saved in the first place, which has as a prerequisite that the thread be suspended.

  • The Old New Thing

    If you wonder why a function can't be found, one thing to check is whether the function exists in the first place

    • 10 Comments

    One of my colleagues was frustrated trying to get some code to build. "Is there something strange about linking variadic functions? Because I keep getting an unresolved external error for the function, but if I move the function definition to the declaration point, then everything works fine."

    // blahblah.h
    
    ... other declarations ...
    
    void LogWidget(Widget* widget, const char* format, ...);
    
    ...
    
    // widgetstuff.cpp
    ...
    #include "blahblah.h"
    ...
    
    // some code that calls LogWidget
    void foo(Widget* widget)
    {
     LogWidget(widget, "starting foo");
     ...
    }
    
    // and then near the end of the file
    
    void LogWidget(Widget* widget, const char* format, ...)
    {
        ... implementation ...
    }
    
    ...
    

    "With the above code, the linker complains that Log­Widget cannot be found. But if I move the implementation of Log­Widget to the top of the file, then everything builds fine."

    // widgetstuff.cpp
    ...
    #include "blahblah.h"
    ...
    
    // move the code up here
    void LogWidget(Widget* widget, const char* format, ...)
    {
        ... implementation ...
    }
    
    // some code that calls LogWidget
    void foo(Widget* widget)
    {
     LogWidget(widget, "starting foo");
     ...
    }
    
    ...
    

    "I tried putting an explicit calling convention in the declaration, I tried using extern "C", nothing seems to help."

    We looked at the resulting object file and observed that in the case where the error occurred, there was an external reference to Log­Widget but no definition. I asked, "Is the definition of the function #ifdef'd out by mistake? You can use this technique to find out."

    That was indeed the problem. The definition of the function was inside some sort of #ifdef that prevented it from being compiled.

    Sometimes, the reason a function cannot be found is that it doesn't exist in the first place.

  • The Old New Thing

    Another way to make sure nobody sends you feedback

    • 28 Comments

    I wanted to report an issue about a problem with our building, let's say that the widget frobnicators were not working. I went to the internal Widget Frobnicators Web site, and it was very pretty, with an FAQ about the current supplier of widget frobnicators, where to look up more information about how the widget frobnicators work, how you can buy your own widget frobnicator for home use, and even how you can become a Widget Frobnicator Ambassador for your building. Widget Frobnicator Ambassadors receive additional information about widget frobnicators and get to participate in the widget frobnicator selection process, among other things.

    I didn't find a link on the Widget Frobnicator Web site that let me search for a Widget Frobnicator Ambassador in my building, or even a list of all Widget Frobnicator Ambassadors. However, I did find a link called Contact Us. Awesome. It is a mailto link addressed to the Widget Frobnicator Ambassadors distribution list.

    I composed a message explaining the problem I was observing and hit Send.

    The message was returned as undeliverable.

    "You do not have permission to send messages to this distribution list."

    Nice going, Widget Frobnicator team. Apparently you don't want to be contacted at all. Maybe it's a front for money laundering.

    (I was able to report the problem by other means, and it was resolved later that day, so everything worked out okay in the end, but as far as I can tell, the Widget Frobnicators Web site still provides no way for you to contact them.)

  • The Old New Thing

    Customizing item enumeration with IShellItem, the old-fashioned way

    • 3 Comments

    If you are targeting Windows 8 or higher, you can use STR_ENUM_ITEMS_FLAGS to customize how shell items are enumerated. But what if you need to run on older systems, too?

    In that case, you will need to drop to the lower-level IShell­Folder::Enum­Objects function, like we did before, and then reconstructe shell items from the low-level IShell­Folder and ITEMID_CHILD. (Note that the term "low-level" is used here only in a relative sense; it's lower level than IShell­Item.)

    We can wrap that inside a helper class.

    #define UNICODE
    #define _UNICODE
    #define STRICT
    #define STRICT_TYPED_ITEMIDS
    #include <windows.h>
    #include <shlobj.h>
    #include <atlbase.h>
    CComModule _Module;
    #include <atlcom.h>
    #include <atlalloc.h>
    #include <stdio.h>
    #include <tchar.h>
    
    class CEnumItemsWithSHCONTF :
        public CComObjectRoot,
        public IEnumShellItems
    {
    public:
      BEGIN_COM_MAP(CEnumItemsWithSHCONTF)
        COM_INTERFACE_ENTRY(IEnumShellItems)
      END_COM_MAP()
    
      static HRESULT Create(HWND hwnd, SHCONTF shcontf,
                     IShellItem *psiFolder, REFIID riid, void **ppv);
    
      STDMETHOD(Next)(ULONG celt, IShellItem **ppsi, ULONG *pceltFetched);
      STDMETHOD(Skip)(ULONG celt);
      STDMETHOD(Reset)();
      STDMETHOD(Clone)(IEnumShellItems **ppesiClone);
    
    private:
      static HRESULT CreateRef1(CComObject<CEnumItemsWithSHCONTF> **ppObj);
      HRESULT Initialize(HWND hwnd, SHCONTF shcontf, IShellItem *psiFolder);
      HRESULT CloneFrom(CEnumItemsWithSHCONTF *pSource);
    private:
      CComPtr<IShellFolder> m_spsfParent;
      CComPtr<IEnumIDList> m_speidl;
    };
    
    HRESULT CEnumItemsWithSHCONTF::CreateRef1(
        CComObject<CEnumItemsWithSHCONTF> **ppObj)
    {
      CComObject<CEnumItemsWithSHCONTF> *pObj;
      HRESULT hr = CComObject<CEnumItemsWithSHCONTF>::
                           CreateInstance(&pObj);
      *ppObj = CComPtr<CComObject<CEnumItemsWithSHCONTF> >(pObj).Detach();
      return hr;
    }
    
    HRESULT CEnumItemsWithSHCONTF::Initialize(
      HWND hwnd, SHCONTF shcontf, IShellItem *psiFolder)
    {
      HRESULT hr = psiFolder->BindToHandler(
        nullptr, BHID_SFObject, IID_PPV_ARGS(&m_spsfParent));
      if (SUCCEEDED(hr)) {
        hr = m_spsfParent->EnumObjects(hwnd, shcontf, &m_speidl);
      }
      return hr;
    }
    
    HRESULT CEnumItemsWithSHCONTF::CloneFrom(
        CEnumItemsWithSHCONTF *pSource)
    {
      HRESULT hr = pSource->m_speidl->Clone(&m_speidl);
      if (SUCCEEDED(hr)) {
        m_spsfParent = pSource->m_spsfParent;
      }
      return hr;
    }
    
    HRESULT CEnumItemsWithSHCONTF::Create(
        HWND hwnd, SHCONTF shcontf,
        IShellItem *psiFolder, REFIID riid, void **ppv)
    {
      *ppv = nullptr;
    
      CComPtr<CComObject<CEnumItemsWithSHCONTF>> spObj;
      HRESULT hr = CreateRef1(&spObj);
    
      if (SUCCEEDED(hr)) {
        hr = spObj->Initialize(hwnd, shcontf, psiFolder);
        if (SUCCEEDED(hr)) {
          hr = spObj->QueryInterface(riid, ppv);
        }
      }
      return hr;
    }
    
    HRESULT CEnumItemsWithSHCONTF::Next(
        ULONG celt, IShellItem **ppsi, ULONG *pceltFetched)
    {
      if (celt != 1 && pceltFetched == nullptr) {
        return E_INVALIDARG;
      }
    
      for (ULONG i = 0; i < celt; i++) ppsi[i] = nullptr;
    
      ULONG celtFetched = 0;
      HRESULT hr = S_OK;
      while (hr == S_OK && celtFetched < celt) {
        CComHeapPtr<ITEMID_CHILD> spidlChild;
        hr = m_speidl->Next(1, &spidlChild, nullptr);
        if (hr == S_OK) {
          hr = SHCreateItemWithParent(nullptr, m_spsfParent,
            spidlChild, IID_PPV_ARGS(&ppsi[celtFetched]));
          if (SUCCEEDED(hr)) celtFetched++;
        }
      }
    
      if (pceltFetched != nullptr) *pceltFetched = celtFetched;
      if (SUCCEEDED(hr)) {
        hr = (celtFetched == celt) ? S_OK : S_FALSE;
      }
      return hr;
    }
    
    HRESULT CEnumItemsWithSHCONTF::Skip(ULONG celt)
    {
      return m_speidl->Skip(celt);
    }
    
    HRESULT CEnumItemsWithSHCONTF::Reset()
    {
      return m_speidl->Reset();
    }
    
    HRESULT CEnumItemsWithSHCONTF::Clone(
        IEnumShellItems **ppesiClone)
    {
      *ppesiClone = nullptr;
    
      CComPtr<CComObject<CEnumItemsWithSHCONTF>> spClone;
      HRESULT hr = CreateRef1(&spClone);
    
      if (SUCCEEDED(hr)) {
        hr = spClone->CloneFrom(this);
        if (SUCCEEDED(hr)) {
            *ppesiClone = spClone.Detach();
        }
      }
      return hr;
    }
    

    The CEnum­Items­With­SHCONTF class does the work of enumerating child items the old-fashioned way, then constructing shell items from the result. Most of this code is boilerplate (including the part to avoid having a live COM object with reference count zero).

    The object has two members, the source folder from which the items are being enumerated and the low-level enumerator itself. We initialize the object by asking for the low-level IEnum­ID­List handler and calling IEnum­ID­List::Enum­Objects with the specific flags we want. When it is time to generate items, we ask the inner enumerator for the next ID list, and construct a shell item around it by comining the ID list with the parent folder.

    The rest is bookkeeping: We keep track of the number of elements fetched so far in order to return it to the caller if requested, and also in order to decide what the return value should be. If all items were retrieved successfully, then return S_OK. If we ran out of items, then return S_FALSE. If something went wrong, we return the error code, possibly with partially-fetched results.

    The other enumerator operations like Reset and Clone are delegated to the inner enumerator. Cloning is a little tricky because we also have to clone ourselves!

    Now we can adapt our program from last time to use this class instead of BHID_Enum­Items.

    int __cdecl wmain(int argc, wchar_t **argv)
    {
     CCoInitialize init;
    
     if (argc < 2) return 0;
     CComPtr<IShellItem> spsiFolder;
     SHCreateItemFromParsingName(argv[1], nullptr,
                                 IID_PPV_ARGS(&spsiFolder));
    
     CComPtr<IEnumShellItems> spesi;
     CEnumItemsWithSHCONTF::Create(nullptr, SHCONTF_FOLDERS,
        spsiFolder, IID_PPV_ARGS(&spesi));
     for (CComPtr<IShellItem> spsi;
          spesi->Next(1, &spsi, nullptr) == S_OK;
          spsi.Release()) {
      PrintDisplayName(spsi, SIGDN_NORMALDISPLAY, L"Display Name");
      PrintDisplayName(spsi, SIGDN_FILESYSPATH, L"File system path");
      wprintf(L"\n");
     }
     return 0;
    }
    
  • The Old New Thing

    Got errands? Now is the time

    • 24 Comments

    This upcoming Sunday is the Super Bowl, the championship game for a sport played only in the United States.¹

    The entire country stops doing anything when the game is on. This makes it a perfect time to get out and run your errands, because the streets will be completely empty.

    Check out this traffic map at the kickoff of the 2014 Super Bowl. For fun, you can go backward in time in 10-minute increments and watch the traffic slowly die out as the start of the game approaches.

    If you're a photographer, it's a good time to go take pictures of public places, because they will all be deserted.

    It's also a good time to go to Costco, though you should wait until the game has started. It takes time for all the people getting last-minute party supplies to drain out.

    ¹ Well, Canada has their own variant. They're so cute, those Canadians.

    Bonus chatter: The American Football League of China is a real thing.

  • The Old New Thing

    Creating a shared memory block that can grow in size

    • 13 Comments

    A little-known feature of shared memory blocks in Win32 is that it is possible to resize them, sort of.

    When you create a shared memory block, you can pass the SEC_RESERVE flag to Create­File­Mapping, then the size you pass to the function is treated as a maximum rather than an exact size. (Don't forget that Create­File­Mapping is used for creating both memory-mapped files and for creating plain old shared memory. The name of the function is misleading unless you're wearing kernel-colored glasses.)

    When you map this shared memory block, you are reserving address space, but no memory is committed yet. You call Virtual­Alloc to commit memory into the shared memory block.

    This means that you can create a growable shared memory block by creating an initially empty block, and then committing a small amount of memory into it. When you want to grow the block, you commit more. However, you cannot shrink the shared memory block. Once the memory is committed, it cannot be decommitted.

    Here's a demonstration. Note that most error checking has been elided for expository purposes. Note also that since the memory isn't actually being shared with anybody, this program working too hard; it could have just used plain old Virtual­Alloc. So pretend that the memory is being shared with somebody else.

    #include <windows.h>
    #include <stdio.h>
    
    #define ONE_GIGABYTE (1024 * 1024 * 1024)
    #define VIEW_SIZE (ONE_GIGABYTE / 2) // We will map half of it
    
    void ReportMemoryPresence(void *p)
    {
     MEMORY_BASIC_INFORMATION mbi;
     VirtualQuery(p, &mbi, sizeof(mbi));
     printf("Memory at %p is %s\n", p,
            (mbi.State & MEM_COMMIT) ? "committed" : "not committed");
    }
    
    void WaitForEnter()
    {
     char dummy[64];
     fgets(dummy, 64, stdin);
    }
    
    int __cdecl wmain(int, wchar_t **)
    {
     BYTE *pView;
     HANDLE h = CreateFileMapping(INVALID_HANDLE_VALUE, NULL,
                                  PAGE_READWRITE,
                                  0, VIEW_SIZE,
                                  NULL);
     printf("Created the file mapping\n");
     WaitForEnter();
    
     pView = (BYTE*)MapViewOfFile(h, FILE_MAP_WRITE, 0, 0, VIEW_SIZE);
     printf("Mapped half of it at %p\n", pView);
    
     ReportMemoryPresence(pView);
     ReportMemoryPresence(pView + VIEW_SIZE - 1);
     WaitForEnter();
    
     return 0;
    }
    

    In this version, we create a one-gigabyte shared memory block with no special flags, which means that all the memory gets committed up front. When you run this program, it reports that the memory at the start and end of the mapping is present. That's because the normal mode for shared memory is to commit it all at creation.

    You can watch the effect of commit by running Task Manager, going to the Performance tab, and looking at the value under Committed. It should jump by a gigabyte when "Created the file mapping" is printed. (For some reason, the Commit size in the Details pane counts the view as commitment, even though the view consists almost entirely of reserved rather than committed pages.)

    Now let's add the SEC_RESERVE flag:

     HANDLE h = CreateFileMapping(INVALID_HANDLE_VALUE, NULL,
                                  PAGE_READWRITE | SEC_RESERVE,
                                  0, VIEW_SIZE,
                                  NULL);
    

    Now when you run the program, Task Manager's Committed memory does not increase. That's because we created an empty shared memory block with the potential to grow up to one gigabyte, but right now it is size zero. This is confirmed by the memory presence check, which reports that the memory at the start and end of the mapped view is not committed.

    Okay, well, a zero-length shared memory block isn't very useful, so let's make it, say, 100 megabytes in size.

    #define BLOCK_SIZE (100 * 1024 * 1024)
    
    int __cdecl wmain(int, wchar_t **)
    {
     BYTE *pView;
     HANDLE h = CreateFileMapping(INVALID_HANDLE_VALUE, NULL,
                                  PAGE_READWRITE | SEC_RESERVE,
                                  0, VIEW_SIZE,
                                  NULL);
     printf("Created the file mapping\n");
     WaitForEnter();
    
     pView = (BYTE*)MapViewOfFile(h, FILE_MAP_WRITE, 0, 0, VIEW_SIZE);
     printf("Mapped half of it at %p\n", pView);
    
     ReportMemoryPresence(pView);
     ReportMemoryPresence(pView + VIEW_SIZE - 1);
     WaitForEnter();
    
     VirtualAlloc(pView, BLOCK_SIZE, MEM_COMMIT, PAGE_READWRITE);
     printf("Committed some of it at %p\n", pView);
    
     ReportMemoryPresence(pView);
     ReportMemoryPresence(pView + BLOCK_SIZE - 1);
     ReportMemoryPresence(pView + BLOCK_SIZE);
     ReportMemoryPresence(pView + VIEW_SIZE - 1);
     WaitForEnter();
    
     return 0;
    }
    

    Watch the Committed memory in Task Manager go up by 0.1 gigabytes when we commit some of it. Also observe that the memory presence checks show that we have exactly 100 megabytes of memory available; the byte at 100 megabytes + 1 is not present.

    Okay, so we were able to grow the shared memory block from zero to 100 megabytes. Let's grow it again up to 200 megabytes.

    int __cdecl wmain(int, wchar_t **)
    {
     ...
    
     VirtualAlloc(pView + BLOCK_SIZE, BLOCK_SIZE, MEM_COMMIT, PAGE_READWRITE);
     printf("Committed some of it at %p\n", pView + BLOCK_SIZE);
    
     ReportMemoryPresence(pView);
     ReportMemoryPresence(pView + 2 * BLOCK_SIZE - 1);
     ReportMemoryPresence(pView + 2 * BLOCK_SIZE);
     ReportMemoryPresence(pView + VIEW_SIZE - 1);
     WaitForEnter();
    
     return 0;
    }
    

    Okay, well there you go, a growable shared memory block. If you wanted to conserve address space, you could use Map­View­Of­File to map only the number of bytes you intend to commit, and each time you want to grow the memory block, you create a new larger view. I didn't bother with that because I'm lazy.

    Bonus chatter: Another way to get the effect of growable and shrinkable shared memory blocks is to cheat and create multiple shared memory blocks, but map them right next to each other.

    Bonus chatter 2: You can get sort of the effect of decommitting memory from the block by resetting it (MEM_RESET). The memory is still committed, but you told the memory manager that if the memory needs to be paged out, just discard it rather than writing it to disk.

    Bonus chatter 3: Be aware that creating very large SEC_RESERVE sections can incur high commit charges for the page tables themselves. This is significantly improved in Windows 8.1, which defers committing the page tables until you actually use them.

Page 9 of 450 (4,498 items) «7891011»