• The Old New Thing

    Luxurifying the camping experience in a different direction

    • 14 Comments

    Some time ago, I noted the increasing luxurification of camping, where people with more money than sense decide to go through the camping experience without building any of the character that comes with it.

    But that's not the only direction luxurification has moved. Unwilling to accept that "getting there is half the fun", some people take chartered planes to and from summer camp. Stick it out for the punch line in the final sentence of the article.

  • The Old New Thing

    Restoring symbols to a stack trace originally generated without symbols

    • 10 Comments

    Has this ever happened to you?

    litware!Ordinal3+0x6042
    litware!DllInstall+0x4c90
    litware!DllInstall+0x4b9e
    contoso!DllGetClassObject+0x93c3
    contoso!DllGetClassObject+0x97a9
    contoso!DllGetClassObject+0x967c
    contoso!DllGetClassObject+0x94d7
    contoso!DllGetClassObject+0x25ce
    contoso!DllGetClassObject+0x2f7b
    contoso!DllGetClassObject+0xad55
    contoso!DllGetClassObject+0xaec7
    contoso!DllGetClassObject+0xadf7
    contoso!DllGetClassObject+0x3c00
    contoso!DllGetClassObject+0x3b2a
    contoso!DllGetClassObject+0x462b
    USER32!UserCallWinProcCheckWow+0x13a
    USER32!DispatchMessageWorker+0x1a7
    contoso!DllCanUnloadNow+0x19b6
    contoso!DllGetClassObject+0xeaf2
    contoso+0x1d6c
    litware!LitImportReportProfile+0x11c4
    litware!LitImportReportProfile+0x1897
    litware!LitImportReportProfile+0x1a3b
    KERNEL32!BaseThreadInitThunk+0x18
    ntdll!RtlUserThreadStart+0x1d
    

    Ugh. A stack trace taken without working symbols. (There's no way that Dll­Get­Class­Object is a deeply recursive 60KB function. Just by casual inspection, you know that the symbols are wrong.)

    To see how to fix this, you just have to understand what the debugger does when it has no symbols to work from: It uses the symbols from the exported function table. For every address it wants to resolve, it looks for the nearest exported function whose address is less than or equal to the target value.

    For example, suppose CONTOSO.DLL has the following exported symbols:

    Symbol Offset
    Dll­Get­Class­Object 0x5132
    Dll­Can­Unload­Now 0xFB0B

    Look at it this way: The debugger is given the following information about your module: (Diagram not to scale.)

     
      Dll­Get­Class­Object Dll­Can­Unload­Now

    It needs to assign a function to every byte in the module. In the absence of any better information, it does it like this:

    ??? Dll­Get­Class­Object Dll­Can­Unload­Now

    In words, it assumes that every function begins at the location specified by the export table, and it ends one byte before the start of the next function. The debugger is trying to make the best of a bad situation.

    Suppose your DLL was loaded at 0x10000000, and the debugger needs to generate a symbolic name for the address 0x1000E4F5.

    First, it converts the address into a relative virtual address by subtracting the DLL base address, leaving 0xE4F5.

    Next, it looks to see what function "contains" that address. From the algorithm described above, the debugger concludes that the address 0xE4F5 is "part of" the Dll­Get­Class­Object function, which began at begins at 0x5132. The offset into the function is therefore 0xE4F5 - 0x5132 = 0x93C3, and it is reported in the debugger as contoso!Dll­Get­Class­Object+0x93c3.

    Repeat this exercise for each address that the debugger needs to resolve, and you get the stack trace above.

    Fine, now that you know how the bad symbols were generated, how do you fix it?

    You fix it by undoing what the debugger did, and then redoing it with better symbols.

    You need to find the better symbols. This is not too difficult if you still have a matching binary and symbol file, because you can just load up the binary into the debugger in the style of a dump file. Like Doron, you can then let the debugger do the hard work.

    C:> ntsd -z contoso.dll
    
    ModLoad: 10000000 10030000   contoso.dll
    

    Now you just ask the debugger, "Could you disassemble this function for me?" You give it the broken symbol+offset above. The debugger looks up the symbol, applies the offset, and then looks up the correct symbol when disassembling.

    0:000> u contoso!DllGetClassObject+0x93c3
    contoso!CReportViewer::ActivateReport+0xe9:
    10000e4f5 eb05            jmp     contoso!CReportViewer::ActivateReport+0xf0
    

    Repeat for each broken symbol in the stack trace, and you have yourself a repaired stack trace.

    litware!Ordinal3+0x6042 ← oops
    litware!CViewFrame::SetInitialKeyboardFocus+0x58
    litware!CViewFrame::ActivateViewInFrame+0xf2
    contoso!CReportViewer::ActivateReport+0xe9
    contoso!CReportViewer::LoadReport+0x12c
    contoso!CReportViewer::OnConnectionCreated+0x13f
    contoso!CViewer::OnConnectionEvent+0x7f
    contoso!CConnectionManager::OnConnectionCreated+0x85
    contoso!CReportFactory::BeginCreateConnection+0x87
    contoso!CReportViewer::CreateConnectionForReport+0x20d
    contoso!CViewer::CreateNewConnection+0x87
    contoso!CReportViewer::CreateNewReport+0x213
    contoso!CViewer::OnChangeView+0xec
    contoso!CReportViewer::WndProc+0x9a7
    contoso!CView::s_WndProc+0xf1
    USER32!UserCallWinProcCheckWow+0x13a
    USER32!DispatchMessageWorker+0x1a7
    contoso!CViewer::MessageLoop+0x24e
    contoso!CViewReportTask::RunViewer+0x12
    contoso+0x1d6c ← oops
    litware!CThreadTask::Run+0x40
    litware!CThread::ThreadProc+0xe5
    litware!CThread::s_ThreadProc+0x42
    KERNEL32!BaseThreadInitThunk+0x18
    ntdll!RtlUserThreadStart+0x1d
    

    Oops, our trick doesn't work for that first entry in the stack trace, the one with Ordinal3. What's up with that? There is no function called Ordinal3!

    If your module exports functions by ordinal without a name, then the debugger doesn't know what name to print for the function (since the name was stripped from the module), so it just prints the ordinal number. You will have to go back to your DLL's DEF file to convert the ordinal back to a function name. Or you can dump the exports from the DLL to see what functions match up with what ordinals. (Of course, for that trick to work, you need to have a matching PDB file in the symbol search path.)

    In our example, suppose litware.dll ordinal 3 corresponds to the function Lit­Debug­Report­Profile. We would then ask the debugger

    0:001> u litware!LitDebugReportProfile+0x6042
    litware!CViewFrame::FindInitialFocusControl+0x66:
    1000084f5 33db            xor     ebx,ebx
    

    Okay, that takes care of our first oops. What about the second one?

    In the second case, the address the debugger was asked to generate a symbol for came before the first symbol in the module. In our diagram above, it was in the area marked with question marks. The debugger has absolutely nothing to work with, so it just disassembles as relative to the start of the module.

    To resolve this symbol, you take the offset and add it to the base of the module as it was loaded into the debugger, which was reported in the ModLoad output:

    ModLoad: 10000000 10030000   contoso.dll
    

    If that output scrolled off the screen, you can ask the debugger to show it again with the help of the lmm command.

    0:001>lmm contoso*
    start    end        module name
    10000000 10030000   contoso    (export symbols)       contoso.dll
    

    Once you have the base address, you add the offset back and ask the debugger what's there:

    0:001> u 0x10000000+0x1d6c
    contoso!CViewReportTask::Run+0x102:
    100001d6c 50              push    eax
    

    Okay, now that we patched up all our oopses, we have the full stack trace with symbols:

    litware!CViewFrame::FindInitialFocusControl+0x66
    litware!CViewFrame::SetInitialKeyboardFocus+0x58
    litware!CViewFrame::ActivateViewInFrame+0xf2
    contoso!CReportViewer::ActivateReport+0xe9
    contoso!CReportViewer::LoadReport+0x12c
    contoso!CReportViewer::OnConnectionCreated+0x13f
    contoso!CViewer::OnConnectionEvent+0x7f
    contoso!CConnectionManager::OnConnectionCreated+0x85
    contoso!CReportFactory::BeginCreateConnection+0x87
    contoso!CReportViewer::CreateConnectionForReport+0x20d
    contoso!CViewer::CreateNewConnection+0x87
    contoso!CReportViewer::CreateNewReport+0x213
    contoso!CViewer::OnChangeView+0xec
    contoso!CReportViewer::WndProc+0x9a7
    contoso!CView::s_WndProc+0xf1
    USER32!UserCallWinProcCheckWow+0x13a
    USER32!DispatchMessageWorker+0x1a7
    contoso!CViewer::MessageLoop+0x24e
    contoso!CViewReportTask::RunViewer+0x12
    contoso!CViewReportTask::Run+0x102
    litware!CThreadTask::Run+0x40
    litware!CThread::ThreadProc+0xe5
    litware!CThread::s_ThreadProc+0x42
    KERNEL32!BaseThreadInitThunk+0x18
    ntdll!RtlUserThreadStart+0x1d
    

    Now the fun actually starts: Figuring out why there was a break in CView­Frame::Find­Initial­Focus­Control. Happy debugging!

    Bonus tip: By default, ntsd does not include line numbers when resolving symbols. Type .lines to toggle line number support.

  • The Old New Thing

    Why doesn't the New Folder command work in the root of a redirected drive resource in a Remote Desktop session?

    • 27 Comments

    When you connect to another computer via Remote Desktop, you have the option of injecting your local drives into the remote computer, known as Device and Resource Redirection. These injected drives are available under the UNC \\tsclient\X where X is a drive letter on the local machine.

    The name TSCLIENT combines a bunch of internal technical terminology, so it makes perfect sense to the people who wrote it, but not as much to outsiders. (They may have chosen this name just to make themselves look smart.) The letters TS stand for Terminal Services, which was the former name of the technology now known as Remote Desktop. And the word client refers to the local computer, the one that is connected to the remote computer. In Terminal Services terminology, the machine you are connect from is the client, and the machine you are connecting to is the server.

    There's another level of confusion in the name of the feature. People often call these \\tsclient\X thingies Redirected Drives, which collides with the existing name for local drive letters that have been mapped to a network resource. In the user interface, these are usually called Mapped Network Drives. From the command line, you create these things via the NET USE command.

    Okay, enough with the confusing terminology. For today, we're talking about Remote Desktop Device Redirection, where the redirected device is a drive letter.

    If you open My Computer and look under Other, you'll see those drives which were injected from the local computer. Your first tip-off that there's something funny about these drives: They don't show up in the Network Location section like other mapped drives; instead they show up under the rather generic-sounding Other.

    That's because these drives aren't really drives. They are folder shortcuts, a special type of shortcut that grafts one part of the shell namespace into another. The ones created by Remote Desktop Device Redirection are shell instance objects, which is a way of creating certain types of shell extensions using just a handful of registry keys.

    Since they aren't really drives, some things that work for real drives don't work for these fake drives. And one of those things is that Explorer thinks that they don't support the New Folder command because when Explorer asks, "Do you support IStorage?" (because that's the interface that Explorer uses to create new folders), the answer is "No, you silly rabbit. I'm an Other!"

    Now, it turns out that the Terminal Services folks could've customized their Other to say, "Actually, yeah, I do support IStorage." You do this by setting the bit 0x00000008 in the Attributes value of the ShellFolder key when you registered your instance object. The Terminal Services folks forgot to set that bit, and the result is no New Folder button.

    Sorry about that.

    As a workaround, you can create your new folder by typing \\tsclient\X into the address bar. That folder is the thing that the folder shortcut is pointing to (so it's just another name for the same thing), but since it's the real thing, it correctly reports the SFGAO_STORAGE flag, and the New Folder button appears.

  • The Old New Thing

    Hey, you look Chinese, we have a class for people like you

    • 30 Comments

    (The title is a callback to this article from a few months ago.)

    A member of my extended family grew up near the city of Nanaimo, Canada. While it's true that she's ethnically Chinese, it's also true that she's a fourth generation Canadian. The community is overwhelmingly English-speaking, and English is her first language. She grew up going to an English-language school, she watched English-language television, spoke English with her friends and family, and probably dreamed dreams in English.

    Yet when the family moved to Vancouver when she was a child (I don't know the exact age, so let's say eight years old), the school district automatically enrolled her in English as a Second Language, presumably based on her Chinese last name and the fact that Mandarin and Cantonese are the mother tongues in 30% of Vancouver homes.

    Since she was only eight years old, she didn't know that the school had tagged her as a non-native English speaker. She cheerfully went to her special class for an hour each day, enjoying her time with a new teacher and new classmates.

    It wasn't for a long time (I don't know exactly, so let's say six months) that she realized, "Wait a second, this is the class for people who don't speak English well!" Once the mistake was recognized, it wasn't long before she was transferred back to the regular class.

    Though I'm kind of surprised the school district (and the class teacher) never figured out that her English was native, or, failing that, that her English was plenty good enough that she didn't need the class any more.

  • The Old New Thing

    Holy cow, those TechReady attendees really love their tchotchkes

    • 16 Comments

    I was at the Ask the Experts event last night at TechReady11, and if I didn't know better, I would have thought the purpose of Ask the Experts was for attendees to wander the room collecting the coolest swag they could get their hands on as quickly as possible. My table was equipped with about two dozen Windows 7 frisbees, and the moment they came out of the box, they disappeared into the hands of passers-by, most of whom didn't even bother reading the sign on the table much less make eye contact with me.

    The table next to mine started with a mountain of mugs, but it wasn't long before it was reduced to a molehill.

    To try to convince people at least to make eye contact with me, I hid the remaining frisbees under the table, handing them only to people who actually stopped to ask a question, or at least tell an interesting story.

    After the frisbees were gone, the swag fairies dropped a few dozen battery-powered light stick thingies. They didn't disappear as quickly, perhaps because the initial surge of swag-hunters had subsided.

    I was kind of surprised at how aggressively people went after the swag. This is, after all, a Microsoft internal event. You'd think these people would be jaded by now, having been surrounded by Microsoft-branded doodads for years.

  • The Old New Thing

    File version information does not appear in the property sheet for some files

    • 26 Comments

    A customer reported that file version information does not appear on the Details page of the property sheet which appears when you right-click the file and select Properties. They reported that the problem began in Windows 7.

    The reason that the file version information was not appearing is that the file's extension was .xyz. Older versions of Windows attempted to extract file version information for all files regardless of type. I believe it was Windows Vista that changed this behavior and extracted version information only for known file types for Win32 modules, specifically .cpl, .dll, .exe, .ocx, .rll, and .sys. If the file's extension is not on the list above, then the shell will not sniff for version information.

    If you want to register a file type as eligible for file version extraction, you can add the following registry key:

    HKEY_LOCAL_MACHINE
     \Software
      \Microsoft
        \Windows
          \CurrentVersion
            \PropertySystem
              \PropertyHandlers
                \.XYZ
                 (Default) = REG_SZ:"{66742402-F9B9-11D1-A202-0000F81FEDEE}"
    

    (Thanks in advance for complaining about this change in behavior. This always happens whenever I post in the Tips/Support category about how to deal with a bad situation. Maybe I should stop trying to explain how to deal with bad situations.)

  • The Old New Thing

    How can I detect that a user's SID has changed and recover their old data?

    • 9 Comments

    A customer maintained a database which recorded information per user. The information in the database is keyed by the user's SID. This works out great most of the time, but there are cases in which a user's SID can change.

    "Wait, I thought SIDs don't change."

    While it's true that SIDs don't change, it is also true that the SID associated with a user can change. Since SIDs encode the domain to which they belong, a user which moves from one domain to another within an organization, will need to be assigned a new SID.

    But wait, does that mean that the user lost access to all their stuff? After all, all their stuff was marked "Owned by X\UserName" but the user's SID is now Y\UserName.

    No, the user doesn't lose access to their stuff thanks to SID history, and if you move users around a lot, the SID history can get quite large.

    A token for a user contains not only their current identity but also all of their earlier identities. That is what permits Y\UserName to continue to access things that was marked "Owned by X\UserName": The token for Y\UserName includes an entry that says, "Oh, I used to be X\UserName."

    The customer's database can take advantage of the SID history to match up users with their former selves. Our customer was lucky in that their database recorded only users who had logged into the local machine, so that list is typically pretty small. The simplest solution for this particular customer is just to go through all the users in the database, and for each one, see if the current user has that database user in their SID history. And the easy way to do that is to make the security system do the work for you: To see if the current user has user X in their SID history, create a security descriptor that grants access only to user X, then call Access­Check to see if the current user can access it. If so, then that means that the current user was at one point in the past known as X.

    (If you have a large database where iterating over all users is impractical, you can ask for the current user's SID-History attribute and walk through the previous identities manually.)

  • The Old New Thing

    Why does the access violation error message put the operation in quotation marks? Is is some sort of euphemism?

    • 24 Comments

    When an application crashes with an access violation, the error message says something like

    The instruction at "XX" referenced memory at "YY". The memory could not be "read".

    Why is the operation in quotation marks? Is this some sort of euphemism?

    The odd phrasing is a consequence of globalization. The operation name is a verb in the infinitive ("read", "write"), but depending on how the containing message is localized, it may need to take a different form. Since the kernel doesn't understand grammar, it just puts the words in quotation marks to avoid having to learn every language on the planet. Imagine if it tried:

    The memory could not be readed.

    The kernel tried to form the passive, which is normally done in English by adding "–ed" to the end of the verb. Too bad "read" and "write" are irregular verbs!

    The more conventional solution for this type of problem is to create a separate error message for each variant so that the text can be translated independently. rather than building sentences at runtime,

    The access violation error message is in a pickle, though, because the underlying status code is STATUS_ACCESS_VIOLATION, and that message contains three insertions, one for the instruction address, one for the address being accessed, and one for the operation. If there were three different status codes, like STATUS_ACCESS_VIOLATION_READ, STATUS_ACCESS_VIOLATION_WRITE, and STATUS_ACCESS_VIOLATION_EXECUTE, then a separate string could be created for each. But that's not how the status codes folks decided to do things, and the translation team was stuck having to use the ugly quotation marks.

  • The Old New Thing

    Why does saving a file in Notepad fire multiple FindFirstChangeNotification events?

    • 9 Comments

    Many people have noticed that the Read­Directory­ChangesW and Find­First­Change­Notification functions (and therefore their BCL equivalent File­System­Watcher and WinRT equivalent Storage­Folder­Query­Result) fire multiple FILE_ACTION_MODIFIED events when you save a file in Notepad. Why is that?

    Because multiple things were modified.

    Notepad opens the file for writing, writes the new data, calls Set­End­Of­File to truncate any excess data (in case the new file is shorter than the old file), then closes the handle. Two things definitely changed, and a third thing might have changed.

    • The file last-modified time definitely changed.
    • The file size definitely changed.
    • The file last-access time might have changed.

    It's therefore not surprising that you got two events, possibly three.

    Remember the original design goals of the Read­Directory­ChangesW function: It's for letting an application cache a directory listing and update it incrementally. Given these design goals, filtering out redundant notifications in the kernel is not required aside from the performance benefits of reduced chatter. In theory, Read­Directory­ChangesW could report a spurious change every 5 seconds, and the target audience for the function would still function correctly (albeit suboptimally).

    Given this intended usage pattern, any consumer of Read­Directory­ChangesW needs to accept that any notifications you receive encompass the minimum information you require in order to keep your cached directory information up to date, but it can contain extra information, too. If you want to respond only to actual changes, you need to compare the new file attributes against the old ones.

    Bonus chatter: Actually, the two things that changed when Notepad set the file size are the allocation size and the file size (which you can think of as the physical and logical file sizes, respectively). Internally, this is done by two separate calls into the I/O manager, so it generates two change notifications.

  • The Old New Thing

    Raymond misreads flyers, episode 2: It Takes You

    • 17 Comments

    As part of a new phase in Microsoft's continuing recycling efforts, the recycling program got a new motto. The new motto was not announced with any fanfare. Rather, any recycling-related announcement had the new motto at the top as part of the letterhead.

    The new motto: It Takes You.

    I had trouble parsing this at first. To me, it sounded like the punch line to a Yakov Smirnoff joke.

    Episode 1.

Page 377 of 448 (4,480 items) «375376377378379»