August, 2012

  • The Old New Thing

    Exiting a batch file without exiting the command shell -and- batch file subroutines


    Prepare your party hats: Batch File Week is almost over.

    In your batch file, you may want to exit batch file processing (say, you encountered an error and want to give up), but if you use the exit command, that will exit the entire command processor. Which is probably not what you intended.

    Batch file processing ends when execution reaches the end of the batch file. The trick therefore is to use the goto command to jump to a label right before the end of the file, so that execution "falls off the end".

    @echo off
    if "%1"=="" echo You must provide a file name.&goto end
    if NOT EXIST "\\server\backup\%USERNAME%\nul" mkdir "\\server\backup\%USERNAME%"
    if NOT EXIST "\\server\backup\%USERNAME%\nul" echo Unable to create output directory.&goto end
    copy "%1" "\\server\backup\%USERNAME%"

    Here, there are two places where we abandon batch file execution. One is on an invalid parameter, and another is if the output directory couldn't be created (or if it isn't a directory at all).

    The batch command interpreter provides a courtesy label to simply this technique: The special goto target goto :eof (with the colon) jumps to the end of the batch file. It's as if every batch file had a hidden goto label called :eof on the very last line.

    The goto :eof trick becomes even more handy when you start playing with batch file subroutines. Okay, let's back up: Batch file subroutines?

    By using the call command, a batch file can invoke another batch file and regain control after that other batch file returns. (If you forget the call, then control does not return. In other words, the default mode for batch file invocation is chain.) In other words, the call command lets you invoke another batch file as a subroutine. The command line parameters are received by the other batch file as the usual numbered parameters %1, %2, etc.

    It's annoying having to put every subroutine inside its own batch file, so the command interpreter folks added a way to call a subroutine inside the same batch file. The syntax for this is call :label parameter parameter parameter. This is logically equivalent to a batch file recursively calling itself, except that execution begins at the specified label instead of the first line of the file. (It's as if a secret goto label were added to the top of the file.)

    And since it is a batch file, execution of the called subroutine ends when execution falls off the end of the file. And that's where the special goto target comes in handy. At the end of your subroutine, you can jump to the end of the batch file (so that execution falls off the end) by doing a goto :eof.

    In other words, goto :eof is the return statement for batch file subroutines.

    Let's take it for a spin:

    @echo off
    call :subroutine a b c
    call :subroutine d e f
    goto :eof
    echo My parameters are 1=%1, 2=%2, 3=%3
    goto :eof

    That final goto :eof is redundant, but it's probably a good habit to get into, like putting a break; at the end of your last case.

    The subroutine technique is handy even if you don't really care about the subroutine, because stashing the arguments into the %n parameters lets you use the tilde operators to process the inbound parameter.

    @echo off
    call :printfilesize "C:\Program Files\Windows NT\Accessories\wordpad.exe"
    goto :eof
    echo The size of %1 is %~z1
    goto :eof

    Okay, this isn't actually much of a handy trick because you can also do it without a subroutine:

    @echo off
    for %%i ^
    in ("C:\Program Files\Windows NT\Accessories\wordpad.exe") ^
    do echo The size of %%i is %%~zi

    On the other hand, the subroutine trick combines well with the FOR command, since it lets you put complex content in the loop body without having to mess with delayed expansion:

    @echo off
    set DISKSIZE=1474560
    set CLUSTER=512
    set DISKS=1
    set TOTAL=0
    for %%i in (*) do call :onefile "%%i"
    set /a DISKS=DISKS+1
    echo Total disks required: %DISKS%
    goto :eof
    set /a SIZE=((%~z1 + CLUSTER - 1) / CLUSTER) * CLUSTER
    if %SIZE% GEQ %DISKSIZE% (
        echo File %1 does not fit on a floppy - skipped
        goto :eof
        echo ---- need another disk
        set /a DISKS=DISKS+1
        set /a TOTAL=SIZE
    echo copy %1
    goto :eof

    This program calculates the number of floppy disks it would take to copy the contents of the current directory without compression.

    The setlocal command takes a snapshot of the environment for restoration when we perform the endlocal at the end. That will clean up our temporary variables when we're done.

    The first two variables are parameters for the calculation, namely the disk capacity and the cluster size. (We're assuming that the root directory can hold all the files we may ultimately copy. Hey, this is just a demonstration, not a real program.)

    The next two variables are our running total of the number of disks we've used so far, and how many bytes we've used on the last disk.

    The for command iterates over all the files in the current directory. For each one, we call :onefile with the file name.

    The :onefile subroutine does all the real work. First, it takes the file size %~z1 and rounds it up to the nearest cluster. It then sees if that size is larger than a floppy disk; if so, then we're doomed, so we just skip the file. Otherwise, we add the file to the current disk and see if it fits. If not, then we declare the disk full and put the file on a brand new disk.

    After the loop is complete, we print the number of floppy disks we calculated.

    (This algorithm erroneously reports that no files require one disk. Fixing that is left as an exercise.)

    There's your quick introduction to the secret :eof label and batch file subroutines.

    [Raymond is currently away; this message was pre-recorded.]

  • The Old New Thing

    FORFILES, for your fancier batch file enumeration needs


    Crack open open the champagne: Batch File Week is finally over!

    Variations on the for /f %%i in ('dir /b ...') will let you repeat an operation on the contents of a directory, possibly even recursively if you add the /s option, with some basic attribute-level filtering if you add the /a or /a- flags.

    For your fancy recursive file operations, there's a tool called FORFILES which iterates through the contents of a directory (recursively if requested), executing a command on each item it finds. It also has additional filtering capability, like selecting files based on their last-modified time. For example, you could copy all files in the current directory which were modified today:

    forfiles /D +0 /c "cmd /c copy @file \\server\today"

    Unfortuantely, the /D option is not as flexible as one might like. For example, while it can pick files modified today, it can't pick files modified in the last week, because the relative-date-picker knows only how to pick files modified on or before a date in the past or files modified on or after a date in the future. (Who the heck wants to operate on files modified in the future? Except perhaps the Microsoft Research folks who are working on that time machine.)

    You can type FORFILES /? for more information on what you can do (and by seeing what's omitted, what you can't do).

    If the command you want to execute is rather long, you can offload it back into the batch file being executed:

    @echo off
    if "%1"=="/callback" goto callback
    forfiles /D +0 /c "cmd /c call "%~f0" /callback @isdir @file @fsize"
    goto :eof
    rem %2 = @isdir
    rem %3 = @file
    rem %4 = @fsize
    if %2==TRUE echo Skipping directory %3.&goto :eof
    echo Copying file %3 to \\server\today (%4 bytes)

    One gotcha here is that since each command runs in a sub-shell, it can read environment variables, but any modifications it makes to environment variables will be lost since the command is modifying only its local environment variables. A workaround for this is to use FORFILES to select the data to operate on, but use FOR to actually perform the operation. Since FOR runs inside the main command interpreter, it can modify environment variables.

    set TOTALSIZE=0
    for /f %%i in ('forfiles /d +0 /c "cmd /c if @isdir==FALSE echo @fsize"') ^
    do set /a TOTALSIZE=TOTALSIZE + %%i

    Here, we use FORFILES to enumerate all the files (not directories) modified today and print their sizes. We wrap this inside a FOR which reads the sizes and adds them up.

    If the operation you want to perform on each file is complex, you can of course offload it into a subroutine call.

    for /f %%i ^
    in ('forfiles /d +0 /c "cmd /c if @isdir==FALSE echo @fsize"') ^
    do call :subroutine %%i

    I'm cheating here because I know that @fsize doesn't contain spaces. If you are processing file names, then you need to be more careful.

    for /f "tokens=*" %%i ^
    in ('forfiles /d +0 /c "cmd /c if @isdir==FALSE echo @fname"') ^
    do call :subroutine %%i
  • The Old New Thing

    How do I find the most recently created file in a directory from a batch file?


    We've reached Hump Day of Batch File Week. Remember, nobody actually likes batch programming. You merely tolerate it.

    Today, we'll find the most recently-created item in a directory. (For example, we have a server that holds our daily builds, and you might want to write a batch file that automatically installs the latest build.)

    There may be better ways, but what I do is ask for a list sorted oldest-to-newest, and then choose the last one.

    for /f %%i in ('dir /b/a-d/od/t:c') do set LAST=%%i
    echo The most recently created file is %LAST%

    This trick works by asking the dir command to list just the names (/b) of just the files /a-d, sorted by date (/od), based on the creation time (/t:c).

    Each time a new file is reported, its name is stored in the LAST variable, overwriting the previous one. When the loop finishes, the LAST variable contains the name of the newest file, since that's the one that didn't get overwritten.

    You can tweak the command line to perform other queries. For example, if you want the newest file, then just ask for a reverse sort (/o-d). If you want the file sorted by modified time rather than creation time, then use /t:w. You get the idea.

    Limitations: The implementation above assumes that no files contain spaces in their name. Removing this limitation is left as an exercise.

  • The Old New Thing

    Microspeak: planful


    Every year, Microsoft invites its employees to fill out a poll which asks questions regarding all sorts of things. One of the things Microsoft employees are asked to evaluate is whether they think that their vice president is acting planfully.

    The tricky part about that question is that nobody knows exactly what the word planfully means.

    Merriam-Webster defines planful as "full of plans : RESOURCEFUL, SCHEMING." Is that what they're asking us? Whether our vice president is resourceful and scheming? Is that a good thing or a bad thing?

    The OED, on the other hand, defines it as "Full or rich in plans; characterized by planning; organized, systematic." It's not clear whether this means that the person goes into a situation with a plan ahead of time, or that once they get into a situation, they develop a plan for getting out. (Maybe both?)

    I caught a vice president using the word planful during a meeting, so I raised my hand and asked, "What does planful mean?" (Most people don't have the nerve to raise their hand during a meeting and ask, "Um, what does that word mean?")

    The vice president replied, "Actually, I don't think it's a real word. I think [name of a senior manager] made it up."

    I followed up: "Well, in that case, why are we asked to evaluate you every year on whether your actions are planful? How can we answer a question that uses a made-up word nobody knows the definition of?"

    Obligatory xpclient example: "Was it a planful decision to introduce the auto-sorting problem?"

    Reminder: Microspeak is not merely terms unique to Microsoft, but also terms used at Microsoft more often than in general.

  • The Old New Thing

    What is SysFader and why is it always crashing?


    If you type SysFader into your favorite search engine, you'll find lots of hits from people asking, "What is SysFader, and why is it always crashing Internet Explorer?"

    SysFader: iexplore.exe - Application Error
    The exception unknown software exception (0xe06d7363) occurred in the application at location 0x7c812afb.

    When a program encounters a fatal error, the system crash dialog appears. And it needs to put somebody's name in the title of the dialog to indicate which application crashed. Sure, it has the process name (iexplore.exe), but it has this nagging feeling that it can do better. After all, not everybody will know that "AcroRd32.exe" is "The menu for my favorite restaurant that I was looking at in Adobe Acrobat Reader". So it goes looking for a window that belongs to the thread so it can steal the window's title and use that to help the user understand what it was that crashed.

    And if can't find any visible windows, it will go for invisible ones, on the theory that, "Well maybe the application crashed before it could show the window."

    Now let's see what happens when we apply this logic to SysFader.

    SysFader is a helper window used by Internet Explorer to perform fade-out animations. It really doesn't do much, but it is a window, albeit an invisible one when there are no animations in progress.

    SysFader happens to run on a shared worker thread. If that worker thread is being borrowed by some other background task, and that background task crashes, then when the crash dialog appears and looks around for a window to put in the title, it says "Rats, I don't have any visible windows, but I do have this invisible one, so I'll go ahead and put that one in the title bar. Better than nothing."

    It's sort of the error-reporting version of the Politician's Fallacy:

    1. A window must be blamed.
    2. This is a window.
    3. Therefore, we must blame it.

    It's like your photo appearing in a newspaper article headlined Robbery at Woodgrove Bank, Suspect At Large, not because you're the suspect, but because you happen to have been in the building at the time of the robbery.

    Bonus chatter: You probably recognize the exception code as an unhandled C++ exception. Internet Explorer doesn't use C++ exceptions, so the exception most likely came from a plug-in.

    [Raymond is currently away; this message was pre-recorded.]

  • The Old New Thing

    The shifting sands of "Run as different user"


    A customer liaison asked the following question on behalf of his customer.

    When I do a shift-right-click on a shortcut to a program, I see the following:

    • On Windows Server 2008, shift-right-click does not offer the option Run as different user.
    • On Windows 7, shift-right-click does offer the option Run as different user.

      On Windows Server 2008 R2 (the server counterpart to Windows 7), shift-right-click does offer the option Run as different user.

    The option to run a program as another user (other than Administrator) was present in Windows XP, but it was lost in Windows Vista. It appears that we responded to those complaints by restoring the functionality in Windows 7.

    Is that right?

    The odd thing is that my customer has the Run as different user option available on their Windows 7 machines, but not on their Windows Server 2008 R2 machines. Does whether you have access to Run as different user depend on how you installed Windows Server 2008 R2? (If it matters, my customer installed it via the Microsoft Deployment Toolkit.)

    I found this question interesting for a variety of reasons.

    First of all, it comes dangerously close to being one of those Please tell me I'm not hallucinating type of requests. These support requests take the following peculiar form:

    We did X, then Y, then Z, and then Q happened. Is that right?

    "Um, if you say so. I wasn't there."

    But in this case, it started out looking like it was going to turn into a Please tell me I'm not hallucinating request, then veered into "Is X the reason the feature was added back?"

    This is a rather peculiar question, because knowing the answer one way or the other doesn't actually take you any closer to a solution to the problem. (And I don't know the answer, but fortunately, it wasn't relevant to solving the customer's problem.)

    Another interesting thing about the customer's question is that he never actually comes out and asks the question. He sort of says a few related things, and asks a tangential question, but never comes right out and asks, "How do I get the Run as different user option to work on my Windows Server 2008 R2 machine?"

    It's like a kid who pointedly hangs around a candy bowl, hoping that an adult will say, "Here, have a piece of candy."

    You're a grown-up now. You don't have to linger around the candy bowl hoping somebody will figure out that you want some candy. You should just ask, "May I have a piece of candy?"

    My psychic powers tell me that they have set the Require trusted path for credential entry policy. The Run as different user feature is disabled if you set this policy.

    The customer liaison replied, "It appears that the option for Require trusted path for credential entry is not enabled. The customer is going to do a clean install and test on a non-domain-joined machine to avoid any GPOs."

    Some time passed, and the customer liaison reported back with a resolution.

    The culprit was indeed the Require trusted path for credential entry policy. It didn't show up in a GPO search because they were setting the policy via a script rather than deploying a group policy object.

    It was very nice of the customer liaison to reply with confirmation that the problem was solved. This is, unfortunately, a notable event. Most of the time, people never report back if your suggestion solved their program; they only come back if your suggestion didn't help. Which means you're not sure if your suggestion solved the problem, or if it didn't solve the problem but they decided to continue the investigation somewhere else.

    Bonus chatter: This shows yet another reason why you should use Group Policy Objects to manage group policies rather than custom scripts that whack registry keys. In addition to the fact that registry key whacking may not work, there are tools for processing Group Policy Objects that make this sort of investigation much easier.

  • The Old New Thing

    Command line tool to manage Windows 7 Libraries, with source code


    A customer asked if there was a command-line tool for managing Windows 7 Libraries so that they could create and deploy libraries across their organization.

    Not only is there such a tool, it even comes with source code.

    The Shell Library Command Line Sample shows how to manage Windows 7 libraries programmatically via the IShell­Library interface. And it's actually a useful program on its own.

    Usage: shlib.exe SUBCOMMAND
    Displays and modifies the attributes of Shell Libraries.
    Supported commands:
      create      Creates a library at the specified path.
      info        Prints info about the given library.
      enum        Enumerates the folders in the library.
      setattrib   Modifies the attributes of the library.
      add         Adds the specified folder to the specified library.
      remove      Removes the specified folder from the library.
      setsaveloc  Sets the default save location of the library.
      resolve     Resolves the specified folder in the library.
      resolveall  Resolves all locations in the library in bulk.
      manage      Displays the Manage Library Dialog for the library.

    For example, to see all the folders that are part of your Documents library, use the command

    shlib enum FOLDERID_DocumentsLibrary

    Each of the commands has further sub-options.

    Usage: create shlib.exe [OPTIONS]  [...]
    Creates a library at the specified path.
     -create[:ARG]       Specifies that a new library should be created.
       <none>              Fail if the library already exists.
       overwrite           Overwrite any existing library.
       uniquename          Generate a unique name in case of conflict.
    The library may be specified by a file system path,
    or by a KNOWNFOLDERID (e.g. "FOLDERID_DocumentsLibrary").

    And the source code shows how to perform each of these actions programmatically.

    The IShellLibrary interface is also supported by the Windows API Code Pack for .NET, so you C# folks can use it too.

    (One thing I noticed that is missing from the sample is a seticon command. Shouldn't be hard to implement. You just call IShell­Library::Set­Icon.)

  • The Old New Thing

    Why am I in the Quake credits?


    Anon wants to know why I am listed in the credits for the video game Quake under the "Special Thanks" section. "Were you an early tester/debugger?"

    I've never played a game of Quake in my entire life. I (and most of the rest of the Windows 95 team) played DOOM, but after a while, first-person-shooter games started giving me a headache. By the time Quake came out, I had already abandoned playing FPS games.

    I don't remember what it was that I did specifically, but it was along the lines of helping them with various technical issues related to running under Windows. At the time, I was a kernel developer, and the advice I gave was almost certainly related to memory management and swapping.

    Sorry it wasn't anything exciting.

  • The Old New Thing

    Dumping a hash table with external chaining from the debugger


    I was doing some crash dump debugging, as I am often called upon to do, and one of the data structure I had to grovel through was something that operated basically like an atom table, so that's what I'll call it. Like an atom table, it manages a collection of strings. You can add a string to the table (getting a unique value back, which we will call an atom), and later you can hand it the atom and it will give you the string back. It looked something like this:

    struct ENTRY
      ENTRY *next;
      UINT   atom;
      PCWSTR value;
    #define ATOMTABLESIZE 19
    struct ATOMTABLE
      ENTRY *buckets[ATOMTABLESIZE];

    (It didn't actually look like this; I've reduced it to the smallest example that still illustrates my point.)

    As part of my debugging, I had an atom and needed to look it up in the table. A program would do this by simply calling the "here is an atom, please give me the string" function, but since this was a crash dump, there's nothing around to execute anything. (Not that having a live machine would've made things much easier, because this was a kernel-mode crash, so you don't get any of this edit-and-continue froofy stuff. This is real debugging™.)

    But even though the crashed system can't execute anything, the debugger can execute stuff, and the debugger engine used by kd comes with its own mini-programming language. Here's how I dumped the atom table:

    // note: this was entered all on one line
    // broken into two lines for readability
    0: kd> .for (r $t0=0; @$t0 < 0n19; r $t0=@$t0+1)
             { dl poi (fffff8a0`06b69930+@$t0*8) 99 2 }
    fffff8a0`06ad3b90  fffff8a0`037a3fc0 fffff8a0`0000000c \
    fffff8a0`037a3fc0  fffff8a0`037a4ae0 00000000`00000025 | $t0=0
    fffff8a0`037a4ae0  fffff8a0`02257580 00000000`00000036 |
    fffff8a0`02257580  00000000`00000000 00000000`00000056 /
    fffff8a0`06ad3b30  fffff8a0`06ad3ad0 a9e8a9d8`0000000d \
    fffff8a0`06ad3ad0  fffff8a0`037a4700 000007fc`0000000e |
    fffff8a0`037a4700  fffff8a0`01f96fb0 00000000`0000003f | $t0=1
    fffff8a0`01f96fb0  fffff8a0`06cfa5d0 fffff8a0`00000044 |
    fffff8a0`06cfa5d0  00000000`00000000 00181000`00000060 /
    fffff8a0`033e7a70  fffff8a0`037a4770 00000020`00000023 \
    fffff8a0`037a4770  fffff8a0`023b52f0 00000000`0000003e | $t0=2
    fffff8a0`023b52f0  fffff8a0`03b6e020 006f0063`00000059 |
    fffff8a0`03b6e020  00000000`00000000 00000000`00000075 /
    fffff8a0`037a0670  fffff8a0`02b08870 fffff8a0`00000026 \ $t0=3
    fffff8a0`03b9e390  00000000`00000000 00240000`00000071 /

    Let's take that weirdo command apart one piece at a time.

    The overall command is

    .for (a; b; c) { d }

    This operates the same as the C programming language. (Sorry, Delphi programmers, for yet another C-centric example.) In our case, we use the $t0 pseudo-register as our loop control.

    • r $t0=0 — this sets $t0 to zero
    • @$t0 < 0n19 — this stops once $t0 reaches 19.
    • r $t0=@$t0+1 — this increments $t0.

    Note that here as well as in the loop body, I prefix the register name with the @ character when I want to obtain its value, in order to force it to be interpreted as a register. (Otherwise, the debugger will look for a symbol called $t0.)

    The command being executed at each iteration is { dl poi (fffff8a0`06b69930+@$t0*8) 99 2 }. Let's break this down, too:

    • dl — this command dumps a singly-linked list.
    • poi (fffff8a0`06b69930+@$t0*8) — this is the head of the linked list. In this example, 0xfffff8a0`06b69930 is the address of the buckets array, so we add the loop counter times the size of a pointer (8, in this case) to get the address of the $t0'th entry, and then dereference it (poi) to get the address of the head of the linked list.
    • 99 — This is the maximum length of the linked list. I picked an arbitrary large-enough number. I like using 9's because it carries the most value per keypress. Other people like to use nice round numbers like 1000, but 999 saves you a whole keypress and is just one less. (On the other hand, I don't use fff because that runs the risk of being misinterpreted as a symbol.)
    • 2 — This is the number of pointer-sized objects to dump at the start of each node. For 32-bit code, I use 4, whereas for 64-bit code, I use 2. Why those values? Because those are the maximum number of pointer-sized objects that the debugger will print on one line.

    Okay, so now I have that linked list dump. The value I'm looking for is the atom whose value is 0x3F, so I skim down the last column looking for 0000003f, and hey there it is.

    fffff8a0`037a4700  fffff8a0`01f96fb0 00000000`0000003f

    Now I have my ENTRY, and I can dump it to see what the corresponding value is.

    0: kd> dt contoso!ENTRY fffff8a0`037a4700
        +0x000 next: 0xfffff8a0`01f96fb0
        +0x008 atom: 0x0000003f
        +0x010 value: 0xffff8a0`01f97e20 -> "foo"
  • The Old New Thing

    What happened to the Windows 2000 "Language settings for the system" control panel?


    In 2011, a customer had a question about migrating from Windows 2000 to Windows XP. (That's right, this customer was still using Windows 2000 in the year 2011.) Specifically, they noted that in Windows 2000, they can select multiple languages in the "Language settings for the system" portion of the Regional Options control panel, and they couldn't find the corresponding control panel setting in Windows XP.

    Regional Options
    Input Locales
    Settings for the current user
    Many programs support international settings for numbers, currencies, times, and dates. Set the locale in order to use the standard settings.
    Your locale (location):
    English (United States)
    Language settings for the system
    Your system is configured to read and write documents in multiple languages.
    Central Europe

    In Windows 2000, "Language settings for the system" provides the option to install support (such as code pages, keyboard layouts, and fonts) for various language groups. In Windows XP, the big list of language groups was reduced to three categories:

    • Basic (Baltic, Central Europe, Cyrillic, Greek, Turkish, Western Europe)
    • Complex (Arabic, Armenian, Georgian, Hebrew, Indic, Vietnamese, Thai)
    • East Asia (Chinese Simplified, Chinese Traditional, Japanese, Korean)

    The Basic category is always installed. To install the Complex or East Asia categories, use the "Supplemental language support" section of the Regional and Language Options control panel.

    Windows XP Regional and Language Options property sheet, with a section titled "Supplemental language support" with options "Install files for complex script and right-to-left languages (including Thai)" and "Install files for East Asian languages

    Someday, that customer might upgrade to Windows Vista, so I may as well answer the question right now. In Windows Vista and onward, things were simplified even more: All language groups are installed at all times. The dialog box went away completely since there were no options remaining.

    As it turns out, the customer's problem had nothing to do with language support. Of course, they didn't come out and describe the problem they were having; rather, they reduced the problem into multiple pieces, and then asked for help on one specific piece. They tried out a solution based on this new information, but it didn't solve the problem, because as it turns out, the Language settings for the system control panel was a red herring. If they had told us what their original problem was, we could have told them "But this setting will do nothing to solve your problem. What you really need is over there."

    Tomorrow, we'll look at the customer's actual problem. (So please don't try to guess or you'll ruin the surprise. I can't believe I had to write that.)

Page 1 of 3 (29 items) 123