Jeremy on stuff

For, Part 2

JMorton — Fri, 22 Jul 2005 02:41:00 GMT

Continuing from last time.

If Command Extensions are enabled, the following additional
forms of the FOR command are supported:

[Note that Windows 2000 and later have Command Extensions enabled by default.]

FOR /D %variable IN (set) DO command [command-parameters]

    If set contains wildcards, then specifies to match against directory
    names instead of file names.

For example, to delete every subdirectory that starts with an underscore in the current directory, you could run this on the command line: [Typo fixed 7/25 12:30 PM]

for /D %D in (_*) do rd /s/q "%D"

Note the quotes in the rd command; these are there in order to make sure that it does the right thing if a directory happens to contain spaces in its name.

FOR /R [[drive:]path] %variable IN (set) DO command [command-parameters]

    Walks the directory tree rooted at [drive:]path, executing the FOR
    statement in each directory of the tree.  If no directory
    specification is specified after /R then the current directory is
    assumed.  If set is just a single period (.) character then it
    will just enumerate the directory tree.

For example, to list the path of every .h file under the current directory, you could run this on the command line [this is a contrived example, "dir /s/b *.h" would be more efficient]:

for /R %F in (*.h) do @echo %F

[I'll cover the usage of the @ character in a later post.]

Another example: to rename every .c file under the C:\Source directory to a .cpp file, you could run this on the command line:

for /R C:\Source %D in (.) do ren "%D\*.c" *.cpp

Note the quotes in the ren command; these are there in order to make sure that it does the right thing if a directory happens to contain spaces in its name.

FOR /L %variable IN (start,step,end) DO command [command-parameters]

    The set is a sequence of numbers from start to end, by step amount.
    So (1,1,5) would generate the sequence 1 2 3 4 5 and (5,-1,1) would
    generate the sequence (5 4 3 2 1)

For example, to defragment your C: drive 5 times in a row and log the results of each one, you could run this on the command line:

for /L %N in (1,1,5) do defrag C: > defrag%N.log 2>&1

2>&1 redirects the standard error [2>] to the same place that the standard output is going [1> or just >]. I'll post on those at some later time.

For, Part 1

JMorton — Fri, 17 Jun 2005 09:00:00 GMT

Continuing from last time.

The usage for the basic for loop looks like:

FOR %variable IN (set) DO command [command-parameters]

  %variable  Specifies a single letter replaceable parameter.
  (set)      Specifies a set of one or more files.  Wildcards may be used.
  command    Specifies the command to carry out for each file.
  command-parameters
             Specifies parameters or switches for the specified command.

For example, to copy every file in the current directory into its own subdirectory named after the file under the \Test directory, you could run this on the command line:

for %F in (*) do xcopy "%F" "\Test\%F\"

This would copy the file foo.txt to \Test\foo.txt\foo.txt and bar.bmp to \Test\bar.bmp\bar.bmp. Note the quotes in the xcopy command; these are there in order to make sure that it does the right thing if a file happens to contain spaces in its name.

The next part of the help message is:

To use the FOR command in a batch program, specify %%variable instead
of %variable.  Variable names are case sensitive, so %i is different
from %I.

This can be troublesome when writing about a for loop [as in email or in a blog]. You have to assume that the reader knows about it, or you have to explain it every time you write a for loop. Even if you type a for loop in someone's Command Prompt, you probably have to explain that if they want to put the line in a batch file, they'll have to replace, e.g. %F with %%F throughout the for loop. For instance, the above for loop would look like this in a .bat or .cmd file:

for %%F in (*) do xcopy "%%F" "\Test\%%F\"

For this reason, I will use a single % in for loops on this site because it's easier to type [and possibly easier to read] and assume that the reader understands this point.

The for Command

JMorton — Fri, 17 Jun 2005 04:58:00 GMT

I stated earlier that the command prompt can be a very powerful scripting shell. Much of this power comes from the "for" command. Let's look into it more closely.

First, the text of "help for" on my Windows XP SP2 machine:

Runs a specified command for each file in a set of files.

FOR %variable IN (set) DO command [command-parameters]

  %variable  Specifies a single letter replaceable parameter.
  (set)      Specifies a set of one or more files.  Wildcards may be used.
  command    Specifies the command to carry out for each file.
  command-parameters
             Specifies parameters or switches for the specified command.

To use the FOR command in a batch program, specify %%variable instead
of %variable.  Variable names are case sensitive, so %i is different
from %I.

If Command Extensions are enabled, the following additional
forms of the FOR command are supported:

FOR /D %variable IN (set) DO command [command-parameters]

    If set contains wildcards, then specifies to match against directory
    names instead of file names.

FOR /R [[drive:]path] %variable IN (set) DO command [command-parameters]

    Walks the directory tree rooted at [drive:]path, executing the FOR
    statement in each directory of the tree.  If no directory
    specification is specified after /R then the current directory is
    assumed.  If set is just a single period (.) character then it
    will just enumerate the directory tree.

FOR /L %variable IN (start,step,end) DO command [command-parameters]

    The set is a sequence of numbers from start to end, by step amount.
    So (1,1,5) would generate the sequence 1 2 3 4 5 and (5,-1,1) would
    generate the sequence (5 4 3 2 1)

FOR /F ["options"] %variable IN (file-set) DO command [command-parameters]
FOR /F ["options"] %variable IN ("string") DO command [command-parameters]
FOR /F ["options"] %variable IN ('command') DO command [command-parameters]

    or, if usebackq option present:

FOR /F ["options"] %variable IN (file-set) DO command [command-parameters]
FOR /F ["options"] %variable IN ('string') DO command [command-parameters]
FOR /F ["options"] %variable IN (`command`) DO command [command-parameters]

    filenameset is one or more file names.  Each file is opened, read
    and processed before going on to the next file in filenameset.
    Processing consists of reading in the file, breaking it up into
    individual lines of text and then parsing each line into zero or
    more tokens.  The body of the for loop is then called with the
    variable value(s) set to the found token string(s).  By default, /F
    passes the first blank separated token from each line of each file.
    Blank lines are skipped.  You can override the default parsing
    behavior by specifying the optional "options" parameter.  This
    is a quoted string which contains one or more keywords to specify
    different parsing options.  The keywords are:

        eol=c           - specifies an end of line comment character
                          (just one)
        skip=n          - specifies the number of lines to skip at the
                          beginning of the file.
        delims=xxx      - specifies a delimiter set.  This replaces the
                          default delimiter set of space and tab.
        tokens=x,y,m-n  - specifies which tokens from each line are to
                          be passed to the for body for each iteration.
                          This will cause additional variable names to
                          be allocated.  The m-n form is a range,
                          specifying the mth through the nth tokens.  If
                          the last character in the tokens= string is an
                          asterisk, then an additional variable is
                          allocated and receives the remaining text on
                          the line after the last token parsed.
        usebackq        - specifies that the new semantics are in force,
                          where a back quoted string is executed as a
                          command and a single quoted string is a
                          literal string command and allows the use of
                          double quotes to quote file names in
                          filenameset.

    Some examples might help:

FOR /F "eol=; tokens=2,3* delims=, " %i in (myfile.txt) do @echo %i %j %k

    would parse each line in myfile.txt, ignoring lines that begin with
    a semicolon, passing the 2nd and 3rd token from each line to the for
    body, with tokens delimited by commas and/or spaces.  Notice the for
    body statements reference %i to get the 2nd token, %j to get the
    3rd token, and %k to get all remaining tokens after the 3rd.  For
    file names that contain spaces, you need to quote the filenames with
    double quotes.  In order to use double quotes in this manner, you also
    need to use the usebackq option, otherwise the double quotes will be
    interpreted as defining a literal string to parse.

    %i is explicitly declared in the for statement and the %j and %k
    are implicitly declared via the tokens= option.  You can specify up
    to 26 tokens via the tokens= line, provided it does not cause an
    attempt to declare a variable higher than the letter 'z' or 'Z'.
    Remember, FOR variables are single-letter, case sensitive, global,
    and you can't have more than 52 total active at any one time.

    You can also use the FOR /F parsing logic on an immediate string, by
    making the filenameset between the parenthesis a quoted string,
    using single quote characters.  It will be treated as a single line
    of input from a file and parsed.

    Finally, you can use the FOR /F command to parse the output of a
    command.  You do this by making the filenameset between the
    parenthesis a back quoted string.  It will be treated as a command
    line, which is passed to a child CMD.EXE and the output is captured
    into memory and parsed as if it was a file.  So the following
    example:

      FOR /F "usebackq delims==" %i IN (`set`) DO @echo %i

    would enumerate the environment variable names in the current
    environment.

In addition, substitution of FOR variable references has been enhanced.
You can now use the following optional syntax:

    %~I         - expands %I removing any surrounding quotes (")
    %~fI        - expands %I to a fully qualified path name
    %~dI        - expands %I to a drive letter only
    %~pI        - expands %I to a path only
    %~nI        - expands %I to a file name only
    %~xI        - expands %I to a file extension only
    %~sI        - expanded path contains short names only
    %~aI        - expands %I to file attributes of file
    %~tI        - expands %I to date/time of file
    %~zI        - expands %I to size of file
    %~$PATH:I   - searches the directories listed in the PATH
                   environment variable and expands %I to the
                   fully qualified name of the first one found.
                   If the environment variable name is not
                   defined or the file is not found by the
                   search, then this modifier expands to the
                   empty string

The modifiers can be combined to get compound results:

    %~dpI       - expands %I to a drive letter and path only
    %~nxI       - expands %I to a file name and extension only
    %~fsI       - expands %I to a full path name with short names only
    %~dp$PATH:I - searches the directories listed in the PATH
                   environment variable for %I and expands to the
                   drive letter and path of the first one found.
    %~ftzaI     - expands %I to a DIR like output line

In the above examples %I and PATH can be replaced by other valid
values.  The %~ syntax is terminated by a valid FOR variable name.
Picking upper case variable names like %I makes it more readable and
avoids confusion with the modifiers, which are not case sensitive.

Wow, that's a lot of options. I'll cover the first couple of these in the next few days.

About Me

JMorton — Fri, 17 Jun 2005 04:47:00 GMT

My name is Jeremy Morton; I have been working at Microsoft for nearly 7 years [for nearly 2 years as a contractor, and as an employee for the last 5]. Currently, I am an SDET [Software Design Engineer in Test] for an internal server product, but most of my career here has been as a Builder¹, first in WMI [as part of Systems Management Server then as part of Windows], and then in the Developer Division, first building the .NET Development Platform then Visual Studio.

After Windows 2000 shipped, I worked to make WMI build in the Windows build environment. I then worked in one of the Windows Build Labs for a few months until I was hired full time in the Developer Division in April, 2000. In the Developer Division, besides my build duties, I started managing the server [for the product I now test] for the NDP team. As Visual Studio .NET [7.0] shipped and we were working on Visual Studio .NET 2003 [7.1], I was part of the Build Working Group that set up the development process for the next version of Visual Studio, which was based on the Windows process. Besides being a builder, I was the principle administrator for the server [for the product I now test] for the entire Developer Division until Fall of 2004, when I became an SDET on my current team.

During this time, I have worked a lot with the Windows OS, both on the client and server side, and have gotten a lot of experience with building and using the Windows Command Prompt for scripting. [Contrary to a lot of people's misconceptions, on Windows 2000 and up, the command prompt can be a very powerful scripting shell, although it definitely has weaknesses in some areas such as string manipulation.] I intend to pass on some of the things that I've learned about using Windows and building through this blog. I also use a fairly diverse set of Microsoft products at home, and I may comment on them here, as well. I tend to work best when I'm answering specific questions, so I'd love to get questions related to these things. However, I do not intend to discuss the product that I currently work on.

¹Builders regularly [often daily] take the source code checked into the Source Control System by Developers and compile them into the product as distributed to customers. The Testers then take this and test it, reporting bugs for developers to fix and check in, etc. A build team is usually much smaller than the Developer or Test teams, often by a couple of orders of magnitude. There are many different things that need to be done to build a product, including compiling, code-signing, propagating to a build server in a logical manner, maintaining debug symbols, building cab files/setup files/cd images, and a lot of other miscellaneous steps. Because of the variety, it offers a wide variety of problems to solve daily. Due to the high turnaround rate for builds, it is often a high stress, tight deadline team where bottlenecks can stall an entire Division. I loved it. :)