Mark Yocom's Script Tips

Windows command line references

myocom — Tue, 29 Jan 2008 01:32:00 GMT

I found that TechNet has references for the Windows command line utilities. Much of these are just a rehash of the help you'd get with /? but at least they're easier to read...

This is a list of commands for Windows Server 2003 (most apply to Windows XP as well), and here is the list for Vista and Server 2008.

On blogging tools

myocom — Sat, 26 Aug 2006 07:02:00 GMT

One of the obstacles to frequent posting (at least mine) is that the types of posts I typically make include code. Code that I would like to be formatted nicely. Unfortunately, this previously meant hand-crafting HTML to make it all look nice. In some cases, I could write up scripts in Visual Studio and paste them into an editor which would (mostly) preserve the formatting, but having to write batch files in VS seems hugely overkill. Anyway, since that was such a pain, I didn't post often.

Enter Windows Live Writer. Everyone else is blogging about it, so hey, why shouldn't I? Out of the box, it's very, very nice (and free, to boot!). It's a truly WYSIWYG editor (not the typical WYSI-more or less-WYG-with a few little problems here and there) and that handles all of my typical blogging needs. Except the whole code formatting thing. Aha, you say, how does that help?

Easy.

WLW supports plugins, and they're (evidently) easy to develop. Even though it's still in beta, there are dozens of plugins already. A few of those plugins enable code formatting. I've chosen to use Steve Dunn's cleverly named Code Formatter Plugin, which uses Actipro's CodeHighlighter web control. Most of the code highlighters out there only seem to support the Visual Studio types of code (your C, your VB.Net, etc.). Try to get scripts highlighted and forget about it. This one does, though, and even though it's not quite the same style of highlighting I'm used to, it's way better than nothing.

Take the scriptlet I wrote in this post. Hand-styling that was a huge pain, and all I really did was keep the indenting and ensure it was all in a fixed-width font. But all that indenting added up to a lot of  's and it was easy to lose track of how many I'd already done.

With this plugin to WLW, though, it's a matter of clicking "Insert Formatted Code" or "Insert Clipboard as Code" and you get this instead:

1 for /f "delims=/ tokens=1-3" %%a in ("%DATE:~4%") do (
2     for /f "delims=:. tokens=1-4" %%m in ("%TIME%") do (
3         set FILENAME=basename-%%c-%%b-%%a-%%m%%n%%o%%p
4     )
5 )

Line numbers and everything. In the future, I'll skip the line numbers for bits of script I intend for people to copy and paste, but to illustrate points, I'll include them.

Ok, enough rah-rah from me. I have scripts to write.

(But seriously. If you blog, this is an awesome tool.)

Definitive list of illegal filenames?

myocom — Tue, 11 Jul 2006 22:11:00 GMT

Earlier, I tried to come up with a definitive list of illegal filesystem names, but it seems this is already documented. Who knew?

http://support.microsoft.com/Default.aspx?id=74496

More on command line redirection

myocom — Wed, 17 May 2006 20:30:00 GMT

For those of you who are more interested in the gritty details of command line redirection, RaymondC has posted a couple of great articles recently on his excellent The Old New Thing blog.

The first discusses how (and more importantly, why) the redirection symbols are parsed by CMD.EXE rather than the program you're calling.

The second points out some of the caveats of blindly accepting input from environment variables. While it may be a pain to quote the parameters your script takes, it can make troubleshooting easier and, as his post points out, can prevent your day from being ruined.

Evolution of a solution

myocom — Sat, 01 Oct 2005 08:15:00 GMT

The other day, someone posted a message at work asking for a way to find out the drive letter for partition 1 on the first disk.

The first response offered this:

C:>echo list volume | diskpart
Microsoft DiskPart version 5.1.3565

Copyright (C) 1999-2003 Microsoft Corporation.

On computer: SINISTAR

DISKPART>
  Volume ###  Ltr  Label        Fs     Type        Size     Status     Info
  ----------  ---  -----------  -----  ----------  -------  ---------  --------
  Volume 0     F                       DVD-ROM         0 B
  Volume 1     C   BOOTDRIVE    FAT32  Partition      9 GB  Healthy    System
  Volume 2     D   Application  NTFS   Partition     15 GB  Healthy    Pagefile
  Volume 3     E   SharedAndUn  NTFS   Partition     10 GB  Healthy

DISKPART>

C:\>

The original poster then asked if anyone already had a script to parse the output of diskpart. I helpfully offered this:

@for /f "tokens=3" %%x in ('echo list volume ^| diskpart ^| findstr /c:"Volume %1"') do @echo %%x

"How clever of me," thought I. "Simply pass in a volume number and it neatly spits out the drive letter." I was all proud of myself until someone pointed out that we already had a perfectly good .exe that did the same thing.

The moral? It's all fun and games 'til you find out you just reinvented the wheel...

Making your scripts look prettier

myocom — Sat, 01 Oct 2005 06:48:00 GMT

Many languages provide a mechanism to make their code look pretty. VBScript, for example, assumes one statement per line, but including an underscore at the end of the line indicates that your statement continues on the next line.

Normal
WScript.Echo "This is line one" & vbcrlf & "This is line two" & vbcrlf & "This is line three"

With Line Continuation Characters
WScript.Echo "This is line one" & vbcrlf & _
             "This is line two" & vbcrlf & _
             "This is line three"

It's much easier to tell what's going on in the second script.

In batch, you often see variables "built up" like this:

set MACHINENAMES=sinistar
set MACHINENAMES=%MACHINENAMES% tempest
set MACHINENAMES=%MACHINENAMES% joust

...and in the end you have a variable with three items in it.

As it happens, you can take advantage of CMD's escape character, the carat, to make this a little easier to read:

set MACHINENAMES=sinistar^
                 tempest^
                 joust

And again you have a variable with three items in it. The second example does have one (largish) difference, though. Rather than MACHINENAMES being "sinistar tempest joust" (as in the first example), it will have "sinistar        tempest        joust". If spacing is important to what you're doing, then by all means stick to the first way. However, if you're only building up a list to use in a for loop, the second will work just fine since for splits its arguments on one or more whitespace characters, so the following are equivalent:

for %x in (%MACHINENAMES%) do @echo %x
for %x in (sinistar tempest joust) do @echo %x
for %x in (sinistar         tempest         joust) do @echo %x

Admittedly, I've only used this once or twice. Generally the lists I have to keep are pretty small, but on the occasions where I have very long lists, it can come in handy.

That's a pretty shifty argument...

myocom — Thu, 30 Jun 2005 23:00:00 GMT

It's pretty easy to deal with up to 9 arguments in batch through built-in variables %1 through %9. For example:

    echoargs.cmd
    @echo off
    echo %1
    echo %2
    echo %3
    echo %4
    echo %5
    echo %6
    echo %7
    echo %8
    echo %9

If you feed this A B C D E F G H I, you'll get back those letters, one per line.

But, if you try this with a tenth argument, it won't work like you might have thought:

    echo10args.cmd
    @echo off
    echo %1
    echo %2
    echo %3
    echo %4
    echo %5
    echo %6
    echo %7
    echo %8
    echo %9
    echo %10

You'll see that the last line echoes the first argument again, followed by a 0 (A0). That is, it gets parsed as "echo %1, then 0". So, how do you deal with this if you need more than 9 arguments? My first suggestion would be to see if there's a way you can deal with fewer arguments, though that's often not possible. If you must, you'll need to loop over each argument and deal with them one by one. Here's an example listing:

    echoallargs.cmd
    @echo off

    :Top
    REM If there aren't any arguments to deal with, bail out
    if "%1" equ "" goto :NoMoreArgs

    call :EchoArg %1
    shift

    REM Start the loop over again with a brand new %1
    goto :Top

    :NoMoreArgs
    echo No more arguments to deal with
    goto :EOF

    :EchoArg
    echo Arg: %1
    goto :EOF

Let's call this script with 11 arguments: A B C D E F G H I J K

At :Top, we check to see if there are any arguments to deal with. If not, we go to :NoMoreArgs and bail out. Then (assuming we didn't bail out), we call :EchoArg with the parameter that's first in the list (in this case, A). :EchoArg takes the parameter it was passed (A) and echos Arg: A. Once it's done, we use goto :EOF to indicate that we're done with this subroutine. Control then passes back up to the line just after our call.

This next step is where all the magic happens. The shift command takes everything you've passed in and shifts them all to the left by one. That is, what used to be %3 becomes %2, %2 becomes %1, and %1 falls off. So now, since there was a tenth parameter, that one becomes %9, which means there's now a way of referring to it.

Once we've shifted everything, we go back to the :Top again. Now since our original second argument (B) has been shifted to %1, we do the same thing all over again. That gets echoed, we shift again, and C is the new %1 (and K is the new %9). We keep doing this until we run out of arguments, and then we bail out.

There are a few important points in here:

For anything you call, whether it's a "subroutine" like :EchoArg or another script (like call foo.cmd), the arguments passed to it are "local". That is, if you were to call :EchoArg %8 (that is, H in our case), as far as :EchoArg is concerned, that's %1. It's its own first argument.
The built-in variable %* (meaning all arguments passed in) is not at all affected by shift. In our example, no matter how much you shift, %* will still be A B C D E F G H I J K. This is sort of unfortunate, in my opinion, since there are times that I've found that I wanted to deal with the first (say) three arguments, then refer to "the rest" in one fell swoop, but the only way to do that is shift your way through them one by one.

While we're on the topic of shift, you can see from shift /? that it can also take a switch that indicates where you want to start shifting. This takes a bit of wrapping your brain around, but it goes like this:

Given a list of arguments A B C D E F G H I J K, a regular shift would change that to B C D E F G H I J K.

But, we could also use shift /3 (for example) to start the shifting at the third argument. So our A B C D E F G H I J K becomes A B D E F G H I J K; the third argument (C) gets swallowed. Note that even though you can address 9 elements at a time %1 through %9, you can't do shift /9 and have the tenth argument become the ninth. You're limited to shift /8.

Oh well, you can't have everything...

Ignoring the output of a command

myocom — Thu, 09 Jun 2005 02:00:00 GMT

Earlier, I used >nul 2>&1 to suppress the output of a command, and the question was brought up as to why this was "better" than just >nul.

To understand what's going on, you need to know that there are two streams of output from any given command. There's STDOUT (or "standard out", or "standard output") and STDERR (or "standard error"). Well-behaved executables will output error messages to STDERR and regular output to STDOUT. In CMD by default, both STDOUT and STDERR are echoed to the console. You can redirect this output by using >. Typically this is done to redirect output to a file. However, this only redirects STDOUT to the file; STDERR will still display on the screen.

On the command line, when redirecting output, 1 (or nothing) means STDOUT and 2 means STDERR. That is, foo.exe >foo.txt redirects foo.exe's STDOUT to foo.txt, and its STDERR (if any) will display on the screen. Running foo.exe 1>foo.txt would do the same thing. To deal with STDERR, use 2>, as in foo.exe 2>foo.txt.

So, if you wanted all output from foo.exe, you could run foo.exe >foo.txt 2>foo.txt. As it happens, since that's a lot of typing, CMD has this little shortcut for tying one stream's redirection to another: foo.exe >foo.txt 2>&1 means "Run foo.exe and redirect its STDOUT to foo.txt, and redirect its STDERR to wherever STDOUT goes."

This is all well and good for redirecting to files, but there are also some built-in "devices" which act like files. NUL is one such "device", which is used to just discard any output. So running foo.exe >nul 2>&1 would discard any and all output from foo.exe. Since devices act like files, you'll notice you can't create files that are the same name as a device.

For those of you who are keeping score at home, other output devices off the top of my head include:

LPT1, LPT2, etc. (parallel ports, usually used for Line PrinTers)
COM1, COM2, etc., (serial ports)
AUX (auxilliary device - COM1 by default)
CON (console - the default device, i.e., your screen)
PRN (printer - same as LPT1, though something in the back of my mind tells me there used to be a way to reassign this).

Edit: Reformatted for clarity.

Opening web pages from the command line

myocom — Wed, 08 Jun 2005 02:00:00 GMT

I spend a lot of time in a CMD window. I mean a lot of time. Often I'd like to open a web page, but opening up Internet Explorer and navigating my way to the page in question is such a chore. I'm lazy. I don't like mouse-clicks. Enter the quicky batch file.

traffic.cmd - This checks my local traffic conditions (thanks to the excellent folks at the WSDOT)
@start http://www.wsdot.wa.gov/Traffic/seattle/I405_53rd.htm

weather.cmd - This tells me the current conditions in my area
@start http://www.weather.com/outlook/recreation/golf/local/98052

You get the idea...

Edit: Changed inadvertant italics

Backslashes vs. slashes - who knew?

myocom — Wed, 08 Jun 2005 01:45:00 GMT

Here's something I'd never noticed until today, when I discovered it quite by accident. In general, to run a program in a different directory, you specify the path with backslashes, such as C:\WINDOWS\system32\notepad.exe. This should be no surprise. What is surprising, though, is that if you use call to run this, you can use forward slashes: call C:/WINDOWS/system32/notepad.exe.

I'd love to know the reasoning behind this; it's not particularly useful unless you're a Linux guy who can't get out of the habit of using /. Even then, though, it's exceedingly unlikely that you'd precede everything with call.

Bugs and assumptions

myocom — Tue, 07 Jun 2005 08:00:00 GMT

There's a fine line between an assumption and a bug. If the assumptions are spelled out ahead of time, then it's not a bug if the script does what's expected, taking the assumptions into consideration.

Having disclaimed that, then, here are some of the assumptions made for the makefiles.cmd and delallbut.cmd scripts:

makefiles.cmd

1. This script assumes that the files that you'd like to create are either all in the same directory or that if you specify paths as arguments, that all directories you refer to already exist. It will not create directories on the fly.

For example, if you were to run makefiles.cmd foo bar baz you would end up with three files in the current directory: foo, bar, and baz. If, however, you were to run makefiles.cmd dir1\foo dir2\bar dir3\subdir3\baz, you'd better make sure that dir1, dir2, and dir3\subdir3 already exist or you'll get "The system cannot find the path specified" for each path that it can't get to.

2. For files you specify on the command line, if there are already files of that name, they'll get clobbered. This could easily be remedied by using >> rather than >. The single > will spew the result of a command into a file specified. The double >> will append to an existing file. This is important to remember when you're writing scripts that create logfiles, for example. I've been bitten by this before...

delallbut.cmd

Anders is absolutely correct on both points:

1. If you have a file that already has the read-only flag set, this script will not delete it.

Example: Create files foo, bar, baz, and gingerbreadman (makefiles.cmd is great for testing this, by the way). Now make, say, gingerbreadman read-only (attrib +r gingerbreadman). When you run delallbut.cmd foo, you'll see that gingerbreadman still exists in that directory (can't del me, I'm the Gingerbread Man!).

2. If the file you choose not to delete already had the read-only flag set, this script will strip it off once it's done running. If you depend on behavior #1 up there, you'll be unpleasantly surprised when you run this and find that your previously saved file suddenly disappears.

Example: Create files foo, bar, baz, and gingerbreadman again. Set the +r flag on gingerbreadman. Run delallbut.cmd gingerbreadman. Now gingerbreadman still exists, but it no longer has the +r flag set. Create file saveme. Run delallbut.cmd saveme. Uh-oh! Gingerbreadman is gone! Gumdrop buttons and all!

So, are these bugs? Not anymore - I've spelled out the assumptions...

So what the heck just happened there...?

myocom — Sat, 04 Jun 2005 02:30:00 GMT

In the previous post, there was a fair amount of code and basically no explanation... Here's how it breaks down:

Naïve method:

set FILENAME=basename-%RANDOM%

CMD has a built-in variable called %RANDOM%. Simply put, it returns a random value between 0 and 32767. You can test this yourself by running echo %RANDOM% a few times at a prompt.

This method, then, just creates an environment variable FILENAME as basename-2369 or basename-7, or whatever the random number was. The problem is that if you run this enough times, eventually you'll get duplicate numbers. You also can't determine when that will be, exactly. You could get duplicates after two runs or it might take 32769 runs. So what we need is a...

__
Better way:

    for /f "delims=:. tokens=1-4" %%m in ("%TIME: =0%") do (
        set FILENAME=basename-%%m%%n%%o%%p
    )

Ok, so this gets a little more complex. First, let's look at the output of echo %TIME% (another built-in variable). Right now, I get 16:21:52.73 as the current time. So the fields from left to right are hours, minutes, seconds, hundredths of a second. Easy enough.

Now, the for loop there is where the magic comes in. Generally the /f switch on for means that you want to operate on a file: say, reading each line of a file and doing something interesting with it. However, the help for for says this:

You can also use the FOR /F parsing logic on an immediate string, by making the filenameset between the parenthesis a quoted string, using single quote characters. It will be treated as a single line of input from a file and parsed. (emphasis mine)

So this is what we're using here. The value of %TIME% is the string we're working on. Note that the script also replaces spaces with zeros. You'll get leading spaces if you run this from 1:00am -9:59am, so to preserve the sorting, we want leading zeros.

The next bit of the for command is this: "delims=:. tokens=1-4"

This delims parameter indicates that we want to use : and . as our field delimiters (that is, split up the string everywhere there's a : or a .). The tokens parameter indicates which fields we're interested in. In this case, we're interested in all four: HH, MM, SS, and hh (for lack of a better abbreviation for hundredths). Then the %%m means we want to assign the first field to the variable %%m.

The nifty (if occasionally annoying) thing about having multiple fields is that CMD automagically puts the rest of the fields we're interested in into subsequent variables. So HH goes into %%m, MM goes into %%n, etc.

Now, once we parse all that, we set FILENAME to basename-HHMMSShh and we're done.

The problem with this is that if you run this loop a bunch of times around midnight, you'll get filenames like:

basename-23500000
basename-23590000
basename-00010000

...and those don't sort properly: The file created after midnight will appear to come before the ones in the 11:00 hour. Again, you could solve this by using dir /od, but that's cheating.

You might be asking at this point, "Why does this use for when it's not really a loop?" Excellent question. That's because *mumble mumble*. The real answer is "beats me". I suppose they (whoever "they" are) needed to stuff this functionality somewhere, and since for already had so many pathological cases, it was as good a place as any.

Ok, so on to the...

__
Best way:

    for /f "delims=/ tokens=1-3" %%a in ("%DATE:~4%") do (
        for /f "delims=:. tokens=1-4" %%m in ("%TIME: -0%") do (
            set FILENAME=basename-%%c-%%b-%%a-%%m%%n%%o%%p
        )
    )

This works much the same way as the "better" way, but it adds a couple of wrinkles. First, it parses the %DATE% variable (another built-in). For me, echo %DATE% returns Fri 06/03/2005 at the moment. But it doesn't just parse %DATE%, there's some funny punctuation in there.

The syntax for %DATE:~4% is hidden under set /?, oddly enough. What it means is "take the value of %DATE% and strip off the first four characters - the day of the week and the space. So, in my case, that means that %DATE:~4% results in 06/03/2005. The outer for loop uses / as its field delimiters, so we end up with the fields DD MM YYYY in %%a, %%b, and %%c.

The next for loop is exactly the same as the one in the "better" method. It stuffs HH MM SS hh into %%m, %%n, %%o, %%p.

Finally, we assign FILENAME the value of basename-%%c-%%b-%%a-%%m%%n%%o%%p, or for my example, basename-2005-06-03-16215273.

If you run this a bunch of times, you'll get filenames that sort correctly all of the time.
__

Edit: Added leading zeros to the times, where necessary, to preserve sorting.

Creating unique filenames in a batch file

myocom — Sat, 04 Jun 2005 02:15:00 GMT

Someone today was asking how to create unique filenames. They had some process which ran in a loop, and they wanted to direct the output of that process to a file, but rather than always overwriting the same file, they wanted to keep history using distinctive filenames.

There are a few ways to do this:

Naïve way:

set FILENAME=basename-%RANDOM%

Later, sort by using dir /od basename*

The problem is that these files have the possibility of being non-unique, since %RANDOM% only generates numbers between 0 and 32767. For a small number of files, you probably won't get a collision, though, and this is less code than the "better" way below...

__
Better way:

    for /f "delims=:. tokens=1-4" %%t in ("%TIME: =0%") do (
        set FILENAME=basename-%%t%%u%%v%%w
    )

This gives you precision down to .01 seconds, which may be overkill, depending on how tight your loop is. It does have one bug as far as sorting goes (unless you use dir /od again, but that's cheating…); if you run this over midnight, your sorting will be off (00:00:01 will sort before 23:59:59). If you don't plan on running it overnight, then this should work fine. But for a truly robust system, use...

__
Best way:

    for /f "delims=/ tokens=1-3" %%a in ("%DATE:~4%") do (
        for /f "delims=:. tokens=1-4" %%m in ("%TIME: =0%") do (
            set FILENAME=basename-%%c-%%b-%%a-%%m%%n%%o%%p
        )
    )

This gives you the same precision (.01 seconds) as the previous way, but sorts nicely since the date (YYYY-MM-DD) comes first in the filename.

__
Caveat to all of these methods: These scripts are not locale-agnostic. If you change your date/time formats, these may need to be tweaked.
__

Edit: Fixed the reference to %TIME% so that the script works in the morning too. Time in Windows includes a leading space for single-digit hours. While spaces aren't technically a problem - they're perfectly legal in filenames - they're sort of a pain...

Edit #2: Several people correctly pointed out that removing the leading space also messes up the sorting. The scripts now use a leading zero. Thanks, all!

Two quick file-manipulation scripts from the vault

myocom — Sat, 04 Jun 2005 02:15:00 GMT

There are a couple of scripts that have been with me since the beginning of time. I don't use them that much anymore, but they occasionally come in handy. I'm always pleasantly surprised to find that I still have them when I need them. One makes files and the other deletes files.

The first is called 'makefiles.cmd'. It just creates any files that you specify on the command line. This is handy for making some little scratch files you might need to test another script.

makefiles.cmd

@for %%x in (%*) do @echo %%x > %%x

This line loops over every argument passed in on the command line, say I run this as: makefiles foo.txt bar.txt donotdelete.txt. This line, then uses foo.txt as the first argument, bar.txt as the second, etc. It then echos the name of that file and redirects that into a file of that same name. That is, 'foo.txt' will just have the text 'foo.txt' in it.

The second is called 'delallbut.cmd'. This deletes all files in the current directory except for the one you specify on the command line.

delallbut.cmd

    @echo off
    if defined ECHO (echo %ECHO%)
    for %%x in (%*) do attrib +r %%x >nul 2>&1
    del /q . >nul 2>&1
    for %%x in (%*) do attrib -r %%x >nul 2>&1

This deserves a little bit of explanation. I often start scripts with those first two lines. If I want to see what a given script is doing (usually when I'm testing it or something unexpected is happening when I run it), I set an environment variable, ECHO, to on: set ECHO=on. So the first line here turns echo off (the default), then checks to see if I have ECHO set to on, and if so, turns it echo on. Note that I can also turn echo back off by setting ECHO to off. There's either a bug or an undocumented feature here, wherein if I set ECHO to, say, "Hi, how are you?", that text will be echoed at the beginning of this script. Sure, I could explicitly check to see whether ECHO is set to on or off, but that's more work than it's worth, in my opinion.

The next three lines are the meat of the script. The first line loops over all of the files you've passed in (like we did in makefiles.cmd). The action it takes for each of these files is to set the Read-only file attribute. The >nul 2>&1 bit basically says "throw away the output of attrib (if any)". I'll go into more detail about that little bit in a later posting.

So now every file we want to save is marked read-only. The next line is a shortcut for del *.*, or 'delete all files in this directory'. And again, we throw away any text that del might output. The following line turns the read-only flag back off, and we're done. All the files in this directory have been blown away except the ones we've specified. If you used the example for makefiles.cmd, then run delallbut foo.txt donotdelete.txt, you'll notice that bar.txt has disappeared, but foo.txt and donotdelete.txt are still intact.

Now, for bonus points, what problems can you find in these scripts? There are some assumptions made in each of these scripts. Leave your answers in the comments.

Typos vs. muscle memory

myocom — Sat, 04 Jun 2005 02:05:00 GMT

It occurs to me that my mistypings of 'notepad' and 'exit' are probably more the result of muscle-memory since I type those so often. The mistakes I make are due to one hand hitting its letter before the other (correct) hand hitting its.

Just a thought.