Dan Crevier's Blog

In search of a better name...

The difference in wildcard expansion between Windows and unix/Macintosh

This one took a little getting used to when switching from the Macintosh to Windows. On unix (including Mac OS X shells) wildcard expansion is done by the shell, and then the expanded list of files is passed on to the program being run. On Windows, the shell doesn't do any expansion, and it's up to programs to do expansion.

The advantage of the unix approach is that programs don't have to do any work to support expansion, and the experience is consistent across all programs. As an example, notepad on Windows doesn't seem to support wildcard expansion. So, if you are in a directory with a file called foo.txt and type notepad *.txt, it will launch notepad and tell you "The filename, directory name, or volume label syntax is incorrect." On the Mac, textedit *.txt would open the file right up.

The disadvantage of the unix approach is that it doesn't let you do something like "rename *.txt *.text". On Windows, this command would rename all files ending with .txt to end with .text. If you were to try to write a similar utility for unix, you'd find that when you tried to run it, you'd just get a list of the .txt files as the input arguments, and you wouldn't get any information about the "*.text". That's because the "*.txt" expands to all the .txt files and "*.text" expands to nothing.

It's one of the many differences I found between Mac and Windows where one isn't really better than the other, they are just different and it takes some getting used to. I'm sure there are some of you out there that are religious about this issue and think that one way or the other is clearly superior.

Published Saturday, May 14, 2005 11:11 PM by dancre

Comments

 

jmstall said:

That's an interesting observation. How does Unix-like OSes handle the rename case?
May 14, 2005 11:50 PM
 

zzz said:

I would guess that in monad you can override default stuff like possible * expansion (if it even has such).. So for the rename program code you'd make it a bit different maybe..

Though I am sure monad has better ways thought for this..
May 15, 2005 12:08 AM
 

ronab49 said:

In Unix, you need a for loop. using csh:
csh -f
foreach file (*.txt)
mv $file $file:r.text
end
exit

This is impossible to achieve for a novice user, while the Windows equivalnet is short and easy to understand.
May 15, 2005 12:17 AM
 

Yaytay said:

IMHO the Unix version's advantage considerably outweighs the Windows version's advantage.
The Unix way of working is consistent.

How many brain dead Windows utilities don't support wildcards at all, simply because their author didn't think to include them?

My preferred route to solving the rename problem on Unix is to use 'find', which can be used effectively like a 'for each'.
The key to getting anywhere on Unix is to learn a few of the command line tools well (I choose grep, find, sed, awk) and then use them if at all possible - if they can't be bent to your will, then learn another one.
On the rare occasions that I can't do something in one command line on Unix I get really miffed!
May 15, 2005 1:02 AM
 

Mike said:

I don't understand why the shell should expand the wildcards. The application should receive the same arguments entered by the user. And the expansion could be done by a function from a library at the application's will.
May 15, 2005 1:04 AM
 

ele said:

Another 'icky' thing in unix is the fact that you can pass commandline parameters using files. eg, in a directory with the file '-rf', running rm * would suddenly delete subdirectories as well.
May 15, 2005 3:32 AM
 

SDiZ said:

In linux (and some unix), you may do that with "rename" comamnd:

rename 's/\.txt$/\.text/' *.txt

not as hard as you think. =)
May 15, 2005 7:57 AM
 

Peter da Silva said:

In UNIX this is handled by not having that particular rename syntax. It's rarely the case that you need to do bulk-renaming on UNIX because the file type on UNIX has always been just part of the name, it wasn't ever really a separate "type" the way it was in CP/M, MS/DOS, and the DEC operating systems that the Digital Research / Microsoft command line syntax was derived from.

People have written utilities that handle rename in various ways, but they've never caught on. usually they did something like use another wildcard:

rename %.txt %.text

The one I wrote some 25 years ago looked like

ren *.txt .txt .text

This was actually more powerful than the Windows rename, in some ways, but ... I don't think I ever had the occasion to use it, and it never survived. Bulk file renaming just isn't that common a problem in real life.

The REAL advantage to the UNIX semantics is that if you have a list of file names and you pass them to an application, you know that the application won't twiddle the file names. This is particularly important when you're writing scripts and you know the filename expansion has already been done by the program that called it, and even if a file LOOKS like it needs to be expanded, it really doesn't.

The disadvantage is that people need to learn that the wildcard always means the same thing, no matter where in the command it shows up. So you really do need to quote non-shell wildcards you're passing to commands like 'grep'.
May 15, 2005 8:57 PM
 

Lorenzo Barbieri @ UGIblogs! said:

May 16, 2005 5:08 AM
 

Klok said:

Another problem is the difference in capital letters.

On windows these two files are the same:
test.txt
Test.txt

In the Unix world, its two different files.
May 16, 2005 7:59 AM
 

Dan McCormack said:

In unix, you can also use the 'rename' program, if it is installed. It replaces the first occurence of one string with another in the filename of each of the files you pass it on the command line:

rename .txt .text *.txt

Another limitation of the unix approach is that if you are working in a directory with many thousands of files, you can exceed the limit for number of command line arguments. I discovered this when I had this mildly amusing exchange while trying to delete a bunch of emails that had accumulated:

$ ls -al | wc -l
272554
$ rm
rm: too few arguments
Try `rm --help' for more information.
$ rm *
-bash: /bin/rm: Argument list too long

Of course, there is always a workaround. For example:

find . -name '*' -maxdepth 1 -print0 | xargs -0 rm

But obviously, that wouldn't be intuitive to a beginner.
May 17, 2005 11:11 AM
 

Jerome Lacoste said:

On Linux, using the rename command, do

rename 's/txt$/text/' *.txt
or
rename 's/\.txt$/\.text/' *

A little bit more verbose than the Windows command. Of course, if really missing one could write a tool that matches more closely your Windows CLI. You could end up with something like:

rename2 "*.txt" "*.text"
May 18, 2005 9:08 AM
 

Jon Daley said:

On unix, I just use `rename`.
In addition to being able to do things like the example given, you can also do more creative things like (from the man page)

> For example, to rename all files matching
> "*.bak" to strip the extension, you might
> say
> rename 's/\.bak$//' *.bak

> To translate uppercase names to lower,
> you'd use
> rename 'y/A-Z/a-z/' *

Quite handy, and much more useful than windows.

By the way, saw you got another obvious patent for e-mail addresses. My tax dollar at work.
May 18, 2005 9:18 AM
 

Anonymous Coward said:

Of course, the Windows shell puts the burden of implementing wildcards on the developer...and you never really know as a user what each command will do with wildcards.

Not being religious, just sayin' is all. :)
May 18, 2005 9:19 AM
 

Ray said:

"This is impossible to achieve for a novice user, while the Windows equivalnet is short and easy to understand."

I haven't seen a novice use a command line in over 10 years... :) Personally, when I need to do this I go to the AppleScript menu and use one of the default "Finder" scripts that lets you do such things.
May 18, 2005 9:20 AM
 

tarrgz said:

http://unixhelp.ed.ac.uk/CGI/man-cgi?rename

RENAME(1) Linux Programmer's Manual RENAME(1)


NAME


rename - Rename files


SYNOPSIS


rename from to file...


DESCRIPTION


rename will rename the specified files by replacing the first occur-
rence of from in their name by to.

For example, given the files foo1, ..., foo9, foo10, ..., foo278, the
commands

rename foo foo0 foo?
rename foo foo0 foo??

will turn them into foo001, ..., foo009, foo010, ..., foo278.

And
rename .htm .html *.htm

will fix the extension of your html files.


SEE ALSO


mv(1)

1 January 2000 RENAME(1)
May 18, 2005 9:25 AM
 

Jordan T. Cox said:

In response to jminstall; heh - I'm quite certain that, in general, most novice users don't even know of the sheer existence of the command prompt, let alone the commands therein. I've encountered MSCEs who don't even know what it is. Sometimes I wonder what will happen if the command prompt really does go away.
May 18, 2005 9:30 AM
 

lovebyte said:

In Linux:
% rename .txt .text *.txt

Maybe you should try to patent this too?
May 18, 2005 9:43 AM
 

Andrew Shuttlewood said:

Interestingly enough, unix works properly if there are NO .text files - for example

$ ls *.fred
*.fred: No such file or directory

Of course, if there are some, the hypothetical rename won't work. You could always quote the * as appropriate however, which would be okay, although not quite as obvious as the user would think.
May 18, 2005 10:01 AM
 

Anonymous said:

Given a 'rename' program was written that did the same internal expansion that DOS' rename does, you could always quote the wildcard - i.e. "rename '*.txt' '*.text'". To rename using mv, you need a loop of some sort; depending on what shell you use, replacing the "file extension" can be trivial or complicated.

Note that the first approach is already of use on unices - e.g. when running unzip: "unzip foo.zip *.txt". Unless there are no .txt files in the current directory, that '*' needs to be quoted in order for unzip to get a crack at expanding it.
May 18, 2005 10:15 AM
 

Dan Moore said:

Actually, I believe I can make the case that the UNIXish (and therefore Mac) way of doing it has a clear advantage over the MSWindows way.

Just because the shell *can* do wildcard processing into lists of filenames doesn't mean it *has* to. There is nothing preventing patterns including whatever wildcards you want from being passed into any program. You can tell the shells not to process/expand a "*" by simply escaping it like "edit \*.txt". Or you can tell the shell not to do any processing on a particular argument at all by quoting it like "edit '*.txt'". Either of these two command lines will result in the "edit" program being called with the argument "*.txt".

Because of this, if you want to write a program that just takes explicit filenames as parameters you can do that, and leave the heavy lifting to the shell. If the program needs to take a pattern, like to duplicate the MSWindows "rename *.txt *.text" functionality above, you can do that too and simply call it with "rename '*.txt' '*.text'" instead. With the MSWindows shell way, you don't have that flexibility.
May 18, 2005 11:20 AM
 

Anon said:

Actually, in Linux we get the best of both worlds. The command "rename" can be used to do exactly that. It would look something like this:
<br /><br />
rename .txt .text *.txt<br />
<br />
This would use the nice expansion features of *nix to get all of the ".txt" files via the *.txt command and then run a regex against them (something Win still lacks as far as I am aware of) to convert all instances of ".txt" in their names to ".text".<br />All of this transparent to the user...no need to do any shell scripting.<br />
I have used all three systems, Mac the least, and both Win and *nix as desktops and as servers. In my opinion the slogan for *nix should be "Any thing you can do I can do better."
May 18, 2005 12:09 PM
 

Jerome Lacoste said:

On Linux, using the rename command, do

rename 's/txt$/text/' *.txt
or
rename 's/\.txt$/\.text/' *

A little bit more verbose than the Windows command. Of course, if really missing one could write a tool that matches more closely your Windows CLI. You could end up with something like:

rename2 "*.txt" "*.text"
May 18, 2005 12:14 PM
 

&amp;#241;ieck said:

On Linux (at least on a RH 8) I've found a command called "rename". The meaning of the parameters slightly differs from the Windows one (as explained in the man page). The equivalent for your example will be:

rename .txt .text *.txt

However I haven't found it on a SunOS, so seems that in some systems you still have to deal with mv and scripts.

In other news, some comands (i.e. find) avoid wildcard shell expansion by escaping it: \*. Maybe that's the solution if you want an exact unix clone of Windows rename.
May 18, 2005 12:25 PM
 

dirk l&amp;#252;sebrink (dirk@sebrink.de) said:

dos rename *.foo *.txt has absolutly nothing to do with wildcards. it is just a program which takes two arguments and (arbitrarily) chooses to interpret them in the described way. rename *.txt *.bar could have been easily defined as rename @.foo @.bar. and such a program simply does not exist on unix. (and there is no need, because it can be scripted yourself :-). so not a question of better or not, but comparing different things (apples and peaches).
May 18, 2005 1:15 PM
 

random reader said:

This ugliness been in W$ since DOS 1.0.
does W$ guarantee atomicity?
what if i have file called '*.txt'?
why can't I "ren *\*.txt *\*.text" ?
May 18, 2005 2:27 PM
 

Morgan Schweers said:

Greetings,
Retrying, as the MSDN blogs was dying on comments before.

Actually, you're not quite right about the '*.txt *.text' problem. The best way to explore shell expansion is to use 'echo'. For example, in a directory with a few .txt files and no .text files, try:

echo *.txt *.text

and you'll see the list of .txt files, followed by '*.text'. See, shell expansion is 'smart'. If there is no file expansion result, it reverts to the original text provided.

So you could write a tool which, if given a wildcard that matched nothing at the end, figured out that '*.text' was the destination name, and did all the right things. The problem with that would come when you have:

1.txt 2.txt 3.txt 4.text

and wanted to renamed *.txt to *.text. That's where it breaks down, and so instead you have the shell programming power (which Windows simply doesn't touch):

for i in *.txt; do mv $i `basename $i .txt`.text; done

See, that's pretty powerful, but it takes a while to learn it, and it's too complex for most users. This is far more valuable in places where you want to manipulate the filename in more complex ways (conditional behavior, for instance) than just changing the extension, but DOS (and the Windows shell, by extension) makes the common task easy at the expense of making the complex task impossible. Once upon a time that was a criticism of MacOS as well, but now at least the complex is possible.

Anyhow, good luck with the transition, I went the other way, to OSX as the best UI atop Unix out there, and am quite happy.

Oh, and nill illigitimi carborundum.

-- Morgan Schweers
May 19, 2005 11:48 PM
New Comments to this post are disabled

© 2009 Microsoft Corporation. All rights reserved. Terms of Use  |  Trademarks  |  Privacy Statement
Microsoft
Page view tracker