News Flash: Spaces are legal characters in both filenames and passwords!

re: News Flash: Spaces are legal characters in both filenames and passwords!

Peter Ritchie — Wed, 04 Jun 2008 03:00:40 GMT

I hear you man. I encounter this sort of thing (not with passwords, but filenames) all the time. I sympathize.

re: News Flash: Spaces are legal characters in both filenames and passwords!

Mark Sowul — Wed, 04 Jun 2008 05:47:01 GMT

And thus, "Program Files" and "Documents and Settings" were born - apocryphally, to force the others that didn't know that filenames can contain spaces. So when you see an app that wants to install in C:\Litware, guess why (often, anyway)?

Heh, we had an issue with our app on 64-bit Windows because Oracle didn't like the parentheses in Program Files (x86).

re: News Flash: Spaces are legal characters in both filenames and passwords!

Mark Sowul — Wed, 04 Jun 2008 05:52:07 GMT

(Eh, I don't know if you're as zealous about "don't name names" as Raymond, so bowdlerize accordingly. I personally don't like to reward vendors with shoddy or lazy things like this and prefer to point them out. I remember a post on "don't spam the quick launch/system tray/top of start menu with your crapware" and a comment was, "I wondeR who hE's tALking about"

http://blogs.msdn.com/oldnewthing/archive/2003/09/03/54760.aspx#54762)

re: News Flash: Spaces are legal characters in both filenames and passwords!

LarryOsterman — Wed, 04 Jun 2008 06:03:18 GMT

Mark, I try not to disparage other vendors, simply because it's stupid to point fingers - there's more than enough blame to go around.

Sometimes you need a forcing function :).

re: News Flash: Spaces are legal characters in both filenames and passwords!

Damit — Wed, 04 Jun 2008 09:56:01 GMT

Unfortunately, this isn't (to my knowledge) the only internal MS tool that fails at handling passwords with spaces in them - there are so many others that I had to give up on using spaces in my passphrases and instead smush everything up into one word. Makes for an interesting experience when you're trying to type your password out in a hurry. :)

re: News Flash: Spaces are legal characters in both filenames and passwords!

Mark Sowul — Wed, 04 Jun 2008 14:58:05 GMT

Of course. We all make mistakes, but then there's willful/negligent/lazy stuff (like Apple's new Safari "flaw"/"not flaw").

re: News Flash: Spaces are legal characters in both filenames and passwords!

Boris — Wed, 04 Jun 2008 14:59:48 GMT

OK, but don't you get tired of typing long passwords? How often do you unlock your workstation?

re: News Flash: Spaces are legal characters in both filenames and passwords!

Dave — Wed, 04 Jun 2008 15:48:27 GMT

Larry, it sounds to me like normal command line parsing. The -useuser option takes two arguments, username and password. In the unquoted version you have given it a password ("The") and several other arguments ("rain in spain..."). One or more spaces separate arguments on the command line. If you want embedded spaces in your argument, quote it.

To see the nightmare that results when software tries to figure out quotes that aren't there, look at CreateProcess:

http://msdn.microsoft.com/en-us/library/ms682425(VS.85).aspx

For example, consider the string "c:\program files\sub dir\program name". This string can be interpreted in a number of ways. The system tries to interpret the possibilities in the following order:

c:\program.exe files\sub dir\program name

c:\program files\sub.exe dir\program name

c:\program files\sub dir\program.exe name

c:\program files\sub dir\program name.exe

re: News Flash: Spaces are legal characters in both filenames and passwords!

LarryOsterman — Wed, 04 Jun 2008 16:50:11 GMT

Dave, the issue here is that the tool correctly handles spaces on the command line but the test harness <i>didn't</i>. The reason that the test harness didn't was because the maintainer of the test harness had no idea that spaces were legal in passwords.

That's why I figuratively threw my hands up in exasperation.

re: News Flash: Spaces are legal characters in both filenames and passwords!

Mihai — Wed, 04 Jun 2008 19:54:54 GMT

Adding quotes solves this problem.

But it would not solve the problem of Unicode characters in user name/passwords, which are also valid :-)

I know, the console has /U and some level of Unicode support, but it is tough to tell in this case (testtool would need to use wmain, for instance :-)

re: News Flash: Spaces are legal characters in both filenames and passwords!

Harry Johnston — Wed, 04 Jun 2008 23:22:08 GMT

On the other hand, quote marks are also legal in passwords. Whether or not you can cover that case as well without modifying testtool.exe depends on the details of how it processes the command-line arguments (and arguably on whether those details are documented).

re: News Flash: Spaces are legal characters in both filenames and passwords!

Dave — Wed, 04 Jun 2008 23:54:42 GMT

Sure, Harry, if the test harness did something like "%password%" (assuming it uses batch file syntax) to call testtool, it could be foiled by a quote. But it is kind of scary that the developer didn't know the args could have spaces. Just don't put him in charge of validating Internet data.

re: News Flash: Spaces are legal characters in both filenames and passwords!

Karellen — Thu, 05 Jun 2008 00:41:33 GMT

"spaces have been legal in filenames since MS-DOS 2.0"

Hmmm.....while technically that may be true for the underlying FAT filesystem, they were effectively useless as there was no way of escaping spaces on the DOS command line. So such filenames were not usable from DOS. They may have been usable from applications that ran on top of DOS and FAT, but they were not usable from DOS itself. I'm not sure when this changed but I suspect not until maybe DOS 5 or more likely DOS 6.

re: News Flash: Spaces are legal characters in both filenames and passwords!

Tony Toews - MS Access MVP — Thu, 05 Jun 2008 03:06:24 GMT

I didn't know spaces were valid in a Windows password until I happened to be watching a Microsoft video a year or three ago. Oh I knew all about file names having spaces but not Windows passwords.

I'm a database developer specializing in MS Access for the last ten or fourteen years. I've been using Windows at least that long.

Many, many users don't know that either. I've asked around. I'd suggest adding some text to the Windows login screen.

News Flash: Spaces are legal characters in both filenames and passwords!

Tonys Microsoft Access Blog — Thu, 05 Jun 2008 03:35:28 GMT

News Flash: Spaces are legal characters in both filenames and passwords! My comments to that posting

re: News Flash: Spaces are legal characters in both filenames and passwords!

John — Thu, 05 Jun 2008 03:59:08 GMT

This is why my password is 16 spaces - nobody would ever guess that!

re: News Flash: Spaces are legal characters in both filenames and passwords!

John Vert — Thu, 05 Jun 2008 07:47:35 GMT

Try installing Vista into "C:\Windows Vista (x64)" instead of C:\Windows then come back and tell us that spaces are legal in filenames.

re: News Flash: Spaces are legal characters in both filenames and passwords!

AndyC — Thu, 05 Jun 2008 12:31:21 GMT

Ooh, that's a new one. I've seen systems break horrendously with other characters (such as , ; or : ) that are used internally as separators, but I've never thought to try it with spaces. I wonder how many of our corporate apps would fall over at that one.

That could make for a fun afternoon....

re: News Flash: Spaces are legal characters in both filenames and passwords!

Jim T — Thu, 05 Jun 2008 13:01:04 GMT

Spaces are also legal in usernames. We can't use a very expensive monitoring tool because the licence manager can't recognise our usernames.

re: News Flash: Spaces are legal characters in both filenames and passwords!

Mark Sowul — Thu, 05 Jun 2008 14:01:50 GMT

The fact that you can create such a folder shows they are legal in filenames, obviously. That software misbehaves or ignores this fact doesn't make it untrue.

re: News Flash: Spaces are legal characters in both filenames and passwords!

Thu, 05 Jun 2008 16:00:33 GMT

Honestly, I'm just surprised that Larry is surprised.

First, I wouldn't be surprised that cmd.exe/runtime library would mangle the spacing anyway (if you had two spaces in a row in your password Larry, you should have been more surprised if it actually did work). Passing anything on the command line with spaces by quoting it is a very old pattern (and it's what you have to do with file paths that have spaces), I really don't know why this would be a surprise to such an experienced DOS/Windows user. I would have just done it automatically.

Secondly, I know you're trying to poke fun, but really I'm surprised you even expected the coder to think of these things. You know that passwords with spaces aren't an 80% scenario (it's at the very best a 5% scenario) which is generally first to the triage room floor. Admittedly it's a higher severity so it should be prioritized higher, but do you really expect it to be considered in the first pass of developing an application, especially an internal tool?

And finally, given that your attempt using quotes actually *worked*, I don't know why you responded in comments that testtool was broken, but maybe there's some higher level problem?

re: News Flash: Spaces are legal characters in both filenames and passwords!

LarryOsterman — Thu, 05 Jun 2008 16:38:58 GMT

J: I said that the test harness was broken (actually it was misconfigured), not that the test tool was broken. The test harness launches the test tool. And I've never tried two spaces together. I have to change my password in the next couple of days, I'll try that and see what happens :).

John Vert: You're nasty (in a good way) :). I've never thought to try that.

re: News Flash: Spaces are legal characters in both filenames and passwords!

budsbd — Thu, 05 Jun 2008 18:47:26 GMT

Larry, you'd better tell that to your colleagues over at Windows Live. Just yesterday I was trying to change my password, and it wouldn't let me put spaces (it failed and complained about illegal characters).

re: News Flash: Spaces are legal characters in both filenames and passwords!

John — Thu, 05 Jun 2008 18:49:41 GMT

Calling out a coworker on a public blog... classy move, dick.

re: News Flash: Spaces are legal characters in both filenames and passwords!

LarryOsterman — Thu, 05 Jun 2008 19:09:42 GMT

budsbd: Live may have their own password policy, I was referring to Windows.

re: News Flash: Spaces are legal characters in both filenames and passwords!

Jonathan — Thu, 05 Jun 2008 21:39:25 GMT

Sound like the test harness is open to "harness injection". How would it handle a password like "password &echo I am an evil injected script"?

I initially wanted to write "format c:" there, but a test harness that installs Windows probably does that anyways.

re: News Flash: Spaces are legal characters in both filenames and passwords!

Peter — Thu, 05 Jun 2008 22:14:13 GMT

Command-line parsing is an "interesting" problem. Just whacking the word into quotes, as people noted, will cause other problems.

It's even hit the funny pages: http://xkcd.com/327/

One of my favorite issues? Parsing does very odd things with backslashes -- RunMyProgram "c:\" "at once" will actually pass in a single parameter <<c:\ "at once>>

re: News Flash: Spaces are legal characters in both filenames and passwords!

LarryOsterman — Thu, 05 Jun 2008 22:51:11 GMT

Oooh, Little Bobby Tables. One of my all time favorite XKCD cartoons.

Jonathan, yup, the test harness is likely to be subject to SQL injection style attacks. It's an internal test tool however and as such it's not really a big deal - you can get the test harness to execute arbitrary test scripts (ie commands) so it's not like there's any additional risk associated with "harness injection" attacks.

re: News Flash: Spaces are legal characters in both filenames and passwords!

Eff Five — Fri, 06 Jun 2008 20:28:03 GMT

Non-printable characters are legal for filenames/users/passwords as well (e.g. ALT-255), and can sometimes defeat validation on fields that don't allow just empty spaces.

re: News Flash: Spaces are legal characters in both filenames and passwords!

Yuhong Bao — Sun, 15 Jun 2008 22:35:09 GMT

"my "password" is actually a passphrase - the theory is that passphrases are harder to crack than passwords because they are drawn from a larger dictionary."

Also it prevents generation of the LM hash. Care to blog about the story behind that?

re: News Flash: Spaces are legal characters in both filenames and passwords!

ndiamond — Mon, 23 Jun 2008 09:29:09 GMT

"On Microsoft operating systems, spaces have been legal in filenames since MS-DOS 2.0 (about 1982)"

But they couldn't be stored in FAT file systems until long file names were supported. They still can't be stored in short file names in FAT file systems.

http://www.microsoft.com/whdc/system/platform/firmware/fatgen.mspx

The document's file name is fatgen103.doc

Page 29, section "Short Directory Entries"

There are some bugs in that section. The total path length of a short name looks like a garbage statement because a path can be a combination of long named components and short named components, but we're really concerned with each individual short name. Short names containing the German sharp s character are not always converted to upper case. However, there is no inconsistency in the exclusion of space characters from the list, so this doesn't look like a bug.

There are some bugs in the subsequent section too, for example because paths containing long names can be a lot longer than 260 characters (on disk even though not in some APIs). However, again there is no inconsistency in the stated permission for embedded spaces in a long name. As well, the stated permission for embedded spaces in a long name makes it pretty clear that the absence of permission in a short name is not a bug.

re: News Flash: Spaces are legal characters in both filenames and passwords!

ndiamond — Sat, 28 Jun 2008 13:04:39 GMT

Mr. Osterman informed me by e-mail that the fatgen103.doc document is incorrect and might get fixed.

I hope someday to know the correct rules.

Now for a comment about passwords. Around 30 years ago I used a system where pressing just the backspace key would delete input of the previous character, but pressing shift + backspace would input a backspace character. When setting my password I intentionally input a backspace character, and it appeared to work. When logging in I did the same thing, and got logged in. But later I learned that backspaces were illegal in passwords on that system.

re: News Flash: Spaces are legal characters in both filenames and passwords!

Maurits — Thu, 10 Jul 2008 03:20:23 GMT

Meh. I'm of two minds on this issue.

On the one hand, the test tool is asking for the password. So you should be expected to just be able to give it the password. Any addition of surrounding quotes (which is easy) or escaping of internal quotes (which is hard) or nonprintable characters (which borders on impossible) is the responsibility of the test tool.

Fine.

On the other hand, looking at it another way, the test tool is a glorified command line builder. From this point of view, it is perfectly reasonable for the test tool to expect the user (Larry) to enter, not the password, but "the thing that goes after -password in the command line". From this POV, the responsibility for escaping quotes and whatnot falls on Larry.

This has the ring of transparency to it.

But this POV would horribly break if the test tool fed the password to two different things that parse command lines differently.