What is up with number sorting?

re: What is up with number sorting?

Tad — Wed, 05 Jan 2005 16:54:00 GMT

So what was there clever solution to sorting the numbers in StrCmpLogicalW? Just curious...

re: What is up with number sorting?

Michael Kaplan — Wed, 05 Jan 2005 17:01:00 GMT

Heh heh heh.... good question. Of course its not *my* clever solution so I don't want to make it sound like I am bragging (I'm not).

Let's just say that whoever wrote the code is a hire. :-)

re: What is up with number sorting?

Dean Harding — Wed, 05 Jan 2005 22:42:00 GMT

One possible way would be to go through each file name and pad any numbers with zero up to a fixed length and then sort with the regular CompareString function.

The only two problems with that would be 1. It's not very efficient to modify all the file names, especially for very long directory listings, and 2. It's difficult to know how much padding you need.

For 2. the obvious solution is to check the maximum length of a string of digits in all the file names and use that (so for example, if you have 3.txt and 1000.txt then you need to pad the first file with three zeros), but that just makes 1. more obvious (since you need to loop through all the file names once to find the maximum length of a string of digits, and again to actually pad digits).

For 1. I'm not sure how you could make it quicker, and still be able to use the normal CompareString to do the main work, because you'd /want/ to use the normal CompareString function otherwise you'd end up having to rewrite a very complex function for a fairly small gain.

re: What is up with number sorting?

Michael Kaplan — Wed, 05 Jan 2005 23:50:00 GMT

Re: clever -- Raymond Chen pointed out to me a slightly less cool version of this code that used to be around which was much less clever. To be clear I am talking about the code that shipped with Server 2003 and then later in XP service packs.... :-)

To Dean -- think of another issue -- what about the file A1A1A1A1A1A1A1A1A1A1A1A1A1A1.txt? Do you extend every one of those numbers?

re: What is up with number sorting?

Dean Harding — Thu, 06 Jan 2005 00:06:00 GMT

Ah, good point.

OK, another solution would be to sort it normally first. This'll get all the text-based ordering out of the way, and at least all the files with the same prefix and numbered suffixes will be together. So lets say you start off like this:

a.txt, b1.txt, def20.txt, b20.txt, def6.txt, b9.txt, c.txt

After you do your normal CompareString sort, you'd have this:

a.txt b1.txt, b20.txt, b9.txt, c.txt, def20.txt, def6.txt

Then you can group them so that files with the same prefix followed by a digit are in a group, this'd give two groups:

b1.txt, b20.txt, b9.txt and
def20.txt, def6.txt

Then it's just a matter of extracting the number from the correct position in each file name, and sorting by that number in-place. So your two groups become:

b1.txt, b9.txt, b20.txt and
def6.txt, def20.txt

And the whole thing is:

a.txt, b1.txt, b9.txt, b20.txt, c.txt, def6.txt, def20.txt

This way, you don't modify strings, you don't have to worry about padding with zero, and you'll still get the correct answer (I think :)

re: What is up with number sorting?

Norman Diamond — Thu, 06 Jan 2005 00:51:00 GMT

> Mom: How can a number be text?

Send her back for a refresher in grade 4 arithmetic, or somewhere around there. The difference between numbers and numerals was taught there, and numerals are text. Different numeral bases came up in grade 6, which should make it more obvious that numerals are text.

Nonetheless, sorting them as numbers instead of numerals is indeed agreeable 95% of the time. I only wonder, as you said you do, why Windows Explorer only does it around 50% of the time instead of 95% of the time.

Since there are some APIs now that recognize the numericity of various kinds of characters, one would think that Explorer would properly sort the department names of the company that I work in, but no I have to find the folder in order by the pronunciation of the number[*] instead of the numeric value. Dai-ichi-nani-nani, dai-san-nani-nani, dai-yon-nani-nani[*], dai-ni-nani-nani, 1 3 4 2 and the 4 isn't even in the expected place by pronunciation[*].

[* The matter of multiple pronunciations of a Kanji has been discussed in other threads. The character meaning four is getting sorted by pronunciation shi instead of yon, which is understandable but still not the first place one would look for it. I've never heard anyone say "dai-shi-nani-nani", it's "dai-yon-nani-nani".]

re: What is up with number sorting?

Michael Kaplan — Thu, 06 Jan 2005 02:24:00 GMT

> Send her back for a refresher in grade 4 arithmetic, or somewhere around there.

Well, I will leave aside the fact that I would not talk to my mother that way, and point out that it would really not behoove Microsoft well to treat customers that way. So I will avoid that approach. :-)

> Since there are some APIs now that recognize the numericity of various kinds of characters

Well, the function in Windows today only handles 0123456789 -- the "ASCII digit" collection. It might be cool to do more, but you start running in to problems when you try to define what "more" should be since it is not always intuitive.

re: What is up with number sorting?

Marcel — Thu, 06 Jan 2005 05:47:00 GMT

OK, I like hacks, so what about this:

Go through the strings and contract all consecutive digits into a single character, for example in the Unicode Private Use Area. After that sort the strings in a normal way (provided the sort routine sorts PUAs according to their value). Prepending the private character with a fixed '0' could also ensure that it is sorted in the place of ASCII digits.

X1A.TXT -> X0'U+E001'A.TXT
X13.TXT -> X0'U+E00D'.TXT
XAB.TXT -> XAB.TXT

re: What is up with number sorting?

Norman Diamond — Thu, 06 Jan 2005 06:57:00 GMT

>> Since there are some APIs now that
>> recognize the numericity of various kinds
>> of characters
>
> Well, the function in Windows today only
> handles 0123456789 -- the "ASCII digit"
> collection.

Where did I read that Windows APIs were added that recognized Thai digits and various other code page digits. I'm sure I asked somewhere and received an answer that Windows Explorer is only designed to recognize the numericity of ASCII digits, but the function in Windows today is comparatively internationalized.

re: What is up with number sorting?

Raymond Chen — Thu, 06 Jan 2005 08:24:00 GMT

Norman: Are you arguing that Explorer should sort files as follows:

two.txt
five.txt
sex.txt
elf.txt
three hundred.txt

?

re: What is up with number sorting?

Michael Kaplan — Thu, 06 Jan 2005 09:13:00 GMT

Norman, re: Thai digits you are mistaken.

Speaking personally, I never claimed this API was interationalized, and I pointed out that doing so leads one to problems of non-intuitive behavior.

There are many functions in Windows that are well-internationalized. This not one of them, certainly when looking at other language digits.

re: What is up with number sorting?

Michael Kaplan — Thu, 06 Jan 2005 09:18:00 GMT

Marcel -- interestng thought. Though one of the problems with that plan is that it puts digits in a wierd place compared to other characters (since the PUA has weight that puts it somewhere very unlike where numbers go).

The fact that these code points already have weight that needs to be primary between them will also lead to problems

And of course it still isn't an unbounded answer....

FWIW, I am not going to be hiring anyone based on their anwsers. But I am impressed at the quality of the answers. :-)

re: What is up with number sorting?

Mike Williams — Thu, 06 Jan 2005 11:28:00 GMT

If you run a search on filenames via Explorer, the results always seem to come back in strict order (ie not using StrCmpLogicalW) and also in lower-case for some reason. Can you shed any light on this? It's a little frustrating trying to filter a folder using a filename substring and having the results presenetd differently to a regular Explorer listing.

re: What is up with number sorting?

Marcel — Thu, 06 Jan 2005 13:09:00 GMT

> Though one of the problems with
> that plan is that it puts digits
> in a wierd place compared to
> other characters

Therefore the prepended '0', which makes all those private values primarily sort into the same spot (of U+0030) and only secondarily use the PUA value.

But true, it's not unbound, though the range should be big enough to sort a few photos. Anyway, it was good enough for me considering I came up with it after a long night's work at 7am in my time zone ;-)

re: What is up with number sorting?

Michael Kaplan — Thu, 06 Jan 2005 15:50:00 GMT

Hi Mike -- The Shell filtering functionality and the way help indexing works are both topics I will have to cover another day. I find them frustrating, too....

Marcel -- I understood that part, I was more interested in how you would handle the string that was engineered to look the same (i.e. sends those exact PUA values). I am not slsmming the idea though, I was just pointing out the problems -- so you wouldn't feel like it had not been thought of and discarded regretfully. :-)

re: What is up with number sorting?

Norman Diamond — Fri, 07 Jan 2005 00:39:00 GMT

1/6/2005 12:24 AM Raymond Chen

> Norman: Are you arguing that Explorer should
> sort files as follows:
> two.txt
> five.txt
[...]

Good point. But I really thought I had read somewhere that Windows APIs had been augmented to recognize digits written in various linguistic character sets. And we both know that digits as written in Chinese characters can (and often are) strung together as digit sequences.

Full formal wording is a more difficult question since, even though each character is numeric, how do we say if the full formal construction is a numeral or a word phrase?

1/6/2005 1:13 AM Michael Kaplan

> Norman, re: Thai digits you are mistaken.

OK, I can't find where I read that Windows APIs had been augmented in that manner. On the other hand though, I saw a Thai use Thai digits and saw Windows accept them. And that was Windows XP Japanese (not Thai, not US with MUI, not anything else) with the input locale set to Thai.

re: What is up with number sorting?

Michael Kaplan — Fri, 07 Jan 2005 00:59:00 GMT

Well digits are accepted. But there is no Windows API that readds them as numbers.

Either someone was mistaken when they spoke or you read them wrong. This applies to every version of Windows, no matter what the input locale is.

Perhaps you are talking about unrelated issues with GDI and rendering based on native digits in a locale, but if so that has nothing to do with collation.

The Daily Grind 533

TrackBack — Sun, 09 Jan 2005 23:36:00 GMT

The Daily Grind 533

numeric file rename

TrackBack — Sun, 09 Jan 2005 23:37:00 GMT

numeric file rename

re: What is up with number sorting?

Larry Osterman — Mon, 10 Jan 2005 22:41:00 GMT

Btw, Valorie's been on me for literally years now about the fact that OE doesn't sort email messages with text in them.

She hates it that OE sorts:
This is message one of three
This is message two of three
This is message three of three

as:
This is message one of three
This is message three of three
This is message two of three

re: What is up with number sorting?

Michael Kaplan — Mon, 10 Jan 2005 22:47:00 GMT

I just mentioned the notion of addressing this to the owner of the collation data and her look could have killed. :-)

Though I am inclined to agree with her -- its a dangerous road to go down for an API like CompareString. Or even for higher level callers like OE (luckily most of them are sorted by date anyway).

re: What is up with number sorting?

Peter Ibbotson — Fri, 14 Jan 2005 11:38:00 GMT

Thanks for that, I'd still like an option to turn it off though. One of the problems I have is that I have loads of file names with that kind of structure, and depending on what program I'm using to look at files, the file I want may be at either end of a list.
It's a real pain as you swap between source control and explorer.
Anyway thanks

re: What is up with number sorting?

Michael Kaplan — Fri, 14 Jan 2005 15:00:00 GMT

Hi Peter,

It actually *can* be changed via a Group Policy setting. Precisely because not everyone will neccesarily like it....

You can even query this policy's settings with the SHRestricted API and the REST_NOSTRCMPLOGICAL member of the RESTRICTIONS enumeration.

Its about File Sorting

Vinod Kumar's Blog — Thu, 27 Jan 2005 10:46:00 GMT

re: What is up with number sorting?

Mike Williams — Thu, 27 Jan 2005 10:55:00 GMT

The other sorting annoyance is this sort of thing:

The Sea II.mp3
The Sea.mp3

This is also seen with Windows Media Player's Album list.

re: What is up with number sorting?

Michael Kaplan — Thu, 27 Jan 2005 10:59:00 GMT

Yes, though (to me) it actually does make sense since space comes before dot....

Though I would not mind if Explorer removed the extension from consideration for the sort, except to break ties. It seems like the results would be more intuitive more often.

Windows Explorer vs Geeks (or Intuitive filename sorting)

Rolando Ramirez's WebLog — Fri, 01 Apr 2005 22:39:00 GMT

What is up with number sorting, redux

Sorting It All Out — Sat, 02 Apr 2005 02:56:00 GMT

Ramblings by PeterI — Mon, 04 Apr 2005 11:52:10 GMT

Back in January I asked Michael Kaplan about the explorers number sorting order here.Meanwhile over here Rolando Ramirez has pointed out that this in TweakUI. Oddly when I mentioned this in the office someone else piped up and told me how to tweak it via

Similar descriptions does not mean similar methodologies

Sorting It All Out — Sun, 08 May 2005 20:47:39 GMT

The other day, I had to take a look at the various unmanaged case insensitive string comparison functions....

Why do NTFS and Explorer disagree on filename sorting?

The Old New Thing — Fri, 17 Jun 2005 18:17:12 GMT

Because they have different goals.

Did software developers ever learn their ABC's?

Sorting It All Out — Tue, 15 Aug 2006 14:43:58 GMT

Developers always think they are right.
Especially when they are not.
I mean, if you asked any random...

StrCmp for humans? It's in there!

Sorting It All Out — Sat, 30 Sep 2006 20:59:51 GMT

Igor Levicki asked in the Suggestion Box:

Any chance to enlighten as whether Windows Explorer in...

Logical StrCmpLogicalW changes in Vista

Sorting It All Out — Sun, 01 Oct 2006 10:04:36 GMT

I mentioned yesterday in StrCmp for humans? It's in there! that some change happened in the behavior...

What would it mean to internationalize StrCmpLogicalW?

Sorting It All Out — Tue, 03 Oct 2006 19:40:54 GMT

If you are a regular reader of this blog then odds are you might be a little sick of hearing about StrCmpLogicalW

Sorting numbers as text AND as digits

Sorting It All Out — Tue, 12 Dec 2006 11:33:15 GMT

It's funny, but the past few years whenever I have had to fly I have noticed (whether sitting in my scooter

Fractions may be your friends, but they won't pick you up at the airport!

Sorting It All Out — Sat, 14 Jul 2007 22:03:10 GMT

To me, fractions will always have a special place. The teacher pointed out we all knew what ½ was, and

Docs can whet SiaO's appetite, but where's the blog?

Sorting It All Out — Sun, 23 Sep 2007 13:31:34 GMT

As I mentioned back in How do I feel about lstrcmpi? I think it blows.... , the Mac CFString stuff has

re: What is up with number sorting?

James — Fri, 12 Oct 2007 22:57:51 GMT

If the shell team is so "amazing", they should add an option to disable number sorting. Its very annoying when file names contain numbers and some of the numbers have leading zeros. I expect this sort order:

000.txt

003.txt

020.txt

100.txt

but now I get this:

000.txt

100.txt

020.txt

003.txt

Sometimes the reason we added leading zeros to the filename in the first place was to make the files sort correctly, now its broken again :-(

File names are text and should be able to be sorted as such.

Gee Thanks for this "new feature"!

re: What is up with number sorting?

Michael S. Kaplan — Fri, 12 Oct 2007 23:17:38 GMT

James,

They do have such an option.

But FWIW I do not see the results you list here -- prefix zeroes still work fine for me.

All things being equal, your mom probably has an easier time with case insensitivity than not

Sorting It All Out — Fri, 26 Oct 2007 18:14:28 GMT

So, response to the recent In Case you have problems that you might think are ǸȦȘȚȲ , Jeff (I suspect

re: What is up with number sorting?

Maurits — Fri, 26 Oct 2007 23:36:29 GMT

> sex.txt

> elf.txt

(Um, ew.)

From personal experience, this number sorting thing is REALLY ANNOYING when you're looking at a huge list of subdirectories, all of which are GUIDs.

I agree with those that want an option to turn it off.

I might argue that the most important part of /any/ feature is the ability to turn it off.

Bury the off switch somewhere Mom can't find it, if it makes you feel better. But implement the off switch before implementing the feature.

re: What is up with number sorting?

Maurits — Fri, 26 Oct 2007 23:38:13 GMT

(Now that I actually read the comments)

> It actually *can* be changed via a Group Policy setting

Ah, perfect.

re: What is up with number sorting?

Maurits — Fri, 26 Oct 2007 23:41:42 GMT

Your links 404 though. Here's what I found by searching MSDN:

SHRestricted: http://msdn2.microsoft.com/en-us/library/bb762245.aspx

RESTRICTIONS: http://msdn2.microsoft.com/en-us/library/bb762534.aspx

re: What is up with number sorting?

Maurits — Sat, 27 Oct 2007 00:19:59 GMT

Oy. I've dug through gpedit.msc on Vista RTM and I can't find anything that looks like it might be this option. Can I buy a vowel?

Incomplete Scenarios: They don't know everything that's up with number sorting

Sorting It All Out — Sat, 22 Dec 2007 18:16:40 GMT

In just a few weeks from now, it will have been three years since I first wrote about What is up with

Insanity defined: In the real world -0 == 0, in Vista -0 < 0, and in Windows Server 2008 -0 ≮ 0

Sorting It All Out — Tue, 19 Feb 2008 18:01:18 GMT

Somewhere in between zero and the smallest possible negative number there lies another number. NEGATIVE

Consistency in the Windows Shell is not overrated; it's just underobserved!

Sorting it all Out — Tue, 04 Mar 2008 18:17:32 GMT

Please read the disclaimer ; content not approved by Microsoft! You may have been around in January 2005,

How the @#%&*! does CBS_SORT choose to sort it all out?

Sorting it all Out — Wed, 29 Oct 2008 17:06:25 GMT

The question was an interesting one: My customer has a tree control with nodes sorted using CString::Compare().

Natural sort order of strings and files

Greg Beech's Tech Blog — Wed, 12 Nov 2008 21:54:49 GMT

In Windows XP and Vista the algorithm used to sort files by name in Explorer does not simply compare

"If you haven't had sex is six months..." the IM read

Sorting it all Out — Fri, 05 Dec 2008 18:11:07 GMT

The mail I got the other day from Brett was pretty amusing (reposted uncensored with his permission,

CString - Sortierung | hilpers

CString - Sortierung | hilpers — Tue, 20 Jan 2009 17:45:50 GMT

PingBack from http://www.hilpers.com/956749-cstring-sortierung