The problem of string comparisons, WORD sorts, and the minus that is treated like the hyphen

William Hooper pointed me to an interesting bug report:

Help Strings in .NET not always transisitve!!!!!!

We need someone to help us!!!!!!

>>>>>>>>>>>>>>>>>>>>>>

https://connect.microsoft.com/feedback/viewfeedback.aspx?FeedbackID=236900&wa=wsignin1.0&siteid=210

>>>>>>>>>>>>>>>>>>>>>>

   string s1 = "-0.67:-0.33:0.33";
   string s2 = "0.67:-0.33:0.33";
   string s3 = "-0.67:0.33:-0.33";
   Console.WriteLine(s1.CompareTo(s2));
   Console.WriteLine(s2.CompareTo(s3));
   Console.WriteLine(s1.CompareTo(s3));

Bug in the hypen logic when there are two hyphens in string...

Now I have talked about string sort vs. word sort in the past. And this is a problem which appears in both Windows and the .NET Framework, in all versions -- and has for all the versions I checked.

Even more interestingly, if you reverse all three comparisons the bug still happens (it is common in these kinds of issues when they come up for there to be a difference!).

Let's look at the sort keys to see how things look for the three strings:

0c 03 07 33 0c 7d 0c 90 07 37 0c 03 07 33 0c 46 0c 46 07 37 0c 03 07 33 0c 46 0c 46 01 01 01 01 80 07 06 82 80 1b 06 82 00
0c 03 07 33 0c 7d 0c 90 07 37 0c 03 07 33 0c 46 0c 46 07 37 0c 03 07 33 0c 46 0c 46 01 01 01 01 80 1b 06 82 00
0c 03 07 33 0c 7d 0c 90 07 37 0c 03 07 33 0c 46 0c 46 07 37 0c 03 07 33 0c 46 0c 46 01 01 01 01 80 07 06 82 80 2f 06 82 00 

 Now let's focus on that end piece that contains the hyphen info, since the main parts of the string are all identical:

80 07 06 82 80 1b 06 82 00
80 1b 06 82 00
80 07 06 82 80 2f 06 82 00 

 Now obviously from a sort key standpoint, s1 < s3 < s2, always. Which is obviously superior to s1 > s2 > s3 > s1 any day of the week, and which points to comparison that has the bug in it -- the comparison of s1 vs s2:

CompareInfo ci = CompareInfo.GetCompareInfo(0x409);
string st1 = "-0.67:-0.33:0.33";
string st2 = "0.67:-0.33:0.33";
string st3 = "-0.67:0.33:-0.33"; 
SortKey sk1 = ci.GetSortKey(st1);
SortKey sk2 = ci.GetSortKey(st2);
SortKey sk3 = ci.GetSortKey(st3);

Console.WriteLine(ci.Compare(st1, st2));
Console.WriteLine(ci.Compare(st2, st3));
Console.WriteLine(ci.Compare(st1, st3));
Console.WriteLine();
Console.WriteLine(SortKey.Compare(sk1, sk2));
Console.WriteLine(SortKey.Compare(sk2, sk3));
Console.WriteLine(SortKey.Compare(sk1, sk3));

The results?

1
1
-1

-1
1
-1 

Definitely a bug in the string comparison, one that has been around for a long time. Perhaps a better reason than ever to keep in mind the importance of linguistically meaningful strings when one is using the NLS/NLS+ collation support? Clearly the intent of the word sort is to handle an entirely different class of usage of U+002d -- basically it is meant to handle the hyphen, not the minus. A meaningful comparison of these strings would have to be based on the underlying source of the numeric data.

Do we have an NLS tester in the house? Let's make sure that Connect bug makes its way to me when it gets ported. And I suppose we need one for v.Next of Windows, too....

It might even be worth treating U+2212 (MINUS SIGN) as different than U+002d (HYPHEN-MINUS), though as a mitigation it won't help since most methods of converting a minus to a string come up with U+002d, not U+2212. Just as most hyphens do not end up as U+2010 (HYPHEN). Though of course the best fit mappings for both characters usully puts them as U+002d. Changing all of those string creating definitions would probably create even more problems with compatibility. :-(

This post brought to you by - (U+002d, a.k.a. HYPHEN-MINUS)