Search Form
Syndication
Administrivia

Technically it *is* a hungarian sort

Sorting it all Out
Michael Kaplan's random stuff of dubious value
Be sure to read the disclaimer here first!
MSDN Blogs > Sorting it all Out > Technically it *is* a hungarian sort

Technically it *is* a hungarian sort

26 Nov 2005 4:01 AM
  • Comments 11

Back in August in the post Double compressions -- Hungarian goulash? I described how double compressions worked in Windows and the .NET Framework.

And then a week ago in Hungarian is even more complicated than I thought I talked about an additional interesting wrinkle in this particular language's collation.

There were some interesting comments in that post, like this one:

I tell you a story. I had a strange error on MS SQL Server. select ... where [Product Identifier] = '%SG%' did no find the product with the identifier of "KCSG01"

A friend suggested that maybe it treats "cs" as one letter. I said impossible, even MS can't be so crazy. And he was right - after setting collation to binary it worked!

I it is completely amazing - who wanted this this feature? Who needs it? Why did it have to be developed and hardcoded into Windows/MS SQL? I agree that a grammatical analyser function library might sometimes useful to someone, but to hardcode it right into the OS!... Why?

When users search for "ddzs", they don't want to find "dzsdzs" - they are searching for LETTERS, you know, they don't want to keep all these grammatical rules in their heads. No one expects that their search input will be grammatically analysed!

So why has this feature been implemented?

To which I thought about the fact the Hungarian Technical Sort exists as an alternate sort for Hungarian (its LCID is 0x1040e). This sort has several characteristics that distinguish it from the standard Hungarian sort (0x040e):

  • None of the compressions that I have talked about previously
  • None of those Hungarian double compressions, either
  • The uppercase letters come before the lowercase ones, unlike most other language collations on Microsoft products

There is, in fact, nothing uniquely Hungarian about it and anyone who was wanting the uppercase/lowercase thing reversed might be happy with the ordering.

The perfect answer for those more technical situations when one does not want to be bogged down by those linguistic collation details, right? :-)

 

This post brought to you by "ʥ" (U+02a5, a.k.a. LATIN SMALL LETTER DZ DIGRAPH WITH CURL)

, ,
Comment on the blather
Leave a Comment
  • Please add 6 and 3 and type the answer here:
  • Post
Blog - Comment List
Comments
  • Nick Lamb
    26 Nov 2005 6:30 AM
    "There is, in fact, nothing uniquely Hungarian about it"

    Yet no-one's surprised people like it. Did Microsoft ever do surveys of real users (not linguists, not people who have a nationalist propaganda mission) to find out how many of them actually want the supposed oddities of "their" language systemised in the computer? It's been mentioned before that a lot of people can't describe the collation rules for their language, but how many really /prefer/ the official collation rules, given an option? How many can look at a dozen lists and pick out the one that's "in the right order" ?

    Unfortunately when something is done wrong in a million computers it may be a lot harder to fix than a mistake (or exageration) in a textbook or research paper. So we ought to be very careful before enshrining such rules in a computer program, even if implementing them is fun.

    IMNSHO when you find yourself forcing people to stay true to someone's ideal of their culture or language, you've made a huge mistake somewhere. This applies to Welsh culture & language laws in the UK (forcing children to learn an essentially dead language to satisfy the ambitions of some nationalist politicians) even more than the Académie française (at least the Académie doesn't have force of law).
  • 26 Nov 2005 5:21 PM
    Nick,

    When people are unhappy with behavior on Windows, we hear about it. And I am not talking about people who read this blog, I mean regular users who do not like the behavior.

    I know you think what Windows does here is incorrect, but I have millions of users who use Windows and find the results to be intuitive that are basically suggesting implicitly that you are in the minority.... :-)
  • Nick Lamb
    27 Nov 2005 4:53 AM
    "I know you think what Windows does here is incorrect"

    You've misconstrued my comment. I don't know whether this is incorrect, I was wondering (and you could have answered) whether it was ever put into a usability lab for comparison. I do know that several people in my circle were relieved when I showed them how to disable the "English" collation rules on their Unix systems, but my sample is biased, lots of technical and scientific types, not many warm fuzzy people.

    By the way, how can they do that in Windows? They'd like to, but I couldn't find any way to help them.
  • 27 Nov 2005 6:32 AM
    Which English collation rules in particular?
  • Nick Lamb
    27 Nov 2005 8:42 AM
    Particularly the case ordering, they want (THAT, THIS, the other) rather than (THAT, the other, THIS).
  • 27 Nov 2005 11:52 AM
    We do not really get a call for binary ordering at all, really. There is no mode in Windows that gives you that currently.

    Possible on UNIX? Well, okay. Perhaps that explains why UNIX isn't really a consumer OS? :-)
  • Szajd
    29 Nov 2005 11:34 AM
    Nick,

    You're thinking is bad, because you can't imagine a language that has, for example, letters consisting of two characters.

    For us, Hungarian's, such a thing is not an "oddity", a "strange thing", or even a thing to think about; it's the most natural thing in the world.

    I suppose, you're language is English, and that's bad, because it's hard for me to tell you an example, to imagine yourself in such a situation.
    But let's try it: imagine, that the English language includes the letter 'ph': if your operating system would handle that one as a letter P and H, THAT would be odd. Because it would brake something, that is actually natural to you.
  • Sorting It All Out
    2 Mar 2006 4:06 PM
    I was having a converation with Shawn earlier and had something of an epiphany that I thought I'd share.......
  • Sorting It All Out
    2 Sep 2006 1:35 PM
    Last night I was upon the stairA little hyphen that wasn't there.It wasn't there again today;Oh how I...
  • Sorting It All Out
    6 Dec 2007 10:32 AM

    Yesterday, I was blathering about how In SQL Server, the distance between A and Z is wider than you might

  • Sorting it all Out
    13 Jul 2010 7:26 AM

    It had been happening for quite literally years. People were complaining about it for years, I mean.

Page 1 of 1 (11 items)
MSDN Blogs > Sorting it all Out > Technically it *is* a hungarian sort