Developing the Microsoft Keyboard Layout Creator was a unique experience.

Almost all of the good ideas and concepts came from Cathy Wissink and Simon Earnshaw. At the time they represented the bulk of the NLS Program Management team (now they each have 5-6 people under them in GIFT -- we have grown quite a bit since then!). The three of us would have weekly meetings and talk about what kinds of tools would meet some of the customer requests that had been coming in. It was a little unconventional as projects go, since I did not have a spec to work from. But at these weekly meetings I was able to show them what I had and they were able to provide corrective measures. We started calling the meeting the Dog and Pony Show, shortened to Dog & Pony and eventually to D&P and these were very creative meetings with lots of good ideas in them.

I mention the D&P because it helps point to the continuous "beta" and "usability" testing that was happening throughout the development cycle of MSKLC. Although it was pretty unconventional, it proved to be very effective when things line up as they did.

One of the most worrying concepts that came up in some of these meetings was accessibility, and not just in the conventional sense of making sure there are shortcuts and accelerators. The entire user interface has to be accessible in an intuitive way. Even though the operation of keyboard authoring was not necessarily an intuitive one.

Look at a simple operation of assigning letters to keys:

You could use the mouse to select buttons on the "faux keyboard" that is the MSKLC user unterface, which opens the editing dialog.

Or you could press the key once to select it and a second time to open up an editing dialog.

Or you could use drag 'n drop to put text onto a key from Word or IE.

Or you could right-click on a key and get an array of editing choices.

Or you can use arrow keys to move around the keys to select the one you want.

Or you could load an existing keyboard layout and modify that ².

The idea was that nobody would use all of these methods, but everyone would use the ones with which they were most comfortable. And very accessible.

Now with that said, there were things that did not quite as well, such as support for high contrast or large fonts. Though both problems are somewhat mitigated by the ability to change the colors and to resize the keyboard, respectively, there are tweaks to the DPI settings that can even mess up the layout of the resizing, which is pretty doubleplusungood.

All three of these functionalities (high contrast, large fonts, DPI changes) are considered important in many cases to accessibility in the conventional sense of the word. And the mitigation is not all that great even when it would work -- it would be better for MSKLC to simply handle these issues without requiring extra tweaks. Its just a feeling I have that if the UI is only usable to a customer after they change it then I have not really done the best I can for them....

There is also another kind of "accessibility". If you look at MSKLC it gives you 50 keys whose assignments you can alter and 10 more that you cannot. But these days even the smallest non-laptop keyboards have 101 keys, so what about the other 41 keys? If they cannot be accessed, isn't that an accessibility problem?

Well, yes, sorta. But most of those keys are things like the numeric keypad and the function keys and things that we really don't want people accessing, so it is not really something that anyone here loses too much sleep about. :-)

Now it is interesting to consider that the ability to create custom keyboards means that if there are specific problems or difficulties a user has such as a mising finger or fingers that a keyboard can be fashioned which will help them by assigning the most common letters to the keys that are most accessible to the individual user. I was indeed contacted by someone from the Accessibility team just before we shipped MSKLC who we talked with about potential usages of the tool.

This is not the only example of such a collabotation. We often end up either working together or doing each others' work -- for example, the Windows Accessibility folks are the ones who own and had developed the OSK (On-Screen Keyboard), which is often used by people trying to use other language keyboards when they do not know the layout. And Wei Wu and I worked with devs on the Tablet team to develop a good international solution for the TabletPC's soft keyboard.

1 - The most visible UI I have ever created would have to be some of the new wizards I added to the huge collection of existing wizards in Access 97 and Access 2000. But MSKLC definitely picks up the silver medal.

2 - Although it is last on the list here, it is probably the most commonly used feature, and it highlighted the pattern of those weekly meetings. I would show the current state of the UI, Simon or Cathy would bring up a feature they felt was crucial, I would push back and explain why it was not feasible, they would grudgingly accept this, but the issues they brought up would keep bothering me until I thought of a way to make it feasible. Then I would show it to them the next week. I am pretty sure that was how the meeting became known as D&P in the first place....

This post sponsored by "" (U+e000, a.k.a. the first PRIVATE USE codepoint).
It ought to be a square box most of the time!

Back in May of 2004, someone asked the following question in the newsgroups:

I'm programming in ASP.NET. How could I convert my date to UK format ?

Thanks in advance.

To which someone else suggested the following code:

DateTime x=DateTime.Now;
string s=x.ToString(System.Globalization.CultureInfo.CreateSpecificCulture("uk").DateTimeFormat);

To his credit, this code was compiled and ran; in the end he was convinced he had the solution.

However, there are many problems with this code:

• He passed a region to CreateSpecificCulture,  which is expecting a possibly neutral culture (from which it would always return a non-neutral one)
• He passed UK for Great Britain/England/United Kingdom and expected back results that match properties in England, which the ISO standard and .NET uses GB
• He used the CreateSpecificCulture method at all in case where you want to get formatting for a specific location, since its behavior is to pick the first non-neutral culture.¹

So why did the code run, you may skeptically be thinking? Though probably not if you thought about the title of this post!

Well, it has to with the overlap of ISO 639 (the two letter language codes) and ISO 3166 (the two letter region codes). Or in his case, using the wrong abbreviation in the wrong method and having it find some random region that happens to use the abbreviation (in this case the Ukranian neutral locale was being created).

This is kinda unavoidable since

1. there are only so many two-letter combinations
2. language and country names are not always the same
3. there is at least an attempt to make the string choice for each somewhat meaningful
4. the two standards are handled by two different committees that do not work together on assignments

See the following table for a list of the overlaps. The bad overlaps where you will get wrong results are marked in red.

For the full lists of cultures and regions, see the CultureInfo and RegionInfo help topics.

This very non-bug has been reported internally by people wondering why creating a RegionInfo from a neutral (region-less) culture like "AR" was succeeding, as they were not thinking about how when you pass it to a RegionInfo you are not getting "Arabic" you are getting "Argentina".

Of course, this is just a symptom about a larger problem that affects international testing in general, which I will be talking about in a future post. Think of this post as describing the tip of the iceberg. :-)

1 - In this specific case the method would quite literally return the wrong results even if you passed the right language associated with the UK -- en, because the default culture returned from CreateSpecificCulture would be en-US, not en-GB.

This post sponsored by "o", "ο", and "о" (U+006f, U+03bf, and U+043e; a.k.a. LATIN SMALL LETTER O; GREEK SMALL LETTER OMICRON; CYRILLIC SMALL LETTER O).

At last, the longstanding problems with end user license agreement (EULA) for MSLU have been fixed. You can look at the text of the EULA here or you can get it by downloading MSLU here.

This new EULA meets the intent that our team has had for MSLU since it was first released back in July of 2001. I am not a lawyer, but in my unofficial, non-legally-binding opinion it certainly is consistent with the behavior of Microsoft during this whole period (since not even "case and desist" letter was sent out to anyone for "violating" the old EULA's points which were contentious).

The main problems boiled down to having the wrong boilerplate...

The old EULA had the standard language that Microsoft uses to cover products that can be downloaded by anyone who agrees with the EULA but whose components then cannot be separately redistributed without making the eventual end user agree to the EULA.

The new EULA has the standard language that Microsoft uses to cover products that can be downloaded by anyone who agrees with the EULA and then the specific components within the download can be redistributed more than once to an eventual end user as part of a product.

A good example of why the difference can be important would be if you are one of those lucky third parties whose code ships with Visual Studio. Say you wanted your component to run on Win9x so you wanted developers building setups to have MSLU redistributed with your runtime. In that case, MSLU would be with your component shipping with VS, it would be added to the setup packages of people using your component, and then their end users who install their application would get MSLU with your cool component.

The same rules apply to components that do not ship with Visual Studio, of course (and many large companies were worried about the old language because it seemed to contraindicate their intended usage of MSLU!).

The ability to do this sort of thing was always intended (and there was even a component that shipped with the first version of VS.NET which had this requirement!). It just took the EULA a while to catch up to the intent of the component. :-)

No Unicode code point was willing to sponsor this posting (Unicode characters, it would seem, are afraid of both lawyers and paralegals!)

Believe it or not, I have gotten this question with language as bad as (and often worse than!) the title line of this post.

The quick answer, as I am sure might have guessed, is no. :-)

For the longer answer, I'll go to the FAQ section of the white paper Cathy Wissink and I wrote for a presentation¹ entitled Unicode and Keyboards on Windows at IUC25 in Washington, D.C. in March of 2004:

Q - "I get the feeling Microsoft just makes up these keyboards by themselves.  Why don’t they represent my language the way I expect them to?"

A - New keyboards for a market always get tested in their respective market.  A great deal of research does go into the keyboards shipped with the system, with feedback from linguists, government officials, other internationalization experts, and local software providers. Often it is the case of Becker’s law applying (that is, for each expert, there is an equal and opposite expert), unfortunately. 

Another form of the question which was much more polite came from Serge Wautier on this blog:

Why are there that many keyboard layouts ?

Why a french belgian layout where only a couple of non-alphanumeric characters are layed out differently from the french layout ? My feeling - I am a french speaking belgian - is that the only goal reached is pure confusion. I have yet to understand why belgian people would prefer the \ as Ctrl + Alt + < rather than Ctrl + Alt + 8 (and vice versa).

The above answer covers the reasons -- because somebody somewhere noted the difference between them. And while this can be confusing, the sad truth is that for every customer who would rather see fewer layouts to avoid confusion, there are six customers who want new layouts added to match their expectations.

But with that said, Serge's question can get an even better answer!

When you add keyboard layouts through the Regional Options control panel applet, you explicitly choose a language and a keyboard layout. So any time you prefer another layout, you can pick your language with this alternate selection.

And of course any time you do not like any layout provided by Windows, you can make your own with MSKLC!

1 - This presentation, which appeared to be a boring talk about keyboards from the description actually turned out to be where we announced the release of MSKLC, thus continuing a bold tradition that Cathy and I started in our IUC18 talk Unicode on Downlevel Windows (when we did the surprise announcement of MSLU). In talking to people after the keyboard talk about why they stayed to see the last talk of the day given its somewhat dry description (the room was very busy), I had two different people tell me that they knew that we would be announcing something because when we cover stuff that appears like it will be boring, we always have something up our sleeves!

This post brought to you by "๒" (U+0e52, a.k.a. THAI DIGIT TWO)

One of the 14 people who read this blog was reading my recent post "How do sort keys work?" and he thought that he may have found a bug. He didn't, but it was an interesting point that he brought up. So I thought I'd mention it here.

(Note to all -- never be afraid to post a comment, neither bugs nor mistakes will ever embarass me!)

His question, and the answer, follow below.

Your post and the Platform SDK list a specific format for sort keys

[all Unicode sort weights] 0x01 [all Diacritic weights] 0x01 [all Case weights] 0x01 [all Special weights] 0x00

But using a single 0x01 byte to separate the different weights seems very dangerous. Couldn't somebody accidentally pick a string that happens to have a single byte in one of the weights that would mimic this section ending marker? This could cause two strings to be considered equal even if they are not, couldn't it?

It is a very good question, but the answer is (thankfully) that it is not possible to hit that situation. The rules for the byte values in the sort key data we provide are such that we never allow a byte with only 0x01 in it. This was the only way we could make sure to always have an unambiguous section terminating marker.

This is obviously a tough loss. When you only have 256 possible values in each of those bytes, it is expensive to lose one of them (well, actually you lose two since we cannot allow the 0 either!). But its obviously required or else we would hit the very potential bugs that occurred to you....

This post sponsored by "
" (U+0001, a.k.a. <control>, a.k.a. START OF HEADING)
Note that this control shouldn't really look like much of anything except maybe accidentally a graphical block character on some versions of Windows -- he just did not think he'd have another chance to be relevant!)

The title for this post actually comes from the SortKey help topic:

Each character in a string is given several categories of sort weights, including script, alphabetic, case, and diacritic weights. A sort key serves as the repository of these weights for a particular string. For example, a sort key might contain a string of alphabetic weights, followed by a string of case weights, and so on. SortKey is equivalent to the Windows API method LCMapString with the LCMAP_SORTKEY flag. However, unlike LCMapString, the sort keys for English characters precede the sort keys for Korean characters.

Someone asked me what the hell that text refers to!

Well, a decision was made back in the early days of Windows (that incidentally many have had cause to regret) to cause ideographs for Korean to be sorted in front of all of the other letters (including the Latin script letters of English). This code exists on all of the Windows NT-based platforms and on the Windows CE platforms, but when the time came to support sort keys on the .NET Framework a decision was made to explicitly not do this.

There is no real linguistic basis for either behavior, its arbitrary either way.

Though since the .NET Compact Framework uses the WinCE OS tables to do its work, it means that the WinCE results will differ from the .NET Framework everywhere else.

It is worth mentioning that text in the SortKey topic is a little confusing since it does not make clear that this only happens for the Korean LCID. And since the Windows behavior is not completely documented it does not cover the fact that neither Extension A nor Extension B ideographs are supported by it (though at present none of them are given intentional Korean-specific weight, a fact that will change in future versions).

In any case, thats why Korean has the Korean coming first. Though when you consider the fact that this affects over 20,000 ideographs, the image of someone asking if they could cut in line at the supermarket "because they only have 20,000 items" is a little scary. :-)

This post brought to you by "ᇴ" (U+11f4, a.k.a. HANGUL JONGSEONG KAPYEOUNPHIEUPH)

A few months ago, I was talking with a customer who simply could not understand the sorting results she was seeing (in this case in a table in MS Word 2003). She distilled it down to a small repro; basically she took a small list of words:

(at this point I knew both the language and what was causing her to have problems, and you may know too!)

What she noticed was that if she marked the left column as being French text (she tried several French choices, including France and Canada), the order was like this:

while if column was marked as being English then the results looked like this:

She could not understand any sorting rules that would explain the way that the words were sorting in the "French" table.

So I talked with her about Académie française and how they have a specific preference related to the way letters with diacritics are to be sorted (I also mentioned incidentally that I thought they had abolished the use of the circumflex in the early 1990's in many words but honestly I did not know if it applied to these two words that use the SMALL O WITH CIRCUMFLEX!).

The specific rule I was talking about here is that diacritics are evaluated from right to left rather than from left to right. Thus côte comes before coté, rather than after it as it does in languages like English that evaluate them from left to right. Because the word côte has no ACUTE on the "e" at the end of the word while coté does. In English and most other languages, the evaluation starts on the left and therefore the CIRCUMFLEX or lack thereof on the "o" is the controlling factor in ordering.

You can see it described in the French sort order exanple in Appendix D of the first edition of Developing International Software for Windows 95 and Windows NT.

This particular rule is interesting in that in all of the native French speakers with whom I have spoken, I never found anyone who could explain the rule to me. In their defense they were pretty much all aware that there were special rules used in dictionaries, but if you think about it there would seldom be a time that one could not find the word one wanted in a dictionary that used this rule. After living with the language for a lifetime, I am sure things like this are simply understood subconsciously when they occur. This phenomenon is common in almost all languages and they pretty much all have rules that native speakers understand even if the speakers cannot articulate the rules.

Another interesting factoid about language that can be seen here has to do with the fact that this use of "reverse diacritics" is seen in every French locale supported by Windows. It is fascinating to see the influence that the "mother country" of a language can sometimes have on changes that are made to other places where it is spoken.

When changes are made, whether by longtime organizations such as the Académie or by direct legislation, other countries will in many cases tend to pick up those changes. To me, the reasons behind such language reforms spreading this way are fascinating to contemplate. It is certainly not any kind of sovereignty or true languge "ownership" issue (and in future posts I may discuss specific cases in other languages where changes were at times intentionally not picked up!).

But I am at times amazed at the way that people will appear to see language as transcending the petty things. Its the kind of behavior that makes me interested in linguistic issues. :-)

This post brought to you by "Ô" and "É" (U+00d4, a.k.a. LATIN CAPITAL LETTER O WITH CIRCUMFLEX, and U+00c9, a.k.a. LATIN CAPITAL LETTER E WITH ACUTE)

I have been noticing more and more web sites lately that contain an interesting type of ad. Have you seen any of the following?

• Shoot the bug and get a free iPod!
• Which one is Snoop-Dogg? Answer for a chance at a free Sidekick!
• Shoot the tank and get a free PS2!
• Shoot the criminal and get a free Nintendo DS!
• Who is the pop diva? (hint from me -- it is not Angelina Jolie or Pamela Anderson)
• Shoot the gangster and get a free iPod!
• And so on, ad infinitum....

In each of these you are playing a game where you are given an "easy" task (provided you don't believe that Pamela Anderson is a pop diva, of course!). All you have to do is click in a specific small area of the screen making up a small part of an image that is not in the tab index, has no accelerator, has no other keyboard access, and is thus generally unreachable to people who have accessibility issues and are unable to use the mouse.

I must admit that there is a part of me that gets mad any time I see inaccessible UI. Its unfair, its thoughtless, and it discrimates. It sucks.

I am also forced to admit that there is another, larger part of me that is a little jealous of people who can be spared the stupidity of these ads. Its almost like someone is saying to them "I know you have it tough, so we won't force you to deal with the pop-up hell our company wants to give everyone else on the web dumb enough to click here."

All of that is probably crap -- if the lifeless wingnuts who design these Beelzebub's hors d'oeuvres and serve them up on web sites that are so desperate to have the Internet make some money for them finally that they will pimp for the video game system target practice contests were to recognize that there was a whole crowd of suckers who would give them money, then they'd be packaging up a system to deliver the con to them.

Luckily they do not seem to be all too bright.

A Sort key is basically an array of bytes. The intention of the sort key is to make for faster comparisons of strings, so that if you compare the sort key values for two strings you will get the same results as comparing the strings themselves. They abstract out all of the irrelevant data (for example if you use NORM_IGNORECASE or CompareOptions.IgnoreCase) then the binary sort key for "AAAA", "AaAa", and  "aaaa" will all be identical. As such, sort keys make a great basis for an index of string values, like you would have in a database engine.

But how are they structured to allow this to happen?

They have the same architecture in both managed code (via the SortKey class) and unmanaged code (via LCMapString with the LCMAP_SORTKEY flag). The structure is described in the LCMapString topic in the Platform SDK:

[all Unicode sort weights] 0x01 [all Diacritic weights] 0x01 [all Case weights] 0x01 [all Special weights] 0x00

Note that the sort key is null-terminated. This is true regardless of the value of cchSrc. Also note that, even if some of the sort weights are absent from the sort key, due to the presence of one or more ignore flags in dwMapFlags, the 0x01 separators and the 0x00 terminator are still present.

The reason for this structure is that the primary weights (called the Unicode weights, above) need to take priority over secondary weights (called Diacritic weights, above), which themselves have to take priority over the tertiary weights (called Case weights, above), and so forth. In this way, all of the following examples are true when using the invariant locale/culture, as described in the last post:

AAAA   <   AAAB      (primary difference)
AAAA   <   AÃAA      (secondary difference)
aaaa   <   AAAA      (tertiary difference)
AÃAA   <   AAAB      (primary difference, secondary difference ignored)
AAAA   <   aaaB      (primary difference, tertiary difference ignored)
aaaaab <   aaab      (primary difference, length and tertiary difference ignored)
ＡＡＡ  <   aaà      (secondary difference, special width and tertiary difference ignored)

And so forth. For that to work, the four different categories need to be kept separate and each one needs to be put in the sort key in its entirety, and if any type of weight is ignored then that whole section will be empty.

You can take the sort key, this structured array of bytes, and use it as an index for the string. Comparisons of two byte arrays will always be faster than comparing the string themselves.

This of course assumes that the sort keys are pre-calculated, like in an index. If they are not and you are looking at the difference between caculating then comparing the sort key values for two strings versus comparing the strings themselves, the string comparison will almost certainly be faster. The reason for that is that the sort key calculation involves analyzing the information of the entire string (and still does not include the actual comparison) whereas string comparisons will exit as soon as they can come up with an answer to the question of which one comes first.

I was doing a presentation a few years ago and it occurred to me that looking at direct string comparison versus sort key calculation/comparison was like looking at the "retail" version of collation vs. the "Wholesale" one. Only some people in the crowd felt it was an illuminating analogy, and I once again learned that I should not blurt out "good ideas thst suddenly occur to me" when I am in the middle of a presentation. :-)

One last thought -- no, there is not an Ordinal type of sort key. Because Ordinal comparisons are already done in a binary manner!

This post brought to you by "р" (U+0440, a.k.a. CYRILLIC SMALL LETTER ER)

There is a great deal of confusion surrounding the meaning of these two different things in the .NET Framework, and when to use each. If you have suffered, are suffering, or think may suffer in the future from such a confusion, then read on!

(Otherwise, I guess you can go away and come back another time)

The invariant culture's direct ancestor is the invariant locale. Officially added to the Windows source tree at 10:23am on May 12, 2001, its intention was not to be used as an actual locale (which would explain why no locale data was added until a month later; until then no one was using it in GetLocaleInfo!).

Originally, LOCALE_INVARIANT had just one noble purpose -- to allow one to use CompareString (and LCMapString with the LCMAP_SORTKEY flag) in a way that would only use the "Default" Windows sorting table as mentioned a little bit here and especially here. The results, as that second article mentioned, would not vary when the user or system locale settings did; they would be invariant within that installation of Windows.

The data was added for this locale a month later, as I said, for obvious reasons -- if you have an LCID that one function considers to be valid, you must have a very good reason if another will not. And it cannot duplicate any other locale, either. Much weird data was added so that no one would be tempted to try to act like they spoke a language called "Invariant" and then all was good.

Note that these string comparisons still had much linguistic value -- half of the locales in Windows use that default table, so an invariant sort would not only avoid varying, it would also look right to a lot of the world.

The .NET framework had similar requirements (with the additional need for invariant parsing/formatting support) and thus CultureInfo.InvariantCulture was created. As with the locale, any string comparions made with InvariantCulture's CompareInfo object would have linguistic validity in a lot of places, and would not vary within that installation of the .NET Framework.

So everyone had what they needed, right?

Well, no.

A bunch of people wanted a method of doing a more binary type of comparison, instead of one that would be based on the "linguistically appropriate" approach gven a particular culture¹.

The difference between what we had and what they wanted was akin to the difference between the C Runtime's strcoll/wcscoll versus strcmp/wcscmp (in the CRT documentation they refer to the difference as being locale based versus lexicographic).

The other advantage to such a "lexicographic" comparison is that it would be faster since a simple binary comparison of the code point values was being used.

To meet this need, the notion of an Ordinal sort was added and an Ordinal member was added to the CompareOptions enumeration. Selecting it would ignore all of those cultural collation features and give you a binary sort that would also, incidentally, not vary.

The only remaining problem at this point is that there were now two useful ways to do these different "niche" type of comparisons but neither name really jumps out at the developers who were looking for such solutions.

That problem remains to this day, though every single time I speak at a conference or answer a question in a newsgroup or get someone to look at posts like this one, then there is at least one less developer who has this problem. Maybe this time it is you? :-)

Now the story does not end here; many people have wanted to do things in a case-insensitive way. Of course if you wanted a case-insensitive invariant comparison then you could have done that all along -- just use the InvariantCulture's CompareInfo methods with the CompareOptions.IgnoreCase flag passed in. Easy!

But some people wanted a case-insensitive ordinal comparison?!?

Now the closet linguist in me shudders at this concept since a casing operation is essentially a linguistic one while an ordinal one is specifically not -- it's lexicographic.

So people are asking for a linguistic non-linguistic support, a request that for me brings to mind the comedian Steven Wright's dog².

However, the technical half of me understands the need and so I got over my linguistic fetish as one of my colleagues on the BCL team worked in Whidbey to add a new OrdinalIgnoreCase member to the CompareOptions enumeration.

The behavior is basically to do the casing operation using the default casing tables prior to doing the binary comparison. This feature has been in the "Whidbey" version of the .NET Framework for some time (first checked into the source code tree on February 7, 2003), so you can try it out today if you have just about any build of Whidbey underfoot.

Hopefully this post will help clear up some of the confusion about these two interesting comparison types.

1 - What can I say? Some people are Некультурные (uncultured) though not in the culturally offensive sense.
2 - Steven Wright claimed to have named his dog Stay so that he could call out "Come here, Stay! Come here, Stay!" and watch the dog walk toward him in a stuttery fashion.

This post brought to you by "Ω" (U+03a9, GREEK CAPITAL LETTER OMEGA)
I talked to Omega just before this post went live. She said that as the last letter in the Greek alphabet (who was pretty much always therefore last in the queue), she understood the cost of keeping letters in order. Any performance benefit is  good one, to her mind. Especially since a binary sort would let her come before her little sister (U+03c9, GREEK SMALL LETTER OMEGA) for once.

My New Year's Resolutions for 2005:

1. I resolve to stay unresolved;
2. I promise to not be too promising;
3. I swear to not stop swearing;
4. I will focus on feeling unfocused;
5. I agree to be disagreeable;
6. I give my word to have words with people;
7. I maintain that I am high maintenance;
8. I declare that I will make declarations;
9. I will opt to keep my options open;
10. I will have exact change for when madness takes its toll.

People who know me will affirm that I was doing this stuff already -- so it should be easy to keep doing it. :-)

This post brought to you by "F" (U+0046, LATIN CAPITAL LETTER F)
I will not attempt to guess why the letter F would want to sponsor here, but he seemed to feel it was fairly important.

I am going to take these two questions out of order because (a) locales existed before cultures did, (b) neutral locales set the stage for neutral cultures, and (c) I think it may help us look less lame. Though that third reason is probably just naive optimism on my part....

To see what Windows does with neutral locales, you can look at the documented behavior of ConvertDefaultLocale. Basically, if you pass a neutral like LANG_ENGLISH then it will return the equivalent of MAKELANGID(LANG_ENGLISH, SUBLANG_DEFAULT) thus 0x009 becomes 0x0409, 0x01a becomes 0x041a, and so forth. Easy, huh?

This ConvertDefaultLocale function calls an internal routine to do its work; the same routine is called by every NLS function, too. Which is a long way around to say that neutral locales do not exist to the Win32 NLS APIs.

Now there is one use for them in Win32 -- resource loading. You can use neutral LCIDs either to more accurately tag resources or to provide an easy fallback mechanism. Of course, if you want to put names on them then you cannot use GetLocaleInfo since asking for the LOCALE_SLANGUAGE of LANG_ENGLISH will give you "English (United States)" which is probably not what you wanted.

(In fact, I wonder what Visual Studio's resource editor does for its strings for these neutrals -- it must have its own strings somewhere, hard coded? Ick!)

In retrospect, it might have been a better idea to not do things that way, but it has shipped this way for at least the last ten versions of Windows. So we are kinda stuck with it.

Anyway, thats neutral locales -- at best, they are tolerated. But you can't really do much with them. Using that ConvertDefaultLocale-ish behavior can actually get you unexpected results sometimes, too. More on this another day.

This brings us to neutral cultures....

In the .NET Framework, a neutral CultureInfo mostly does not do this weird implied LCID fallback thing. There is actual data behind these culures that you can query and use -- and you can get back the actual names and everything. It also does a great job on the resource loading fallback -- using the CultureInfo object's Parent property. The parent property is not based on LCID tricks, either -- it's actual planned data for each culture. Obviously much cooler and a bit more thought out.There is even a CreateSpecificCulture method on a CultureInfo that does the same sort of thing as ConvertDefaultLocale, creating a specific culture from a neutral one.

A lot of you probably noticed where I said "mostly" in that last paragraph (an occupational hazard of having readers who can be as cynical as I am!). Unfortunately, that weird LCID-esque fallback behavior still basically happens for collation and encoding via the culture's CompareInfo and TextInfo objects. Which is not such a big deal, and it is really necessary since both of those objects need the context of a specific culture. 

In retrospect, it might have been a better idea to not do things that way -- CompareInfo and TextInfo should not have been made available (as happens for other objects like the associated DateTimeFormatInfo and NumberFormatInfo), but it has shipped this way for two versions so we are kinda stuck with it.

One important difference that distinguishes them from neutral locales is that one could create a class that is derived from CultureInfo that contains language-specific information which would make more sense in neutral cultures, which is really a fancy way of saying "language-only cultures" (which is itself kind of a fancy way of say something).

There are also some odd situations with the LCID property of the CultureInfo, but thats a separate issue. More on this another day, too.

So, thats neutral cultures. Mostly not useful, except for resources -- except in the same way that they are useful in Win32 (by pretending its a specific culture). Or for potential extensibility either by Microsoft or by developers in the future.

Three steps forward, one step back? :-)

This post brought to you by "♎" (U+264e, a.k.a. LIBRA)

The quote in the title is an allusion to a quote from the Simpsons. Now the ANY key quote is not my favorite quote from Homer J. Simpson (actually, this is), but its in the top five.

And it popped into my head as I prepared to talk about the AltGR key, which is a key that does not exist on the keyboard upon which I am typing right now.

Anyway....

In a comment from a couple of days ago, Norman Diamond suggested that AltGr stood for Alternate Graphics. Thats fine as far as it goes, but only the first two words of his post suggest what AltGR stands for, and the words do not really explain what it means, though he talks a bit about what it does. So the question remains, where does ALTGR come from as a term?

Well, it does indeed stand for Alternate Graphics (or Alternative Grafiken). The original intended purpose of it was to have an easy way to get at the table-based graphical characters that were so handy to use in a console application, located on the right side of the spacebar.

It was, however, used quite actively for keyboards that needed extra keys (and there is no layout I know of today in Windows that supports the graphical characters except by accident in consoles, and they do not use ALTGR to get there). This does not apply to the US english keyboard hardware, so they just put a RIGHT ALT key there which will actually act as if it were the ALTGR key any time you switch to a layout that makes use of this extra shift state. Note that this extra shift state is also available by hitting <Ctrl>+<Alt>, but thats more work to type. So having a single key to type instead is much cooler.

Of course, this can cause problems since sometimes people make shortcuts using <Ctrl>+<Alt>, which screws with what people might want to actually do with a keyboard layout. In fact, Raymond Chen talked about Why Ctrl+Alt shouldn't be used as a shortcut modifier last March, explaining this fact.

I would extend Raymond's very good advice to anybody who uses MSKLC to create custom keyboards (note that MSKLC warns about assigning <Ctrl>+<Shift> to keyboards since many shortcuts are assigned there). Or people who uses Word to create shortcuts. Or people on the Microsoft Word team who created tons of "useful" shortcuts that do not mind stomping on what a keyboard layout may have assigned to a keystroke combination¹. The key is to think about the keyboards and/or the shortcuts you create in the larger context of where you may either step on others or be stepped upon by them.

And if you create a custom keyboard with MSKLC, consider putting one of the graphical BOX DRAWING characters in the ALTGR state somewhere, so that you can be one of the cool people that makes the AltGR key meaningful again. Its like having an easter egg in software, but with an important recreational purpose!

1 - Every few months I start looking at the Word object model and its KeyBindings collection and related trivia to create a Word Add-in that will listen for keyboard changes and any time a WM_INPUTLANGCHANGE notification is received it would remove the Word shortcuts that conflicted with actual keyboard assignments. I find the undone project I was working on, get into it for a few hours, and then realize that this is something that the Word team or the Offce team ought to put together and build into the product. So I send off some mails and they agree with me and then it seems to go nowhere. A few months later it starts over again. Maybe one day one of us will have a finished solution for this problem. :-) 

This post brought to you by "╦" (U+2566, a.k.a. BOX DRAWINGS DOUBLE DOWN AND HORIZONTAL).
The competition in the BOX DRAWING block of Unicode to do a topical sponsorship of the post was fierce; it was finally chosen by the drawing of lots, in order to avoid violence.
In the future, an effort will be made to woo "appropriate sponsorship" from Unicode characters based on actual relevance to the specific post. Otherwise, its like a celebrity endorsement for a product that the famous person does not use -- and I hate that.

I have a box of candy in my office, and the two methods that seem most effective in keeping me between 161 and 165 pounds are to (a) try to fill the candy box with candy I do not like but other people do, and (b) not spend too much time at work and instead work from home. The second method is important since I also keep a large fridge filled with Limonata and if I am there too much I'll gain weight even without the candy and because other people like to fill the box, too. The first is important because I probably buy the most and I can dilute these other nice people.

Every once in a while I will see something new and I'll buy a bag of something that I have never seen before because I am curious what the hell it is. Most of them will be eaten before I get aroung to trying one and then I'll finally figure out if I liked it or not.

Have you ever had a particular kind of candy that you could not tell if you liked by trying it?

And have you ever found that trying it again later that day or the next day you still weren't sure?

Its probably just me.

Anyway, I haven't decided yet if I like blogs.

I mean, I have been doing this one for two months. I have been trying to post something relevant every day for at a little over a month of that. And at least four of those posts have been interesting. Its hard to sort out in the stats the people who link to you from the people who are just subscribing to hundreds of blogs and are thus not really reading any of this, but I think there are at least 20 people who read it now and again.

Yet, I can't tell if blogs annoy me yet.

I may not know until I can't think of something to post, and at that point I will be annoyed so it probably won't be fair to take that impression as an indication of how I feel about a whole technology.

Robert Scoble likes them a lot, but I have known him since he worked for Jim Fawcette and he always thinks some technology is the coolest thing in the world, and its always non-coinidentally what he is very involved with at the time. I'm not criticizing that, since if nothing else it proves he has convictions and believes in what he is doing/saying - which is a good thing. But its not a good way to get independent advice on what is cool. After all, he used to love the Offramp, too (and proably still does!), and I never really did, even when I used to post there....

Maybe the whole blog opinion thing will come to me one day when I don't suspect it.

Maybe blogs just taste like chicken. And I kind of like chicken. Though not every day....

On the other hand, maybe blogs taste like Limonata. In that case, you can expect about a case of posts every week!

This post brought to you by "✌" (U+270c, a.k.a. VICTORY HAND)