June, 2008

So regular readers might remember the recent blog entitled "It makes total sense when you explain that the Turks have four I's".

Dale then commented:

Oh, the transcript of the podcast is better:

"It makes total sense when you explain that the Turks have four eyes"

What, they wear glasses???

Indeed. Here is the piece in question:

Ah well. As my ex-fiancé used to say, pobody's nerfect....

The corrected piece of the transcript would be:

Scott Hanselman: Right, exactly and a very good example of that would be say that you had an application that was going to check for the word 'fail' in a TextBox and the user typed in 'fail' f-a-i-l, all lowercase with an i with a dot on it and in your code you said TextBox.ToUpper(), you would get, in Turkish, FA-capital I with a dot on it-L, and then you would check if that equaled 'fail' and it wouldn't, then your application would fail.

Michael Kaplan: Right and…

Scott Hanselman: Makes total sense when you explain that the Turks have four I's.

Michael Kaplan: Yes.

Scott Hanselman: But it was very not intuitive.

Michael Kaplan: But all the wonderful things in language about vowel harmony and the way that the words flow together is lost on everybody. What people remember is that darn Turkish bug.

Doing a transcript of any kind of a technical chat is a real challenge, truly.

We should all go shopping!

This one was a true LOL (laugh out loud) one, for me -- so I really said HA! :-)

This post brought to you by ሀ (U+1200, ETHIOPIC SYLLABLE HA)

There was a mail thread that happened recently on one of those "if you aren't a fulltime employee then why the hell are you here?" kind of aliases, one that I belong to because by following along the problems (and occasionally looking at the remotes) I became much better at debugging.

Plus sometimes I even have unique information about a problem (since my areas of expertise occasionally come up!). A few times I even ended up with International Fundamentals consulting work, helping people out with issues that I noticed. It isn't really ambulance chasing, but it is being somewhat near the ambulances in case the patients happen to need my help. :-)

Anyway, they were talking about one of those interesting kind of bugs that pop up from time to time where somebody was delay loading a library during process exit and of course this was leading to other interesting problems.

That horror story Raymond covered in Quick overview of how processes exit on Windows XP? That came up at one point, kind of a reminder about all the things that happened on thread tear-down that you just didn't want to bother with on process tear-down. Because they take too much time and may deadlock (plus in the case of this bug, actually did).

And then someone asked me if I had this problem with the MSLU (Microsoft Layer for Unicode on Win9x Systems). He knew that we did some unload work....

Actually, we don't do much here.

What we almost did, that is the interesting part....

We added a function to UNICOWS.LIB that you can find if you spelunk through the symbols (which some folks have) named:

___FreeAllLibrariesInMsluLoader

And a sister function exported from the UNICOWS.DLL called:

__FreeAllLibrariesInMsluDll

These functions take no parameters and need none, either -- the former calls FreeLibrary on everything it called LoadLibrary on in the code inside the .LIB that is compiled into the application, and the latter does the same with all of the libraries loaded by the DLL, and clears all the function pointers.

Technically, these actions are useful in the case of MSLU being completely unloaded, which is why they exist. There was even some discussion at the time about putting it in the DLL_PROCESS_DETACH code in MSLU's DllMain, which at the time just had TLS cleanup code and such.

But in the end, the decision was to leave the call out, since most people did not need function pointers private to MSLU to be so religiously unloaded -- given that once MSLU is unloaded no one could be calling them anyway.

In fact, the main benefit of them is for when the thing that loaded MSLU may have leaked out function pointers to MSLU functions -- which means someone else may try to call the DLL. If it still happens to be in memory but its TLS information is gone, trying to call into it can cause it to use random memory information as if it was valid handles and function pointers. And in such cases it is much better for the code to crash immediately (which it will when all the pointers have been changed to NULL).

A rare scenario, obviously. But one that is important every once in a great while.

And people have found these functions in the past; there are even components and applications that use them due to their wholesome, cleanupy sound.

So I figure they at least deserve a mention, now.... :-)

This blog brought to you by ᠃ (U+1803, a.k.a. MONGOLIAN FULL STOP)

One funny thing about facebook is that there are pages for celebrities.

You know, famous people.

The pages provide info about appearances and such and if one follows and if one is a fan of the famous person, then this is very useful information....

Now in facebook if you want to "follow" the person the exact action is called becoming a fan of the person.

This is a bit of an oversimplification, since there are many reasons one may wish to follow a person in this way.

But it is no less reasonable of an oversimplification than the act of "friending" someone being the only way to get someone on your list of people who you have that connected kind of relationship with that in real life may be either significantly more than or significantly less or even just significantly different than what any normal person would call "friends".

This is what makes videos like Facebook in Real Life so funny -- because this oversimplified world is pretty funny when looked at through the real world filter!

Now of course famous people, like everyone else, are allowed to be on facebook as people.

You know, since famous people are people too and all.

And a famous person may well want to be able to follow that page about themselves -- for example to make sure that events and information are properly reported there, and in a timely manner.

So how does the famous person indicate in the world of facebook the desire to follow the events.

You see where this is going, right?

Here, I'll give an example, one that people here will probably know quite well:

If I were famous (which I am not), then one day the big feed might say to all of those who have friend'ed me:

Michael Kaplan has become a fan of Michael Kaplan

..which would be just plain silly, for me -- it is probably one of the reasons I am not famous!

I won't even discuss the calculus of meaning inherent in this one; suffice to say that there are enough times that I am not a fan of what I do that I might not choose to become a fan. I also won't talk about what happens with those who "friend'ed" me would decide they aren't fans? :-)

A rather amusing aspect of the oversimplified world of facebook, in my opinion....

This blog brought to you by ᐯ (U+142f, aka CANADIAN SYLLABICS PE)

Colemak devotees -- please control your enthusiasm; comments that get overly preachy will never be posted (as this is my pulpit, not yours!)....

The keyboard without a Caps Lock key that has come up now and again here in posts like this one and this one and this other one, also known as Colemak.

Anyway, it got its article deleted from Wikipedia again (the ASfDs for the article - here and here make for fascinating reading).

Yes I am the Microsoft blog "source" Vquex is talking about:

The only other sources cited are the article's own previous deletion debate here on Wikipedia and a blog entry by a Microsoft developer stating that they would not be including Colemak in Windows precisely because it is non-notable.

Well, not exactly. But that is how the Wikipedia text put it too, and it is probably close enough.

The Colemak forum discussion about all this is 24 posts long spanning about 10 months, and there is an interesting conversation there about me and how I am so polarized on the issue that I would somehow block the keyboard even if there was a valid business case for it. In fact that is the only reason I am writing this blog, to set the record straight. :-)

As a side note, the million dollar keyboard challenge that actually raised under 200 euro was also interesting. I probably would have not named the contest in such a way had it been me....

For the record:

1) I am not one true source of all the keyboardy goodness at Microsoft. I have a voice, and sometimes people want to hear it when I use it. But if there was a keyboard council (strictly speaking, there isn't), in my current role on the Windows International Fundamentals team I am hardly even an official member of it, let alone the only one with a valid set of opinions.

2) My bias about specific people in a community who choose to annoy me is something I keep separate from when I am asked about a specific business decision. That separation is important to me and is really the only way I can act in a way that I can feel comfortable is ethically appropriate. So while the way no Colemak fan was able to comment without extolling the virtues of the keyboard is something that grew old so fast that I started trimming comments and will probably moderate comments in this post, that is an entirely separate issue from whether it makes sense for Windows to include a specific keyboard in the box.

3) The more emotional/zealous in the Colemak community have polarized me about them, to be sure. But I could not list their names now if my life depended on it and wouldn't be looking them up to know them now unless my life did depend on it. So the next group have a clean slate, and will the only reason I'll probably have the same reaction is that the same rhetoric will probably be used and my patience, while almost infinite in some cases, is astoundingly scant in this case.

4) People who are reading here are unlikely to ask the question "Michael Kaplan who?" but outside of a somewhat narrow group of people in a couple of professions, I am sure it would be a common question. I would probably word it more strongly than they did, with a kind of a Reservoir Dogs-esque "Who the fuck is Michael Kaplan" if you know what I mean. Even inside of Microsoft, even if the topic is something like MSLU where I really am "the guy" and not looping me in can sometimes be mildly insane and a tremendous waste of everyone's time, it will sometimes be months before someone knows to loop me in. So when you take something like keyboards, especially when MSKLC is not directly involved, my involvement is not completely assured every time.

5) Getting off me for a moment.... you can certainly hear Microsoft folks, from the most highly paid executives to the greenest of just started program managers, bandy words like innovation around. But innovation is not a place where the piece of Microsoft involved with software layouts distinguishes itself as an innovator -- because this is a type of decision that is designed to always be following what customers are using. If there was a huge customer demand for a brand new keyboard and that demand had objective and provasble numbers behind it, then the hardware folks would be on board and everyone would have the keyboard they wanted. But these items are not created out of subjective interest, any more than they are killed for spite.

And there it is -- if the Colemak folks convince the people, and then convince the hardware folks to create this keyboard by having the valid business case, then it will happen and no changes to the software layouts are needed.

Note that this is the kind of innovative keyboard design that laptops of all kinds tend to do, for entirely different reasons.

It is highly unlikely that Microsoft would ever in a million years just go off and add software layouts that would go against what is on the hardware keys. Microsoft isn't a college dorm with members trying to sneak in past the RA after curfew. The software layouts will follow the reality of what is expected on the hardware layouts -- and there is no Windows OEM who even has this on the radar as a reasonable thing to do.

Colemak has not yet met that burden. Not by a long shot.

Which is interestingly different but not entirely unrelated to why they can't get the article to stay on Wikipedia (though of course their burden is much lower than Microsoft's)....

This blog brought to you by ဝ (U+101d, aka MYANMAR LETTER WA)

I've been blogging about Sinhalese keyboard on and off for some time.

Like in November of 2005 when in Custom keyboard, custom language? I bemoaned the lack of ability to extend the language list in MSKLC 1.3, which blocked Madhava Temmakoon's efforts to properly label the keyboard.

Or in August of 2006 when in Creation of transliterating input methods I agreed with Thakara's conclusion that any regular keyboard such as those created by MSKLC is ultimately insufficient for languages like Sinhalese, at least in terms of usability. And this includes the input support in Vista and Server 2008, by the way.

There are experts in the government in Sri Lanka who agree with all of this -- the distance between

• (barely) internationally tolerable, and
• internationally sufficient, and
• internationally delightful

is rather vast, from a language/linguistic standpoint.

The fact that the latest versions of Windows are only at the first stage is a little embarrassing. :-(

But taking up this cause, I have two sets of data that can be plugged into the framework from the Table Driven Text Service discussed in that so far 11-part but still continuing series of blogs:

• 0 (You have to start somewhere)
• 1 (Starting with a dictionary simple in every way)
• 2 (Three ways to convert!)
• 3 (Where do keystrokes come in?)
• 4 (Specifying when to modify, if not how)
• 5 (All about the language, baby!)
• 6 (it is sufficient to be iconic, aka DaYi, DaYi Aynu!)
• 7 (Configuration 'junk in the trunk', part 1)
• 8 (Configuration 'junk in the trunk', part 2)
• 9 (Will you be content if I tell you how some content can be defined?)
• 10 ("Even Jove nods," Atlas shrugged, "so we'll convert their old files, now.")
• 11 (The knights who say நீ, redux, #1)

Now between the data that Thakara sent me, and another set from another source, and the Tamil one that I still need to post (which is also relevant to Sri Lanka), I have the data almost together for three layouts covering two languages used in Sri Lanka.

Starting tomorrow, I am going to jump back in to the Table Driven Text Service once again, will these three new ones to add to the list of samples and examples I have mentioned in prior blogs:

• We weren't Vista heroes, but I think we were kinda heroic
• Return of the Unicode IME
• Behind the return of the Unicode IME
• Creation of transliterating input methods
• The Cantonese IME (not for input of characters from Canton, Ohio)
• And we are the knights who say நீ (NII)

I am pretty excited about this. How about you? :-)

This blog brought to you by ඣ (U+0da3, aka SINHALA LETTER MAHAAPRAANA JAYANNA)

So I was given a 4 GB Zune, and I do think that is very cool (how I got this is really beyond the scope of this blog).

But I discovered something really interesting.

If you look at all of the music I have, between commercial CDs and bootlegs and online purchases and everything else, I have somewhere just barely south of 500 GB of music.

And although I have no problem coming up with reasonable subsets for the older 30 GB Zune or even the newer 80 GB Zune, there is a real problem with the newest one.

I can't reasonably express an appropriate subset that will fit into 4 GB.

Now probably over half of the music that is there is not really something I'd consider for this situation, but we're still probably talking about over 225 GB of music which I can't subset into 4 GB without re-syncing several times a week (and even then the music I end up wanting isn't there and I have to suffer through the fact that there is really no way to predict with 100% accuracy what one will want to listen to later.

So if I realize that the gift is probably the wrong size for me, does it make me ungrateful if I feel like it doesn't fit?

Here I am trying to figure out if there are ways to automate the sync process so I can just build a different device each day. Really hoping to make it work, but realizing it isn't the most productive use of my time.

An kind of relevant additional example is:

A shirt I was given at TechEd -- the only one left was a small. and I don't fit in smalls.

I was grateful for the folks who really did their best to get me the shirt, but really I'll probably end up giving it away to someone who it will fit.

I am baffled, really -- if in the end they both are not ultimately useful to me, why if neither item cost me anything am I so much more reluctant to give away the Zune?

This blog brought to you by  ? (U+003f, aka QUESTION MARK)

I've always had a soft spot for Nancy Davolio.

One of the fictional employees from the Northwind Traders sample database that shipped with Access for so long, she was just synmbolic of something that I can't fully explain that I liked about Micrsoft Access back during the Access 95 beta (all the others from the database, like Laura Callahan and all the others -- had their fans. But I was a Davolian, all the way, which made the first time I saw her in the halls of Building 1 really awkward!).

The low resolution picture from the database hardly does her justice:

Somewhere around here I have an actual posed picture of me and the woman from the access User Education team whose face was used for Nancy, who I met when I was working on the "Publoish to the Web" wizard for Access 95, during the 97 development cycle. I couldn't find it but the above one is good enough for our present purposes. :-)

I wonder -- there are few people left on the Access team who will even recognize who this is; some of the youngest folks I know from the team weren't even in high school back then! Could anyone there identify her? :-)

I suppose I really am an old-timer!

Anyway, I was thinking about Nancy Davolio the other day, you see. Right after the topic of DAV came up.

The mail sent to my Hotmail account about a month ago:

Dear Microsoft Outlook Express customer,

Thank you for using Microsoft® Outlook® Express. Our information indicates that you use Outlook Express to access a Windows Live™ Hotmail® e-mail account via a protocol called DAV (Distributed Authoring and Versioning protocol). DAV, like POP3 or IMAP, is the way that a mail client communicates with a web-based mail server.

As a valued customer, we want to provide advanced notice that as of June 30, 2008, Microsoft is disabling the DAV protocol and you will no longer be able to access your Hotmail Inbox via Outlook Express. As an alternative, we recommend that you download Windows Live Mail, a free desktop e-mail client that has the familiarity of Outlook Express and much more. This next generation of free e-mail software will allow you to easily manage multiple e-mail accounts—including Windows Live Hotmail, plus other e-mail accounts that support POP3/IMAP. Better yet, Windows Live Mail integrates well with other Windows Live services, and downloads in minutes. After you provide your user name and password, you will automatically be linked to your Hotmail account, providing continued access to your email and contacts.

We encourage you to download Windows Live Mail at http://get.live.com/wlmail/overview.

And, to make your transition smoother, we've provided answers to frequently asked questions below. 

Again, thank you for your use of Outlook Express and we are confident that you'll be just as delighted with the new Windows Live Mail.

Your Windows Live Mail team

Frequently asked questions:

Why are we disabling DAV?

DAV is a legacy protocol that is not well suited for client access to large inboxes. Over time, as we've provided more e-mail storage to our users—and now offer 5GB inboxes for free—a more efficient access protocol is needed.

What are we replacing DAV with?

We have developed a new, much more efficient protocol called DeltaSynch that is far superior to DAV especially for large e-mail inboxes. It enables email clients to only download changes since the last time the client polled the email server for changes. This is much more efficient and high performing than having to download all the headers in every folder as is the case with DAV.

Is DeltaSynch compatible with Outlook Express?

The new protocol unfortunately is NOT supported by Outlook Express and support would require too many changes to the Outlook Express software.

Is there a different or new mail client I can try that uses DeltaSynch?

Microsoft is providing Windows Live Mail, a free e-mail client that has the familiarity of Outlook Express and much more. This free, next generation email client  enables users to easily manage multiple e-mail accounts including Windows Live Hotmail and other e-mail accounts that support POP3/IMAP. Windows Live Mail also integrates well with other Windows Live services, is optimized to work with Windows Live Hotmail, and offers: 

• Offline mail
• Windows Live Hotmail account aggregation for those users with multiple Hotmail accounts
• Account aggregation for POP and IMAP mail accounts
• Rich photo-sharing capabilities
• Advanced search via integration with Desktop Search
• Safety tools (Anti-Virus scanning, anti-phishing, anti-spam features across aggregated accounts for customers who do not have an Anti-Virus product)
• Integration with Windows Live services including Windows Live Spaces
• RSS (Real Simple Syndication) feed aggregation
• Ability to send SMS (short message service) text to a mobile phone from Windows Live Mail

Where can I download the new Windows Live Mail client?

You can download the new client at http://get.live.com/wlmail/overview.

Now I am not being a dinosaur when I tell you that I read this mail with about the same amount of excitement that one brings to the prospect of a root canal without novicaine. I don't dislike Windows Live Mail, but I am not a diehard "upgrade immediately" kind of guy on my home machines.

Hell, I didn't even move from the Exchange client to Outlook until I had to, and as a consequence I had a client that launched much faster, and I neatly avoided the bulk of the email virus epidemic. Plus with my combination approach of accepting every meeting request but only episodically launching Schedule Plus, I had a built-in affirmative defense for not showing up at meetings -- "Sorry, I forgot to launch Schedule+. Oops!". 

All of which I think proves that being slow to upgrade is not always a bad thing. :-)

I mean, who are these Windows Live Mail people kidding? I copy the mail I want to keep locally and the only time I have had more than even 10mb of storage used was when messages moved to the Deleted Items folder via spam rules had not yet been deleted by the Hotmail servers.

I don't need 5 gigs of storage in this account now and really never plan to. DAV was working just find for me and so was Outlook Express. I have other machines that I run the WL client on, and kind of resented the push. Why couldn't they just say that they were dumping this feature in this old product, and not try to act like they were doing me a favor? I much prefer people to be less smarmy about depriving me of services, you know?

So anyway, I looked toward the end of the month and just decided that on or about the 25th I'd switch over to a gmail account and just call it a day. In the back of my mind was hope that Microsoft would back off of this more aggressive move, but I was hardly going to push that agenda.

Howeer, today's email made me feel much better:

Dear Microsoft Outlook Express customer,

You may have received an e-mail from us letting you know that Microsoft is planning to retire the DAV protocol that Outlook Express uses to access Windows Live Hotmail. In response to customer feedback requesting more time to evaluate alternative solutions, we have decided to postpone retiring DAV and we are investigating other alternatives for accessing Windows Live Hotmail via Outlook Express. This means that if you use Outlook Express to access your Windows Live Hotmail account, you will continue to be able to do so beyond the previously announced June 30 transition deadline. We will be sure to update you once we have additional plans to share and early enough in advance to help ensure a smooth transition in the future.

Additionally, Outlook Express customers that use Windows Vista or Windows XP are always welcome to download and use our next generation free email client, Windows Live Mail, providing the familiarity of Outlook Express and much more. You can download the new client at http://get.live.com/wlmail/overview.

We appreciate your feedback, and encourage you to continue to provide it.

Sincerely,

Windows Live Mail Team

This is very good, and omce again underscores the benefits of not being so eager to jump in and do stuff just to get it done.

They say the early bird gets the worm, though to that I have three answers (which one I use depends on who is saying it and my mood at the time):

• That may be true, but the second mouse gets the cheese.
• Yes that is true, but it only proves that the worm should have slept in.
• I live for and love the snooze button, you miserable redneck time nazi with a hat!

So we'll see what they come up with, and in the meantime allow me to thank the customers who did not go softly into that good night on this one. You rock! :-)

Nancy would have been so proud....

This blog brought to you by Ꭰ (U+13a0, aka CHEROKEE LETTER A)

Charles Riley (a Unicode colleague from Yale) posted a link to In South Africa, Chinese is the New Black.

As I went through the ups and downs of the article, I didn't know what to think.

I wondered about the notions that came into play -- being able to be considered an "honorary white", being able to be considered black if you happened to be "retrospectively black" there at the right time.

Of course most of the Chinese living in South Africa don't qualify as being black (only the 10-12,000 or so who were there before 1994 do). The rest of those who are Chinese in South Africa are still honorary whites.

The lawsuit was (according to this report) targeted to try and create this situation.

But if you are Chinese how do you establish that you an "honorary white" or a "reclassified black" exactly? And is this a status you have to hang onto for all of the time you are there?

Do you have to self-identify for jobs irregardless of whether you want to qualify under the post-apartheid benefits?

And I wonder how diverse does the workplace feel if you don't know the status that everyone is there at the job under. Come to think of it, I wonder how diverse does the workplace feel if you do?

I have friends, both white and black, in the USA, and some of them have strong feelings about affirmative action. But beyond a Law & Order episode or two, I don't think any of them ever talked about the idea of folks being classified as having a skin color based on circumstances that have nothing to do with one's skin color.

Don't even think I am going to wonder about whether this needs to be an Active Directory attribute or something like that. I'm going to pretend that this thought did not even occur to me, but are there any consequences that would affect software like bookkeeping programs that have to keep track of quotas to ensure compliance?

I'm really having trouble getting my head around this one....

Remember that old Bloom County where Michael Binkley asked Steve Dallas whether Adam and Eve had navels, to which Steve said something like "Well you can just rock me to sleep!".

Now you know what I'll be saying to friend Riley next time I see him. :-)

This blog brought to you by ▣ (U+25a3, aka WHITE SQUARE CONTAINING BLACK SMALL SQUARE)

The other day, Raymond Chen blogged about How do the common controls convert between ANSI and Unicode?, in response to a question in his suggestion box:

In the context of an ansi (not unicode) app: How do the common controls (listview for example) decide which code page to use when translating multibyte to widestring?

I had to debug an ansi app that was displaying corrupt strings on a traditional chinese system because the dialog font was causing the listview to use a codepage other than the system ACP when translating multibyte to widechar.

Although I would seldom if ever disagree with about anything that builds out of the Shell depot, in this particular case I know of two specific exceptions to the CP_ACP rule one generally sees, though the differences may have less of a direct relationship to the Shell/comctl32 code, meaning he might still be right within his domain. :-)

The two other behaviors I have run across in various versions of the common controls:

• Use of the thread code page (CP_THREAD_ACP)¹.
• Use of the code page associated with the font charset selected in a device context.

I honestly don't know much about the first one, but I remember reports of bugs where changing the thread locale (which changes the thread code page) would change the behavior here, and particularly on the pre-6.0 controls there was a real ANSI-Plus thing going on here that tried to move beyond CP_ACP, so while I had no proof it was true I suspected it might be.

The second one, I have more insight into since I had to debug it on a few occasions -- basically the text would not always be converted to Unicode at all; and the ANSI text is sent to GDI with a DC containing a font set to use a charset most associated with some other code page. GDI would then do its job to render and make choices there that it was kind of asked to, in a bizarre and not well understood sense.

As a rule, any time GDI tries to get into NLS stuff, the results are predictable -- buggered, every time. Thus we have problems like the ones I pointed out in What the hell is wrong with TranslateCharsetInfo, anyway?. Between problems like that and the one discussed in Double Secret ANSI, part 2 (the brokenest one yet, sorry 'bout that!) and Sometimes when you say 'the fix is in' you mean it in a good way, one thing is clear: the GDI folks should consider taking a trip over to the NLS team and giving them all atomic wedgies.

Just kidding, but you know what I mean.

For the Common Controls, when I was doing MSLU work I ran across many cases where having the latest updates on Win9x would give a lot of GDI-influenced support of text where adding MSLU and a CP_ACP mechanism broke test applications until I changed the code to do something more like this to get the code page to convert with:

UINT CpgFromHdc(HDC hdc) {
    int chs;
    CHARSETINFO csi;

    chs = GetTextCharset(hdc);
    if(TranslateCharsetInfo(&(DWORD)chs, &csi, TCI_SRCCHARSET))
        return(csi.ciACP);
    else
        return(g_acp);
}

So anyway, the CP_ACP rule should be the only rule. but there are way too many pieces of Windows that assume they know better what to use....

On the other hand, so do I -- UNICODE! :-)

1 - Now you know how I feel about this one if you've ever seen Nothing stinks worse than the thread locale, other than the thread code page. I think I was fairly unsubtle on my feelings.

This blog brought to you by ਸ਼ (U+0a36, aka GURMUKHI LETTER SHA)

In the before time, in the long long ago, when this blog was first finding its voice, I blogged two blogs:

• Accessibility, Internationalization, and Keyboards (#1: Shortcuts)
• Accessibility, Internationalization, and Keyboards (#2: Accelerators)

And a recent conversation with some localizers made me think about these two items from the point of view of a localizer.

In a strangely ironic turn, LocStudio (the Microsoft internal and privished tool, mentioned previously) reportedly refers to both accelerators and shortcuts as accelerators.

Anyway, the message I sent out to try and contrast these two items from the standpoint of localization went something like this:

For shortcuts, localizers do need to translate the term for the key (e.g. CTRL vs. STRG), and they will usually put in the letter that appears on the keyboard layout the user of the language is most likely to be using for whatever is under VK_C (if they don't then the user will not know what the text means). The localizer has no way to change the behavior since it is VK-based and code mediated.

For accelerators, localizers still need to translate the term for the key (e.g. CTRL vs. STRG). But for the letter itself, they are fully in control of the accelerator (on menu or dialog) and can change it to anything they like, with the choice usually based on a letter in the word they have chosen in the translation. The localizer has the complete control to change the behavior (assuming the developer did not hardcode the strings!), since it is character based and localizable text mediated.

Localizers therefore have a much more controlling role for accelerators than for shortcuts; in the shortcut case they are forced to just be descriptive, and in rare interesting cases where the VK the code uses does not exist on the keyboard (a problem that has come up in the past), the localizer has no real choice since the application is broken when using that keyboard worldwide (and is just more noticeable in their language).

Thus the fact that the main tool used by MS localizers calls them by the same term even though they play very different roles in the work of those same MS localizers is pretty tragically ironic....

This blog brought to you by ණ (U+0dab, aka SINHALA LETTER MUURDHAJA NAYANNA)

So yesterday via the Contact link, I got the following mail:

Dear Michael Kaplan,

Our editors recently reviewed your blog and have given it an 8.3 score out of (10) in the Technology category of Blogged.com.

This is quite an achievement!

http://www.blogged.com/directory/technology

We evaluated your blog based on the following criteria: Frequency of Updates, Relevance of Content, Site Design, and Writing Style.

After carefully reviewing each of these criteria, your site was given its 8.3 score.

We've also created Blogged.com score badges with your score prominently displayed. {Instructions deleted, I did not do them!}

Please accept my congratulations on a blog well-done!!

Sincerely,

Amy Liu
Marketing Department
www.blogged.com

Turns out that out of 5,486 ranked blogs at the time I got the mail, I was #523.

I feel like I was in the Boston Marathon or something. Only without realizing I was running? :-)

I have mixed feelings about this. It is I suppose an honor, and being ranked #523 is interesting. But when I looked at some of the ratings of a few other blogs I was not so sure....

I mean, I am fine with TechCrunch as the #1 slot any day of the week, and some of the blogs given 1.0 ratings (the lowest number the site seems to give) deserve that. But in between?

Let's just say that the wisdom/judgment of anyone who puts my blog over some of the ones that this site did should seriously be questioned. :-)

More seriously, I do have be skeptical about such judgments -- I am not so worried about people ranked higher than me, it is more the ones rsnked lower that probably should not be (and would not be if I were one of the editors).

Plus just yesterday a friend was asking me if occasional reference to pornography and suh were really needed, even if the underlying blogs were about language or linguistics -- shouldn't I have points subtracted for such juvenile things on an MSDN blog?

Anyway, you can see the details here. It was harder than it should have been to find out the overall ranking, since you have to look a the full list and find yours in it. I did it but I can't imagine spending a ton of time doing it again.

I'll keep the link on my blog for now (it is at the bottom of the Disclaimers Redux link, on the right) but I've never been into the cult of me quite enough to want to play king of the mountain with anyone (so I masy remove it eventually).

No naked pictures of any of the editors were involved with this process!

This blog brought to you by ꡰ (U+A870, aka PHAGS-PA LETTER ASPIRATED FA)

Apologies for the Sound of Music allusion!

Via the Contact link, tiwunster asked me:

Hi I have a question regarding the changing of default East Asian fonts. Recently Microsoft has release East Asian ClearType fonts for Windows XP.

I've installed all of them however I have to configure the applications individually to make use of them since Windows XP default to non-ClearType ones.

Is there any way to 'force' XP to default to the new fonts when rendering East Asian characters? Thanks.

Best regards

Now indeed, the cool Vista ClearType fonts are now available for XP (for genuine XP installations only!):

• Traditional Chinese ClearType fonts for Windows XP (Microsoft JhengHei Regular and Bold)
• Simplified Chinese ClearType fonts for Windows XP (Microsoft YaHei Regular and Bold)
• Korean ClearType fonts for Windows XP (Malgun Gothic Regular and Bold)
• Japanese ClearType fonts for Windows XP (Meiryo and Meiryo Bold)

But no, there is no supported (or indeed supportable!) way to change the defaults here.

There are several reasons for this....

The first is that the reason these fonts were made available on XP is (as the various blurbs indicate) for the benefit of WPF (Windows Presentation Foundation), which really does not work as well with the conventional East Asian fonts that ship in Windows prior to Vista, since they are TrueType fonts that contain embedded bitmaps (and those embedded bitmaps do not work in WPF, or in GDI+ incidentally), which means at smaller sizes the fonts are pretty unsightly in WPF.

That is the scenario, and the reason they were released. Their use in ways beyond that -- such as adding them to the GDI font link chain -- was not tested or verified.

And everyone remembers the First Tester's Axiom, right? If you have not tested it, assume it is broken.

What could be broken? Well, any application using default fonts that was suddenly getting glyphs from a different font, just as the most obvious example!

Second, if you look at the GDI font link registry key (HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\FontLink\SystemLink) for these values of these fonts in Vista, you will see something different. For example the Meiryo value:

MSGOTHIC.TTC,MS UI Gothic
MSJH.TTF,128,96
MSJH.TTF
MSYH.TTF,128,96
MSYH.TTF
MALGUN.TTF,128,96
MALGUN.TTF

See all of those numbers? That was some crucial scaling support that was added to Vista to help allow for the very different nature of these fonts and the way that GDI font linking was causing very strange effects as the font link chain as being followed. The underlying support for this different format requires work that was done in GDI in Vista, and that support could not be backported for the sake of a WPF, which never even uses the settings anyway....

Adding these fonts to the font link chain, in addition to being unsupported (and therefore probably buggy) and lacking the scaling support to have the font sizes appear properly in XP when picked up through font linking (and therefore pretty much guaranteed to be incorrectly scaled in an unknown number of cases, is also something that was neither done for or tested in XP or Server 2003.

You can certainly try to add values yourself without the scaling information (since that feature does not exist prior to Vista) but if you make a mistake in the registry key you could break all font linking, which is a very big risk to take to pick up some other fonts for your applications without specifying them....

This blog brought to you by ㅀ (U+3140, aka HANGUL LETTER RIEUL-HIEUH)

Microsoft program manager Garrett McGowan mentioned yesterday:

Please compile Audiosurf as a Unicode application. Surely the game isn't supported on Win9x? Around 15 percent of my music collection is unavailable in the game because the album and song metadata...

This is the site where you can read about Audiosurf.

Dag Mitchell, is there really an excuse for this kind of thing anymore?

Huge parts of the game industry have moved to Unicode, it really is unfortunate that holdovers like this still happen along.

Now lots of people have been complaining about this, like here in their forums and in blogs like this one. People have been pointing out this problem for months now, and it really is time they paid attention to the complaints....

This blog brought to you by Ｕ (U+ff35, aka FULLWIDTH LATIN CAPITAL LETTER U)

Ian Boyd asked over in the Suggestion Box:

Dialog Units.

What they were designed to do. Where they would have hopefully worked. How it is they originally did work. When they began to break down. The workaround used to try to keep them working. How they no longer work for their original purpose. Technique to try to roll your own.

If you think about, Ian has managed to pose seven inquiries without asking a single question as a question!

Quite an accomplishment, when you think about it.... :-)

Now covering all of this is almost the topic for a book chapter, so I couldn't really cover all seven topics in depth here is this one little old blog.

But I'll try to give a quick answer for each (when possible) and reserve some of the more complicated issues in each for another day.

So, here we go, with one initial question added that should likely have been there, and with each of them reworded to be questions (I am not being scientific here, these are all of the cuff and any I get into deeper I will be a bit more formal about.

The nitpickers should have a field day here!

What is a Dialog Unit?

If you take an "average character" (by convention, Win32 decided long ago that the letter x made a nice average character), a dialog unit is 1/4 of the width of that character (think TEXTMETRIC.tmAveCharWidth) and 1/8 of the height of not the average character but of the normal character height (think TEXTMETRIC.tmHeight). Obviously these two metrics will vary depending on the font you use, so the Dialog Unit is based on the specific dialog font for the Windows session.-- which is itself a choice based on both DPI settings and language.

What were Dialog Units designed to do?

In the world where the default font itself might have different sizes (which implies the text would have different sizes) and the DPI would cause more such weirdnesses, the way to make sure that by default the dialog would resize when these other factors changed was to make the size of the dialogs change when these other things did.

Where would Dialog Units have hopefully worked?

The place where they would have hopefully worked is when you wanted your dialogs to scale even as these various settings changed.

How it is that Dialog Units originally did work?

How they originally did work is the one place the still do work -- when your dialogs use regular old Win32 USER/GDI and the text in your dialogs uses the default dialog font face and size and weight. When you do all of this, then changing the language/font/DPI settings should allow the dialog to scale so that text that fit in one case will continue to fit in the other cases.

When did Dialog Units began to break down?

There are lots of changes that happened over time that caused this ideal to break down; I'll list a few here:

• Some dialogs would use other font faces, weights, and/or sizes with very different metrics from that default dialog font;
• Some dialogs would use multiple fonts, with not all of them having the same metrics as that default dialog font;
• In situations where the language changed, the size of the translated text would also change in many cases;
• Other technologies came along that used different scaling systems (e.g. pixels, twips, etc.) to avoid the confusion that the scaling and perceived lack of predictability thereof that the Dialog Unit system often led to.

Because more and more reasons for the scaling system to not work out kept happening, the system became more and more arcane appearing and less and less relevant to what people actually wanted for their dialogs.

What workaround was used to try to keep Dialog Units working?

No specific "workaround" was architected as a workaround -- these other technologies had other ways to define things and tried to solve the problem different ways. They just chugged along and tried to do their job.

How do Dialog Units no longer work for their original purpose?

These new technologies were still using GDI to create the fonts and USER to create the windows, which meant that these separate technologies occasionally affected by the good old fashioned Dialog Unit scaling -- an interesting problem for Visual Basic <= 6.0 (for example) where people would work out sizes perfectly that suddenly would not work well on another system.

What technique can one use to try to roll one's own solutions for the problems that Dialog Units were originally intended to solve?

This one I am going to hold off on for a bit because a deeper and more formal look at the consequences of all of the above issues will perhaps make Ian and others believe that as bad as the original solution may or may not have been, the things that have come later have not really distinguished themselves as much as they might have liked, and that rolling one's own solutions is likely to cause more problems rather than solve the ones that are there.

And there we go. Obviously I'll becoming back to some of these topics another day. :-)

This blog brought to you by ; (U+037e, aka GREEK QUESTION MARK)

Regular readers know how I am always encouraging, cajoling, and sometimes even threatening developers to respect the user's locale settings choices.

Not everyone is of a mind to do this, however....

Like just the other day when someone was sending me mail about a problem they were having:

We found that the GDI uses the user locale regardless of  the current thread locale for rendering. Data gets formatted using Current thread locale but at last moment the digit replacement happen using the user locale. Is there a strong reason behind this inconsistent behavior or is this a bug in GDI.

Of course formatting behavior, via functions like

• GetCurrencyFormat
• GetCurrencyFormatEx
• GetDurationFormat
• GetDurationFormatEx
• GetNumberFormat
• GetNumberFormatEx
• GetTimeFormat
• GetTimeFormatEx

is based on the locale you give it, and there is no constant that tells these NLS API functions to "use the thread locale" (though there are ones that will look up the user locale for the caller if requested. :-)

So that text in red above is not really accurate, unless there is some rogue library or program clling GetThreadLocale and pasing the results to one of the non-Ex functions in the list above....

Becsause the truth is that GDI doesn't give a crap about formatting or really anything related to locales, with one signle exception:

Digit Substitution

Any time you go to render text it will grab those digit substitution settings in the user locale (including the user override information) and use the info to decide how to display numbers.

And since there is no way to override those settings at the level where GDI uses them like there is (kind of) for Uniscribe (ref: And the digits just keep on coming).

Well, there is one way....

If you are doing your GDI rendering vie ExtTextOutW, then there are two flags that have been around since Windows 95:

Basically these are the (very differently named!) equivalents of GetLocaleInfo's LOCALE_IDIGITSUBSTITUTION settings:

So you can override the assumption of the GDI code that the user locale preferences are to be respected if yo uwant to, by calling low level functions and pasing one of those two ETO_* flags.

But there is no way to make it easier than that, sorry. And not because the OS respects me and my advice,  but because sometimes the only way the user will get any respect is if we force developers to give it to them.... :-)

This behavior is completely by design.

This blog brought to you by R (U+0052, aka LATIN CAPITAL LETTER R)