February, 2008

Trends are not rules. Hell, they aren't even guidelines.

Trends really are really little more than our attempt to make sense out of what is happening and perhaps to try to figure out what will be happening if our analysis of trend(s) was valid....

Anyway, thinking back to Hello! I am bored this afternoon. I am a nice Blog that would like to blog with you....  and YASSU (Yet another SIAO spam update) the spam was definitely on the way down, at an all time low, in fact.

However, the trend did not last, and the current spam as of midnight is actually sitting at 4793, which is an all time high in the time that I have been looking at the amount of spam caught by the filters within the window of the last seven days.

Yes, that is right.

4,793

That is a lot of spam caught in the trap -- with none making it into the blog!

And it is currently trending upward still -- with the incoming each day greater than the outgoing from seven days prior.

In that huge list, I have almost certainly (occasionally) lost genuine comments caught as false positives, and since I am not reviewing them all, those comments are lost for good. So if a comment of yours never made it live, then I am sorry about that....

Now my experiment of turning on trackbacks/pingbacks just before I post and then immediately turning them off again has successfully removed the threat of all trackback/pingback spam, at the cost of losing all external referrals to posts that I do.

I am still not sure how I feel about that, because even though I can look at referral logs, I usually don't.

I also find my regular reading of blogs has suffered as of late. I get to it eventually, but there is now a pause of between a week and a month before I see posts sometimes.

So the net effect today is that I now feel much more isolated from the blogosphere, which may or may not be such a bad thing -- I am still deciding. It certainly leaves me more time to live my life (the side effect of that is that I probably blog less about it, which is probably not a bad thing).

Maybe it is time to let some of the spam back in, to get connected again!

And then to get back into posting in the two series that I started while I was in India on text based TIPs and reversing sort keys that people have been holding their breaths waiting for!

This spam weather report brought to you by ☔ (U+2614, aka UMBRELLA WITH RAIN DROPS)

So Leili sent me some email yesterday.

I've known her for a few years now. She works on the .NET globalization team and I reported to her husband a few years back when the World Ready Guides were the World Ready Guys -- the alias now forwards to an entirely different one that I am on, I guess you could way they are World Ready Gone now? :-)

Anyway, Leili (I wonder if her name means the same thing in Persian as it does in Hebrew? Riddle me a good time to ask that question!) had been forwarded an article entitled The Best Time To Do Everything by Michael Kaplan (the same text as this article, in fact) and she wondered

Did you write this article? It’s very interesting. Did you really interview 200 people?

I had to break the news that it wasn't me (this other one write a book with the same title). It did help her figure out one issue troubling her:

...I was wondering if it was you.  I thought you didn’t have that much free time for non work related stuff...

But the truth is that there are many Michael Kaplan's out there in the world.

In fact, as I go off to (in the words of Mike Freed on Love Monkey) Google myself, I found out something interesting (for some values of interesting, this is not the wort of thing I do all that much!).

Windows Live Search finds two that are me in position #29 for my mostly defunct cimpany site and #30 and #32 for the blog (see it here), while Google gives me three in the top five including the top two slots (see it here), which I suppose means that Google find me more relevant than Microsoft -- or at least Live Search -- does. Admittedly I did miss a meeting with the Search folks while I was in India so maybe it was a punitive downgrade in the rankings. :-)

For other sites, ask.com puts my mostly defunct company site first, my out of print book second and fifth, and the blog ninth. And Yahoo puts the blog third and fourth and the defunct website fifth. And AltaVista puts the blog in the sixth and seventh positions with the defunct website eighth.

Though all kidding aside, it might be fascinating to actually analyze these results in full for say the first hundred entries on both to see what makes a person "important" in these different search engines. Not really a scientific study reverse engineering the technical process behind their ratings but a pseudo-scientific study that assumes the rankings are based on personal preference of those good friends Dr. Google, Dr. Live, Dr. Ask, Dr Yahoo, and Dr. AltaVista, and the importance they place on various achievements in life.

One would have to ignore some of the problems like the overall importance or Britney Spears in such results, of course. But it might still be entertaining.

On the other hand, I think I'd be happier being farther down the list anyway. The whole cool factor issue makes me wonder if it would be better to be harder to find. Blogs seem to take on a life of their own, and as far as I can tell the other Michael Kaplan's book doesn't say anything about the best time to blog!

This post brought to you by ; (U+037e, aka GREEK QUESTION MARK)

Another of the stories about that India trip....

The last night I was there, Kumaran wanted to take me out to dinner. What with the publishing deadline that just struggled to hit, he and I did not have much time to chat and all.

But then his boss Anandan wanted to take everyone out, and the wishes of the boss often trump in such cases. We went to dinner and then a bunch of otehrs joined us and ended up seeing Across the Universe, that "Beatles" musical.

I had seen the movie in the US with some friends but I have to admit I enjoyed it more in India with that large group sitting in the back row of the theater. It was ust a better experience all around, even with the oddly placed intermission!

Kumaran and I got to talk the next day before I left, it was more workish talk anyway so I think everything worked out well.

So on my last day, I went shopping.

I had a cab driver at my disposal for the day and traveled to many different stores and shops all around Bangalore.

On my list there were many things -- books I wanted, authors I wanted to look into, movies I had the titles of and some directors to check out, CD listening material.

I also had requests for two other translations of Harry Potter books (Bengali and Malayalam) from people who had read How would Harry Potter have pronounced शहिवाख़्‍ का दर्पण, anyway? and It is not always a conjunct; there are some letters, too!, hoping perhaps that I would perhaps embarrass myself in their languages as well. :-)

(I did find hints about translations but not in Bangalore -- they may end up coming to me at some point.)

And of course sandalwood. Can you go to Bangalore and not get sandalwood?

Many of the other items I was looking for had reasons behind them, but I did not have particular goals in mind for the Sandalwood I was looking for, other than one idea for a quest. Basically I have a friend whose IM avatar is a picture like the one I talk about here of Oolong and who also likes sandalwood. My path seemed clear.

I needed a sandalwood bunny.

Every person I mentioned my quest to was really amused. Elephants? Easy. Tigers? Also easy. But a rabbit? They were all amused, but recommended I not judge the day's or the whole trip's success on the success of the quest....

So I traveled to many different places, and in between each store I went I went to a place that sold sandalwood. This was a serious quest.

Almost invariably, every shop that sold sandalwood also sold silk and cashmere and rugs. They each had a shopkeeper who had a brother who came out to offer me tea. Even if I did not recognize the string sales pitch at the first store, I certainly would have by the third!

And I had a unique way to keep them at bay -- I simply tell them I am prepared to be incredibly generous to the store that is able to find me the unique item I needed to achieve my quest.

{pause for the shopkeeper to smile at me like an adopted son returning home with an unexpected fortune}

I have been searching far and wide in many stores looking a sandalwood --

{the look on the shopkeeper's face is clearly elation at this point}

--bunny.

{Have you ever seen a smile that remained on a face while all of the joy behind it recedes? I have -- in every sandalwood shop I visited that day!}

They would scour the store and try and interest me in the myriad of other animals they had to offer -- turtles, ducks, snakes, donkeys, kangaroos, owls, and do on. The tea flowed heavily and I could have covered an acre of land had I bought every carpet offered to me that I refused....

But there was no sandalwood bunny.

Luckily there was good alternation between the sandalwood and the other stores, so that I could be buoyed by success as I was wrecked upon the rocks on my quest.

As the shopping trip had almost wound down and we were on our way to the last store selling sandalwood, the taxi driver asked me if I considered commissioning a sandalwood bunny, since he doubted I would ever find one.

I look at him like he was Captain Kirk finding a way around the personal Kobayashi Maru I had set myself up with. I wonder if he would have understood the reference? :-)

Of course there was not enough time to get that done this trip, but maybe next time I might look into it.

And there probably will be a next time....

Back at MSRI I met with Kumaran finally, and said goodbye to all the people I had met (thanking all of those who had given me advice on where to go looking for things during the day!), followed by a trip to the airport and a nineteen hour flight that involved a scooter that was reported broken in Frankfurt and reported lost in Portland, only to be delivered to me over the weekend working just fine. Too bad there is no room for a Shakespearian reference in the title. Oh wait, there is room. Never mind!

And tucked away with the small mound of things I bought was a small collection of sandalwood animals of various types and next to them a few alabaster bunnies (conclusive proof that this blog post is about a true account of the quest (and it has only now occurred to me that I forgot to take a picture of all of the animals while I had them, dammit!), it felt a little like I was flying home on Lufthansa's Ark....

All things considered, an unforgettable trip that is definitely not going to mark my last visit to India.

This post brought to you by ꘖ (U+a616, aka VAI SYMBOL NII)

It has been a while since I had a Broom-Hilda strip that had me laughing like this one:

I suppose I could ask people what they thought the optimal sorting algorithm might be.

Could it be any worse than Obama's foray in this space? :-)

(Hat tip to Tina)

This post brought to you by L (U+004c, aka LATIN CAPITAL LETTER L)

(This is information I talked a bit about at Microsoft Research India when I was in Bangalore)

Now there have been many times that I have talked about the wonderful feature that Uniscribe's font fallback provides for us (ref: these blogs).

Of course there is one thing I forgot to mention about it.

And that is that it has a pretty fundamental flaw that comes up quite easily in the most common scenario under which it is used....

You see, one of the main goals of this default built-in fallback behavior is to allow you to keep your fonts as they are and then any time your fonts don't have a glyph for the text you are showing, it finds one from its own little list of fonts that ought to work.

As an example, we'll take our poetic word from Learning to spell in Bengali (when one doesn't know the language) and intersperse it inside the English, making the string something like:

Godhuli গোধূলি Godhuli গোধূলি Godhuli

Now I am going to take that string and shove it over in Notepad, where my default font is usually Consolas 8 pt. Here is what it looks like:

OMG that is awful - take a poetic word and turn it into an ink smudgy looking kind of thing. I guess we have to make it bigger and try 10pt:

That still looks pretty awful, maybe 12pt will be better?

Okay, it is less of a smudge now -- I can actually tell it is Bengali. But I am struggling to read it. I guess we have to move to 14pt:

Okay, now that text I can read. For completeness we'll look at 16pt too:

Maybe you can see the problem when you look at these last two ones. The trick is not look at the Bengali so much and instead look at the English.

Do you see it?

Yes, that's the problem -- you have the choice of either illegibly small Bengali or obnoxiously large Latin text.

Kind of the same problems that came up in When a font looks like crap.... and 'But there's no Latins in our Divehi font, Duckman.' 'Well, they don't *speak* Latin in the Maldives, do they?', come to think of it.

But wait a minute, why should that be? I mean even in XP, the Vrinda font does have those Latins in it. So why should this be a problem if the font used for Bengali has the Latins?

Think about it -- the way font fallback works, you are combining your original font for the Latins with the fallback font for the non-Latins!

In other words, Microsoft paid someone to build a font that might have done extensive work to make sure that the scripts within the font are scaled together quite well, and the most common way that people will use the feature will never benefit from the work!

Let's test the theory and look at with Vrinda as the font for all of the text, explicitly. First at 16pt:

Wow that looks better! Let's see 14pt:

Also really attractive. How about 12pt?

Still looking good! If we move much further down we'll run into problems again, as you can see with 10pt:

Well, let's also look at some of these in Vista. Like here is 8pt an 14pt Consolas:

Looks like the same problems are there, and we'll verify with 8pt, 10,pt, and 12pt Vrinda:

Wow, Vista even looks pretty good at the 10pt size!

And the main point, that the effort to scale the two scripts such that they look good together is a very powerful way to support better looking text.

Now the only problem is that the principal goal of Uniscribe font fallback (to not require one to explicitly choose the best font) is kind of subverted by requiring that explicit choice to be made, now isn't it?

Now as I pointed out in 'But there's no Latins in our Divehi font, Duckman.' 'Well, they don't *speak* Latin in the Maldives, do they?', there is clearly an effort to include the Latins in the fonts used for fallback, but that ah not completely happened yet, and even if was there of course the problem for all the other characters in Unicode exists -- the hard work of typographers to do the best scaling should indeed be leveraged, but a reasonable automatic scaling should exist for the less than ideal case where not all of the needed characters are provided.

But what would be needed is an effort for Uniscribe to take a more holistic approach to font choices that works beyond the individual run when adjacent runs cross script boundaries, so that the right font will be selected in such cases.

This is a reasonable approach used for the analogous problem of handling combining diacritic placement -- ideally you should provide precomposed glyphs for the font to use, but if not then information about attachment points can do a less than awful job for fallback?

Of course it would require some work to do that scaling between different fonts.

We will assume that the attempt to scale different scripts within the same font when the font foundry did not do the work to make them look good together -- case in point Arial Unicode MS -- is completely out of scope and thus will continue to look as bad as it always has looked; perhaps they wanted it this way!

Now some people might look at this blog oddly since I seem to be providing not just workaround but suggestions on how the product itself could work to fix the problem. But the issues in providing that scaling are quite complex, to which I'll add the discussion from the crap cartoon font blogs (Part 1 and Part 2) that talk about other issues that complicate the problem further....

This post brought to you by গ (U+0997, aka BENGALI LETTER GA)

Once you get a reputation among your readers, you can find them emailing you as they run into items of interest on the web, or just the opportunity to talk about your own oddities....

Like when fellow MSFTie Kim Hamilton will claim in Facebook that I'm the most likely to have the answer to my obscure glob loc question in his blog or to have a unicode character of the day in his blog.

Or when an xkcd comic comes out that has a Unicode joke in it and you get six emails about it within five hours (Tobi, you were actually first!).

Here it is:

(Click on the comic to go the site, and as with all xkcd comics, the ToolTip is critical to the joke!)

Within NameList.txt, there is a nice explanation of them:

@ Noncharacters
@+ These codes are intended for process-internal uses, but are not permitted for interchange.
FDD0 <not a character>
FDD1 <not a character>
FDD2 <not a character>
FDD3 <not a character>
FDD4 <not a character>
FDD5 <not a character>
FDD6 <not a character>
FDD7 <not a character>
FDD8 <not a character>
FDD9 <not a character>
FDDA <not a character>
FDDB <not a character>
FDDC <not a character>
FDDD <not a character>
FDDE <not a character>
FDDF <not a character>
FDE0 <not a character>
FDE1 <not a character>
FDE2 <not a character>
FDE3 <not a character>
FDE4 <not a character>
FDE5 <not a character>
FDE6 <not a character>
FDE7 <not a character>
FDE8 <not a character>
FDE9 <not a character>
FDEA <not a character>
FDEB <not a character>
FDEC <not a character>
FDED <not a character>
FDEE <not a character>
FDEF <not a character>

Kind of puts it all in perspective, trying to describe the need to capture text that explains a joke in a comic strip that contains the non-characters in order describe them, in plain text, huh? :-)

Sponsorship of blogs by non-characters within Unicode is still a very controversial issue here

In Windows, as in life, it is important to keep a bit of independence from one's parents.

They may have  lot to do with your initial placement, sure. But beyond that, they don't get to be too involved....

I remembered this a while back when Naga asked:

Hi,

My application has an edit box in a dialog. Who will get the message WM_CHAR when user enters characters or tab etc.. when the focus is on the edit box?

I am presuming that the message WM_CHAR will come to parent. But it seems that message is not coming to parent.

Do I need to handle WM_CHAR messages for child controls separately in their own dialog proc?

Can’t we handle them in the parent, Dialog proc itself?

Regards,
Nag

I shudder to think about what it would mean if one's parent's were notified about all of the random characters one had spending time in one's room -- I didn't even always do this when I lived in their house, if you must know!

Thus my response to Naga's question:

The EDIT box's own WNDPROC gets the WM_CHAR notification; this is the EDIT control's bread and butter, and like in life our parents don't have to be quite *that* involved with our core work....

Now although the truth is that there are times that a parent can have a subtle effect on things, in Window just as in life, it is usually someone else that decides things.

Like when Jeffrey asked the other day, based on a newsgroup thread:

My customer configured his Windows XP for Arabic using MUI, and we can see IE drawing its file open dialog mirrored.

Then, he called SetProcessDefaultLayout( LAYOUT_RTL) in his win32 C++ app, but still the file open dialog is drawn LTR, ie. not mirrored as we would like.(Everything is RTL except for the OPENFILENAME dialog)

Any ideas what more we need to do?

Thanks in advance!

After a bunch of work, it turned out that the directionality of the hwndOwner window of the OPENFILENAME structure was the deciding factor for whether or not the dialog would be mirrored.

Do we need better proof that our parents don't own us and have no say in the characters that live with us?

Or does it all point to the immaturity of the windows themselves -- don't we look at phrases

I'm not a child for you to control the characters I hang out with!

or

I am not a child for you to control what direction my life takes!

and shake our heads at these children who just don't understand that we do not wish to hem our children in way such as this?

On the other hand, I could simply be over anthropomorphizing an already way too over-extended metaphor to try to tie two disparate points together in a single blog.

Yes, on reflection that is what is going on here.

Never mind.... :-)

This post brought to you by ⽗ (U+2f57, aka KANGXI RADICAL FATHER)

This post brought to you in care of associative linkage -- I know where it starts but no one knows where it will end! :-)

So yesterday was the first day of Carnival in a bunch of places.

And tomorrow is Ash Wednesday in a whole bunch of places.

And today?

Today is Pancake Day in Lithuania!

Now it is the kind of name that could easily inspire all kinds of crazy stories (or you could read the Wikipedia article about it here and skip the drama).

Or you could have a drink, remember fondly Oolong, and then say those immortal words Oolong was famous for.

If you aren't sure what I mean, then you can just say

I have no idea what you're talking about... so here's a bunny with a pancake on it's head.

and not only will you have fulfilled your need to ,look me squarely in eye and say

HUH?

But you will also have said the words Oolong was famously associated with!

Now obviously Shrove Tuesday has nothing to do with the famous images of Oolong, but even so, Outlook's inspired view of the week in five day view

made it all come together, for me.

So let's remember Oolong, everyone....

I wonder what he did when he didn't know what people were talking about, which I imagine was fairly often since he was just a bunny.

It seems slightly egotistical to show a picture of himself each time, doesn't it?

Poor Oolong. I wonder if he ever just took a bite out of the pancake, like on Pancake Day? He could then just stop eating them and tell people he gave them up for Lent....

(Lent remind me of forty days which reminds me of the ark which reminds me that Noah was an asshole -- Martin Cruz Smith was the most famous person to suggest it, but he was actually the second person to point it out to me, the first being a rabbi several years ago who was pointing out to me that we always learn the wrong lessons from sacred texts -- and that sometimes you make the argument, even if it with someone you aren't supposed to disagree with!)

Not that I think one should disagree with someone who has a pancake on their head.

Though speaking of that Renko book, I wonder whether "Was Noah a carpenter or a sailor, and why?" would make a good interview question? Probably not.

Though I'd virtually hire anyone who had not read the book or found the reference but had a defensible answer to the question on this fine Pancake Day we are having....

This post brought to you by 끼 (U+b07c, aka HANGUL SYLLABLE SSANGKIYEOK I, who I do not even want to think about the inspiration for this sponsorship)

Reader Georges asked the other day via the Contact link:

Do you have a list of the national standards supported by the various collations that Windows and SQL Server and .NET support?

And similarly, regular reader Ivan Petrov asked in the Suggestion Box:

Hi Michael,

Do you have a list with the supported Keyboard layouts by Windows and the corresponding standard documents in which they are defined with their full document numbers and titles ?

I mean national standards like ANSI (US), DIN (DE), GOST (RU) and etc. or any other Standards like ISO, EN and etc. or Norm documents .

I will be very grateful to you if you post such a list as attachment to a post.

Regards,
Ivan.

Funny having two such similar questions in two areas so completely different from each other, no? :-)

The answer is really the same for both questions (fortunately or unfortunately, depending on your point of view)....

And that is no such list really exists.

Both keyboards and collations are attempts by Microsoft to capture language and market requirements and include them in Windows, and although standards are an important source of inspiration for each, the connection between them is seldom so complete that one could really claim that they are 100% implementations (as an example, some National collation standards tend to specify behavior for punctuation as well as language, while Windows does not tend to change its punctuation-specific behavior for individual locales).

Also (and this is me just speculating), if we documented such a direct connection via a compliance claim, Microsoft could be required to break its own stability guarantees regarding results anytime there was such a difference between the MS implementation and the one in the standard -- and changes in the standard would lead to new sorts and keyboards being needed whether they are used in country or nor....

Instead by using standards as "important, knowledgeable informants" to be considered alongside other sources of information, the appropriate distance is kept that allows the unique technical factors affecting Microsoft platforms to be taken into account without leading to conformance problems....

In addition, I am kind of being polite by implying that National standards for keyboards and collation differ from Microsoft only in such details and that they are completely valid descriptions with no problems in them; there are definitely times where (were I a less polite version of myself) I could easily start blasting countries or regions for flaws within their standards, causing all sorts of incidents.

And of course there are the times (especially in keyboards, especially with MSKLC available) that a government standard is the source for the layout that we ship which differs from their actual standard in some way (we do not do compliance testing on such things since of course it isn't our job to make sure they are following their own standards in such cases!), and putting direct connections on a web page (even a non-official page such as blogs are!) could easily get Microsoft into an uncomfortable support situation.

So I apologize to both Ivan and Georges for not having such lists (or similar lists for code pages that I to put up here). Sometimes the best way to not find oneself holding onto something nasty is to not pick it up in the first place....

This post brought to you by dž (U+01c6, aka LATIN SMALL LETTER DZ WITH CARON)

In response to my blog yesterday entitled A sign that is actually a symptom, and not in Unicode anyway, Dana asked in a comment:

What exactly is the "typical ideal kissing height for me while standing up"?

To provide some additional context, the text from the post:

And it is still with me even today, though only when I cause it by bending my neck forward. Usually for me it only runs down to my fingertips though during acute exacerbations (both genuine and inspired by subclinical illness) I sometimes feel it down my legs as well. It kind of screws up the whole "making a person's toes tingle" notion for me, as it is kind of like my waking state, and if I kiss a person of typical ideal kissing height for me while standing up then I will feel tingling in my limbs independent of how good or bad the kiss is.

In my late teens when I discovered how much fun it was to kiss girls just for the sake of kissing them, I had a fair amount of opportunity to compare and contrast various ways of kissing people of different heights.

I also picked up some bad habits like kissings too aggressively or messily that I was only able to shed later as I realized the habits were (in fact) less than ideal, despite mostly being picked up from others and thus were not such a bad idea in specific cases even though a terrible idea in the general case!

An additional lesson I learned is that aggressive kisses that lead to hickeys are a really bad idea if a girl has an older brother who fancies himself a protector of virtue eventually, even if both boy and girl think them to be a good idea at the time; with the 3wisdom of afterthought, I know they were not....

But one thing I learned when kissing someone while standing up, it tended to be a better kiss when there was a height differential of 4-8 inches so that one person is kind of "kissing down a bit" toward the other person (when kissing while not standing up, height is not nearly as relevant since we are all the same height lying down).

I'm about 5'10".

Although in theory the woman could be that much taller than the man, the number of women I dated who were between 6'2" and 6'6" is best described as scant (not through any prejudice of mine, most women who are taller seem to not prefer men shorter than them), so I have more data for the other way around.

Not this is hardly an absolute with people -- I have been involved in some earth-shattering kisses with people where the differential did not exist, and also one can quite handily "fake" the differential by standing in heels (not my personal preference on my feet since they are so uncomfortable to wear but women don't seem to mind them as much!) or standing on a step or something. It is just that all things being equal the differential seems quite effective at helping to deliver those kisses that helps to make toes tingle even without Lhermitte-induced symptomatology, and in my mind is the typical ideal height for kissing while standing up.

In later years, when the whole Lhermitte thing reared its head (or neck, as it were) which could cause "toe tingling" without the effectiveness of the kiss being the main inspiring factor, as did the "positive" Romberg sign (which would cause me to fall over if my eyes were closed and I were not braced against something), the ideal kissing height issue became more theoretical and I don't think I am as good of a kisser in such situations now (given the distraction that the whole "falling down" thing adds to the situation).

So I don't think I can stay sufficiently undistracted from falling on a person accidentally to inspire their toes to tingle in such a way that they would otherwise like to have someone fall atop them (thus in a choice bit of language manipulation I am less able to inspire such an action as my body somehow finds it to be so much easier of a thing to do!).

The other irony here is that people can talk about a great kiss being one that causes one's toes to tingle, yet if one is kind of bored in a relationship a common expression to use is that they are "putting our feet to sleep" which is kind of like the way flammable and inflammable mean the same thing. :-)

As for me, I am kind of retired from all of this now, most of the time. But I can think back to that ideal kissing height....

This post brought to you by ঁ (U+0981, aka BENGALI SIGN CANDRABINDU, who, while sitting a bit above the letters that it is attached to and thus feeling quite topical, wanted a second sponsorship opportunity on this follow-up post since the first time was so much more accidentally relevant!)

If you are a regular reader, you may recall either Cutting the cord while someone else is shoring it up or Cutting the cord, revisited -- and documenting how to get the job done!, or maybe even both. Between them they pretty clearly show how the Text Services Framework has really been supplanting the former glory of

Anyway, in a comment to that second post, John Daintree noted another problem related to LoadKeyboardLayout:

Hi Michael,

Interesting stuff, thanks for talking through this.  What I'm seeing on Vista is that if I call ActivateKeyboardLayout (or LoadKeyboardLayout), to activate an already installed layout, "at some time later" the keyboard reverts back to the default.

What seems to happen is that next time I get to the Message Queue something in msctf.dll (Text Services I guess) switches the layout back (C stack is attached).  Do you have any ideas about what could be causing this?

What I've done here is put a breakpoint on a Window proc when it gets a WM_INPUTLANGCHANGE message.

The culprit seems to be:

msctf.dll!PostInputLangRequest()  + 0xc9 bytes

Thanks for your continued input (pardon the pun),

/john

> dyalog.exe!Session_wnd_proc(HWND__ * hWnd=0x002608fc, unsigned int msg=0x00000051, unsigned int wParam=0x00000000, long lParam=0x08090809)  Line 1879 C
user32.dll!_InternalCallWinProc@20()  + 0x23 bytes
user32.dll!_UserCallWinProcCheckWow@32()  + 0xb3 bytes
user32.dll!_CallWindowProcAorW@24()  + 0x51 bytes
user32.dll!_CallWindowProcW@20()  + 0x1b bytes
dyalog.exe!OwnerProc(HWND__ * hWnd=0x002608fc, unsigned int msg=0x00000051, unsigned int wParam=0x00000000, long lParam=0x08090809)  Line 456 C
user32.dll!_InternalCallWinProc@20()  + 0x23 bytes
user32.dll!_UserCallWinProcCheckWow@32()  + 0xb3 bytes
user32.dll!_DispatchClientMessage@20()  + 0x4b bytes
user32.dll!___fnDWORD@4()  + 0x24 bytes
ntdll.dll!_KiUserCallbackDispatcher@12()  + 0x2e bytes
user32.dll!_NtUserActivateKeyboardLayout@8()  + 0xc bytes
msctf.dll!PostInputLangRequest()  + 0xc9 bytes
msctf.dll!SyncActivateAssemblyByAsm()  + 0x1b914 bytes
msctf.dll!SyncActivateAssembly()  + 0x76 bytes
msctf.dll!ActivateAssemblyPostCleanupCallback()  + 0x22 bytes
msctf.dll!CCleanupShared::~CCleanupShared()  + 0x2a bytes
msctf.dll!CCleanupShared::`scalar deleting destructor'()  + 0xd bytes
msctf.dll!CThreadInputMgr::_CleanupContextsWorker()  + 0x69 bytes
msctf.dll!CThreadInputMgr::_CleanupContexts()  + 0x38 bytes
msctf.dll!ActivateAssembly()  + 0xa4 bytes
msctf.dll!SetFocusDIMForAssembly()  + 0x5e bytes
msctf.dll!CThreadInputMgr::_SetFocus()  + 0x133 bytes
msctf.dll!SetDIMFocus()  + 0x55 bytes
msctf.dll!_GetMsgHook()  + 0x4afd bytes
msctf.dll!_TF_Notify@12()  + 0x91 bytes
user32.dll!_CtfHookProcWorker@16()  + 0x21 bytes
user32.dll!_CallHookWithSEH@16()  + 0x21 bytes
user32.dll!___fnHkINLPMSG@4()  + 0x25 bytes
ntdll.dll!_KiUserCallbackDispatcher@12()  + 0x2e bytes
user32.dll!_NtUserGetMessage@16()  + 0xc bytes
user32.dll!_GetMessageW@16()  + 0x2b bytes
dyalog.exe!w3_next_message()  Line 848 + 0x10 bytes C
dyalog.exe!MessageLoop(int (void)* readfn=0x00676ed0)

Now my prior posts I had been mostly focusing on how the Text Services Framework and its changes to the input stack and (let's be frank) the input model affected the loading of Input Method Editors, but John's issue kind of points out a situation where the state that USER32/win32k keeps may not always be consulted by TSF (I guess it thinks that it knows better?).

Looking at the stack, I am not so eager to blame that msctf.dll!PostInputLangRequest call, though.

That function has a purpose, and it appears to be succeeding at it, The blame would have to lie with whatever decided that getting this function involved was the answer!

It appears that there is some component (maybe an input thread?) in TSF that is getting recycled and as a part of its shutdown it seems to be making sure that USER32/win32k have their story right -- so it is setting the input language.

Clearly this looks like a bug, especially since in this case a clearly documented member of the Win32 API and the lower level input stack has the correct notion of what is going on, and TSF really has no excuse to be changing it.

And I don't disagree with the assessment that this is a bug.

But more than that, it seems like those last two posts and this one are successively more encroaching warnings to stop relying on LoadKeyboardLayout and all of the lower level stuff. The higher level TSF component and the model that it provides atop that older model just does not seem quite compatible with the notion of making changes out from under it.

How many problems do we have to run across before we move on to TSF? Will we delay the migration until we finally get after a future version of Window inspires an MSKB article with a title like LoadKeyboardLayout will physically harm the user or something like that?

When will all of us (including me) finally get the hint? :-)

This post brought to you by । (U+0964, aka DEVANAGARI DANDA)

I have Lhermitte's sign.

It is named for the French neurologist Jean Lhermitte, who actually has quire a few different medically interesting doodads bearing his name:

• Lhermitte's sign
• Lhermitte's peduncular hallucinosis
• Lhermitte's syndrome
• Lhermitte-Comil-Quesnel syndrome
• Lhermitte-Duclos syndrome
• Lhermitte-Lévy syndrome
• Lhermitte-McAlpine syndrome
• Lhermitte-Trelles syndrome

You can look at that Wikipedia article about him if you want to see the descriptions of them.

The weirdest one for me is the first one (aka Barber Chair syndrome), which is described as:

...an electrical sensation that runs down the back and into the limbs, and is produced by bending the neck forward. The sign suggests a lesion of the dorsal columns of the cervical cord or of the caudal medulla.

Although it is called a sign, it really is a symptom (it is based on a patient's description rather than something a doctor could independently note) but the name has been around for a shade under 100 years so it is unlikely to change.

It is one of my first symptoms of multiple sclerosis, and to be honest one that I remember from several years before my initial diagnosis.

And it is still with me even today, though only when I cause it by bending my neck forward. Usually for me it only runs down to my fingertips though during acute exacerbations (both genuine and inspired by subclinical illness) I sometimes feel it down my legs as well. It kind of screws up the whole "making a person's toes tingle" notion for me, as it is kind of like my waking state, and if I kiss a person of typical ideal kissing height for me while standing up then I will feel tingling in my limbs independent of how good or bad the kiss is.

Though I recommend leaning me against something or I will fall since I usually close my eyes when I kiss, which causes another sign I'll talk about another time....

It is particularly bad at the moment, which is why I am writing about it.

This post brought to you by ঁ (U+0981, aka BENGALI SIGN CANDRABINDU)

The question that came in:

...problems with the Alt Gr Key on the German keyboard layout under Windows 2000/XP but it works under Windows Vista.

Function calls:
    IntPtr vHkl = Win32Api.GetKeyboardLayout(Win32Api.GetWindowThreadProcessId(Win32Api.GetForegroundWindow(), IntPtr.Zero));
    Keys virtualKey = Win32Api.MapVirtualKeyEx(0xe038, 3, vHkl);

Details regarding the function calls above:
*  Win32Api is customer’s own P/Invoke wrapper class around the Win32 API.
*  His current keyboard layout is Swiss German.
*  0xe038 is an extended 2 byte scancode for the Right Alt key (=Alt Gr)
*  3 represents MAPVK_VSC_TO_VK_EX which is supposed to differentiate between left and right keys
*  vHkl is the keyboard layout handle to the foreground window.

Problem:
*  On Vista MapVirtualKey returns Keys.RMenu which is correct.
*  On XP/2000 it returns Keys.None (0) which is wrong.

Is this a known problem?  Is there a work around?

My pre-processing thought was that it was not that the left key and the right key are different per se; it was the difference between the left alt and the right alt (which is also ALTGR, which is also CTRL+ALT). So the easiest workaround is to do CTRL+ALT to get what one wants, as this will work in all platforms.

(Notice how the new feature provides a way for a whole generation of developers who have never heard of CTRL+ALT to find a whole new way to get at the longstanding functionality?)

But the generic issue is still true in regard to new Vista functionality either way (even for some keys other than the ALTGR key).

Regular readers might see what is going on here if they remember an older blog (When you think it couldn't get any harder, it gets easier, where I reference how to do all of this the harder, indirect way like before this new functionality was added in Vista, in Getting all you can out of a keyboard layout, Part #9a).

My first thought (and indeed my first response) was that they should have read the documentation more carefully - this is new functionality.

But you can kind of see the problem here -- although the function is documented as being able to distinguish between right and left keys, it cannot do so when the difference between their scan codes is one of those extended codes (which is listed as a >= Vista functionality) -- the very example that the new functionality was added to address.

This led to my second thought -- that I should report it as a documentation bug to get the issue clarified.

But to be perfectly honest, I have been wracking my brain trying to figure out what should be said to clarify all of this that will not confuse it further for the majority of people.

The issue is so uncommon that it seems scary to try to try to cover it in a place where it is almost never needed.

I'll report it anyway, I think -- a note explaining the case where this is important like the ALTGR case (which itself has an easier workaround anyway)....

This post brought to you by ஃ (U+0b83, aka TAMIL SIGN VISARGA [sic])

It might remind you a bit of the whole Uighur/Uyghur issue (ref: here), but it is not really quite that.

Maybe it is a bit closer to the Farsi/Persian thing (ref here and here).

Or even the Macau/Macao issue (ref: here) 

To be honest there are shades of all of them that might be perhaps be inspired.

But in truth it is different from all of those issues.

You see, for one this is not about a language's preferred name in English, it is about a region.

And for another, there does not seem to be quite the government sponsored push to change things. at all costs with real attempts to change what everyone else is doing.

Well, at least not yet....

Perhaps, now that I am so far into contrasting the issue, it may be an opportune time to take a moment, back up, and describe it, huh? :-)

It is about Bangalore, the city I just left less than 24 hours ago.

The name is part-way through being converted to Bengaluru, which as honest transliterations go is much closer to the original name in Kannada (ಬೆಂಗಳೂರು, aka BA E ANUSVARA GA LLA UU RA U).

Now what is funny is that the government in Karnataka accept the name change idea an although it is not yet decided in terms of timeline, lots of places (like the airport, a lot of the Jet Airways printed material, the ITC Windsor hotel, and other places, have moved to the new name even before it becomes an official request.

Of course one can never be sure what the timeline will be when/if that pronouncement happens, so to be honest it looks like these companies are just figuring to avoid the rush?

Microsoft's various presences in Bangalore have not renamed themselves, but no company really has to do so (yet). I can't help but wonder what the perception is of those who do make the change. What do people in Bangalore think about the name change, the companies that are doing it now, and the companies that are waiting.

I suppose we're lucky there are no city lists in the NLS data; I'd hate to be in that discussion....

 Bad enough that spellcheckers like to take sides here!

This post brought to you by ಬ (U+0cac, KANNADA LETTER BA)

A few times since I posted How would Harry Potter have pronounced शहिवाख़्‍ का दर्पण, anyway?, I have found myself talking with people who know Hindi (fancy that, me being in India and all!) and once in a presentation as well. Each time, I mentioned one additional point about the story that showed how a little knowledge can be a dangerous thing....

And I was encouraged to tell the story here. Thus this post...

You see, although I have a good feel for most of the letters and many of the conjuncts in Hundi, it is hardly a 100% thing.

And not all conjuncts are "easy" to glean ones just by looking at them like ন (U+09a8, BENGALI LETTER NA) + ন (U+09a8, BENGALI LETTER NA) == ন্ন (U+09a8 U+09cd U+09a8, aka NA Hasant NA).

I mean it kind of looks like a NA NA? I mean I know it is not like Bowzer is a member of শন্ন but you know what I mean, right?

And before you ask, the SHA NA NA joke is me winging it right now; I know it is not actually something yiou would pronounce that way and I did not mention that to the people I was talking to in India!

Well in any case, they aren't all that easy, let me tell you.

So anyway, I told the story about how I was frantically trying to take क (U+0915, DEVANAGARI LETTER KA) + ् (U+094d, DEVANAGARI SIGN VIRAMA) + ह (U+0939, DEVANAGARI LETTER HA) in various combinations, with and without ZWJ and ZWNJ, and could not get a KHA conjunct that looked right -- only क्ह or क्‌ह or क्‍ह.

Then I stumbled on ख (U+0916, aka DEVANAGARI LETTER KHA) instead.

There's a letter?

And yes that is much easier. :-)

Oops!

By tomorrow I will be building b/d/p out of left over lowercase L and lowercase O pieces in English, which is almost as silly....

Some of the people I would explain the story to understood what was happening before it got to the punchline, and they were smiling much more quickly. Almost s widely as when I would pronounce Hindi words! :-)

This post brought to you by ख (U+0916, aka DEVANAGARI LETTER KHA)