June, 2006

(I suspect that there will be nothing technical in this post)

I am going to ramble a bit so if that sort of thing bothers you then go ahead and skip this post. :-)

My Italian is less than perfect, though I believe that the title of this post roughly translates to "When god wishes to punish us, our prayers are answered."

I like to keep this in mind sometimes. It keeps me from being tempted to pray for anything....

It was probably a little under three decades that I first "got" MS, and it has been a little over a decade and half since the first stab at a diagnosis took place. And the way I have felt about the disease has certainly changed quite a bit over that time. But some things have stayed pretty consistent.

For example, it generally does not depress me. Maybe it just my cynical nature (one of my really good friends in high school, she used to point what a cynical bastard I was, even back then), but as diseases go there are lots that can be a lot worse. I have had friends pass away from accidents, a lover pass away from a brain tumor, and I have a couple of friends now who have terminal illnesses. So I figure who the frak am I to get depressed about my lot in life, right?

But there are other things that have changed over the years.

Like exacerbations.

At first I hated them, since they were a bit more involved than some odd symptoms, and could sometimes hit me hard. They were a psychological reminder that the occasional weird feelings were not like some sort of high without drugs, they were a disease.

Then there was a period of time where my ex called them exasturbations and I never had the heart to correct her. For one thing, it is never good to make the person you are dating feel foolish, and for another having her tell me that it was exasturbation was a normal part of my life? It just made me smile each time and (immature geek that I am) it never got old -- I would sometimes not mind getting an execerbation because it would mean I'd have to bite my tongue to keep from smiling when she looked at me concerned and asked if I was having an exasturbation....

I think I waited a few years before I actually said anything. I probably ought to grow up a bit, but I still laugh when I think about it.

As things stand now now, when I sit somewhere in between relapsing/remitting-secondary progressive and primary progressive M.S. where I don't know whether to blame the lack of exacerbations on the fact that I am doing well four doses into a Novantrone course or that I have finally moved into what the Novantrone docs like to call worsening M.S., a stupid name given the fact that the one thing all forms of M.S. do is worsen to some degree.

Anyway, before each Novantrone dose the folks in the hospital always gave me a Zofran pill for nausea, though I hadn't actually ever felt nauseous. My neurologist even gave me a prescription for it which I had never used since I just didn't feel like throwing up.

Though this last time, I was feeling a little nauseous, so I was taking the Zofran. Three times a day. I always waited until I was feeling a little nautious again. I actually had to refill the prescription that I was just a month away from having the pills expire.

I began wondering if one of the side effects of Zofran was nausea? I meen, its not as funny as saliva causing stomach cancer (only when it is taken slowly over a long period of time!), but still it would be kind of ironic, in the Katie sense, if you know what I mean.

And I was feeling some disequilibrium again. My long time M.S. symptom. I ended up calling my neurologist between the two problems, and she called my back pretty fast (I think it's because I don't complain much, so anything I call about must be serious?), and we decided that they really were unlikely be to related. The nausea would hopefully go away soon, and after I described it at greater length, that the disequilibrium may be an actual exacerbation.

Somehow it made me feel good. The knowledge that I was perhaops still in that secondary progressive category. Where the uncertainty of not getting one was somehow worse now. Where it was better to know the M.S. was out there, in the background somewhere.

I haven't been praying in the past to not get exacerbations, and I don't pray now to get them. But I think now about how if I had been doing so, and if she had been listening, and if she decided to answer the prayers, that it would really have been a punishment of sorts. I want it out of my hands so I can react to it without fear that my desires have actually influenced the course of the disease.

I had a friend a few years back who I would visit in Orem, Utah. I once went with her to hear one of the quorum of the twelve apostles of the LDS chuch speak, and I remember him talking about how he used to find himself praying for little things like finding a good parking spot and then saying a prayer to give thanks afterwards. I remember my friend saying that she understood why she was having such a hard time getting her prayers answered -- God was too busy finding Jeffrey R. Holland his parking spaces!

Of course she was joking. But I remember thinking later about how disturbing it would be to feel like prayers were answered that immediately -- especially in light of that Italian proverb in the title of this post.

It is probably better for me that there is some distance, I think. Especially because anything I'd ask for in relation to my disease would almost certainly be worse for me -- there is never a time when an exacerbation is a good thing, truly there isn't. But if I never have another one again then technically I would have primary progressive M.S., which isn't all that good either from a morale standpoint. So either of them can be a punishment, and it might be good to keep the whole thing out of the realm of prayer, all things considered.

Perhaps this is not very logical. Sorry about that, it seemed much clearer in my head than it looks now that I am reading this post.

I'll post about something interesting tomorrow, I promise. :-)

Yesterday when I was talking about WYSIWYG font dropdowns and using the GetGlyphIndices function to determine if a font supported a character or set of characters, regular reader Mihai mentioned in a comment:

GetGlyphIndices does not handle chars outside BMP (true in XP SP2 and an older Vista build)

This is true, it doesn't. As a function it looks at each individual UTF-16 code unit and checks to see if it is in the font.

Now in theory you could create a font that puts surrogate code points in the ligature table that might work here, but (a) that won't help GetGlyphIndices since it will just globally say the high surrogate and low surrogate are there and not whether there is an actual ligature defined, and (b) it is not generally considered good typographic practice to create an "off the BMP" font that way.

(We don't mind it so much in keyboards, but that's a story for another day!)

There is actually a real solution that can handle things off the Basic Multilingual Plane -- the ScriptGetCMap function, which is described as being able to retrieve "...the glyph indexes of the Unicode characters in a string according to either the TrueType cmap table or the standard cmap table implemented for old style fonts."

And it will do the extra work with supplementary characters defined as surrogate pairs, and it is even easier than GetGlyphIndices in terms of determining whether the font supports the string since it will simply return S_FALSE if one or more of the code points were mapped to the default glyph.

There is one warning in the documentation which is theoretically troubling:

Note that some code points can be rendered by a combination of glyphs as well as by a single glyph — for example, 00C9; LATIN CAPITAL LETTER E WITH ACUTE. In this case, if the font supports the capital E glyph and the acute glyph but not a single glyph for 00C9, ScriptGetCMap will show 00C9 is unsupported.

Although in practice the situation is very uncommon, because in general any time a particular composite sequence is suppprted in a font, the precomposed character (if it exists) is also supported. It is just the way things usually work in fonts, and it is much less common for the precomposed character to not be supported.

Though there is info in the docs on how to handle that situation if one suspects it may be happening, anyway, via the ScriptShape function....

Now I was talking to Peter about this whole issue yesterday, and he pointed out that people could simply special case their code to use GetGlyphIndices for BMP cases and ScriptGetCMap for when things are off the BMP (in truth GDI handles nothing off the BMP anyway, so it is hardly an artificial split -- if you are using supplementary characters, you are using complex scripts as far as GDI is concerned).

Though in truth if one is going to use Uniscribe here, it seems using it all the time when it is available is probably better, or at least more consistent. And why not use Uniscribe in little ways explicitly if it is going to be use implicitly in bigger ways like rendering anyway? :-)

This post brought to you by ফ (U+09ab, a.k.a. BENGALI LETTER PHA)

So, the question that Bill asked was clear:

I have a WYSIWYG font dropdown in my toolbar. Some international fonts (like Estrangelo Edessa) show up as squares, apparently because they don’t support all Unicode ranges. I need to detect these fonts and display them in the WYSIWYG list using Tahoma (like Office does). What’s the best solution?

My app is a managed app but I’m assuming I’ll need to use native interop to solve this problem.

Thanks,

Bill

The list he is referring to is like the one you see in Word's font dropdown:

Or  even in Outlook's mail message dropdown:

So, the real question here is even simpler: "How can one determine if a font supports the characters in a string?"

The easiest way is via the GetGlyphIndices function, which has in the initial description:

The GetGlyphIndices function translates a string into an array of glyph indices. The function can be used to determine whether a glyph exists in a font.

So you can pass the name you were going to use to the function (making sure to pass the GGI_MARK_NONEXISTING_GLYPHS flag), and then look at the glyph indices for the 0xffff marker.

If it shows up for any of the characters, then you know you should use some other font.

Now the Office dropdown has a few more tricks up its sleeve that I scrolled to avoid, so I should probably mention them now....

If you look, some of the fonts have some additional stuff next to the names:

All of the Hebrew fonts have the first eight letters of the Hebrew alphabet (U+05d0 to U+05d7), and all of the symbol fonts have a sampling of the symbols. And this extra feature is done even when there is a fallback done for the font name itself (due to the letters in the name not being supported).

Incidentally, Arabic fonts do the same sort of thing that the Hebrew fonts do....

Now it is easy enough to detect a symbol font, but figuring out that a font is specifically a Hebrew or an Arabic font (e.g. Miriam Fixed or Arabic Transparent) rather than just a font that contains Hebrew or Arabic (e.g. Arial Unicode MS) is a bit more complicated.

I have no idea how they actually do it -- perhaps they are calling the GetFontUnicodeRanges function, perhaps they are grabbing the FONTSIGNATURE-like info that is inside the font and using it to find out. Though of course this is the kind of feature that one could quickly go wild with in showing the characters of many different language-specific fonts, from Tibetan to Sinhalese.

Paradoxically, the only "boring" fonts on the list would be ones that support multiple scripts!

At some point, there are also performance concerns, plus there are people who find the list somewhat visually distracting (not to mention the list as it exists in Word has an obvious bias for the myriad of ways one might display the Latin script!)....

This post brought to you by ܨ (U+0728, a.k.a. SYRIAC LETTER SADHE)
(A character that is well-supported by Estrangelo Edessa!)

Peter Constable asked some Unicode folks:

And Ken Whistler stepped up with a good historical look at these two characters (which in my humble opinion deserves a more permanent location for others to see!):

0F71;TIBETAN VOWEL SIGN AA;Mn;129;NSM;;;;;N;;;;;

0F76;TIBETAN VOWEL SIGN VOCALIC R;Mn;0;NSM;0FB2 0F80;;;;N;;;;;
0F77;TIBETAN VOWEL SIGN VOCALIC RR;Mn;0;NSM;<compat> 0FB2 0F81;;;;N;;;;;
0F78;TIBETAN VOWEL SIGN VOCALIC L;Mn;0;NSM;0FB3 0F80;;;;N;;;;;
0F79;TIBETAN VOWEL SIGN VOCALIC LL;Mn;0;NSM;<compat> 0FB3 0F81;;;;N;;;;;
0F80;TIBETAN VOWEL SIGN REVERSED I;Mn;130;NSM;;;;;N;;;;;
0F81;TIBETAN VOWEL SIGN REVERSED II;Mn;0;NSM;0F71 0F80;;;;N;;;;;

0FB2;TIBETAN SUBJOINED LETTER RA;Mn;0;NSM;;;;;N;;*;;;
0FB3;TIBETAN SUBJOINED LETTER LA;Mn;0;NSM;;;;;N;;;;;

 
                NFD             NFC
0F76            0FB2 0F80       0FB2 0F80
0F77            0F77            0F77            <-- discouraged (strongly)
0FB2 0F71 0F80  0FB2 0F71 0F80  0FB2 0F71 0F80  <-- preferred
0F78            0FB3 0F80       0FB3 0F80
0F79            0F79            0F79            <-- discouraged (strongly)
0FB3 0F71 0F80  0FB3 0F71 0F80  0FB3 0F71 0F80  <-- preferred
0F80            0F80            0F80
0F81            0F71 0F80       0F71 0F80       <-- discouraged
0F71 0F80       0F71 0F80       0F71 0F80       <-- preferred

Note that the preferred forms appear in both NFD and NFC, with the decomposed form for 0F81 resulting from the non-starter exclusion and the decomposed forms for 0F76 and 0F78 resulting from explicit addition to the script-specific composition exclusions.

If you gave 0F77 and 0F79 *canonical* decompositions, then:

0F77 --> <0FB2, 0F81> --> <0FB2, 0F71, 0F80>
  0         0     0          0    129   130
 
0F79 --> <0FB3, 0F81> --> <0FB3, 0F71, 0F80>
  0         0     0          0    129   130

                NFD             NFC
0F76            0FB2 0F80       0FB2 0F80
0F77            0FB2 0F71 0F80  ????            <-- discouraged (strongly)
0FB2 0F71 0F80  0FB2 0F71 0F80  0FB2 0F71 0F80  <-- preferred
0F78            0FB3 0F80       0FB3 0F80
0F79            0FB3 0F71 0F80  ????            <-- discouraged (strongly)
0FB3 0F71 0F80  0FB3 0F71 0F80  0FB3 0F71 0F80  <-- preferred
0F80            0F80            0F80
0F81            0F71 0F80       0F71 0F80       <-- discouraged
0F71 0F80       0F71 0F80       0F71 0F80       <-- preferred

Now you've made your life more difficult and normalization implementations maybe more complex. The decompositions <0FB2, 0F71, 0F80> have to be prevented from recomposing. They won't  decompose partwise, because <0F71, 0F80> is blocked from recomposing, and <0FB2, 0F80> is also blocked from recomposing, but the sequence of 3 has, at least in principle, a target it should recompose to, unless blocked. Depending on how you set up your tables, you might or might not get this right, and in any case, you end up introducing the strongly discouraged characters as a source of valid sequences that you have to contend with in NFC and NFD, whereas under the current scheme you don't.

Also, this was all part of a very head-breaking set of problems for Tibetan when decompositions and canonical combining classes were being reviewed for the introduction of normalization in the first place.

In Unicode 2.0, 0F77 and 0F79 *were* given canonical decompositions, but they were *different* decompositions, to wit:

0F77 = 0F76 + 0F71 = 0FB2 + 0F80 + 0F71
0F79 = 0F78 + 0F71 = 0FB3 + 0F80 + 0F71

*and* they had funky fixed position class assignments, as well:

0F77 = 0F76 + 0F71 = 0FB2 + 0F80 + 0F71  (not in canonical order)
 135    134    129     6     143    129
 
0F79 = 0F78 + 0F71 = 0FB3 + 0F80 + 0F71  (not in canonical order)
 137    136    129     6     143    129
 
That was clearly hosed, as it broke all kinds of rules that we were trying to establish for normalization, including ensuring that all decomposition mappings produced sequences in canonical order and ensuring, as much as was possible, given the constraints in place, that the resulting sequences would follow the logic of the script *and* that NFC forms would decompose if that was what the users of the script preferred (hence the introduction of script-specific composition exclusions for several scripts, including Tibetan).

During that conversion from Unicode 2.0 to Unicode 3.0 with normalization, the UTC did the best it could with the mess for Tibetan. It was clear after the analysis that 0F77 and 0F79 should never have been encoded at all -- which was why they got those "strongly discouraged" labels -- but there was nothing to do about that mistake at that point. The compatibility decompositions
were the best compromise to keep them from contaminating the normalization processing of the rest of the Tibetan vowels.

--Ken

Anyway, like I said, this seemed to me like good historical information to put out there, and certainly to help show that Every character has a story!

This post brought to you by     ཷ and     ཹ (U+0f77 and U+0f79, a.k.a. TIBETAN VOWEL SIGN VOCALIC RR and TIBETAN VOWEL SIGN VOCALIC LL)

Earlier today when I talked about Keyboards under LUA, i was very focused on installing the keyboard layouts that MSKLC creates under Vista.

And I did talk a bit about what might be a Vista bug.

But I didn't talk at all about Microsoft Keyboard Layout Creator itself. And it too has a bug that affects Vista most prominently (but can affect earlier versions of Windows, too).

Let's say that you go to the MSKLC download page and download MSKLC.EXE. When you run the program, it pops up a nice dialog that clearly suggests wht you might want to do:

Unfortunately, if you hit the Setup button, you will then get another dialog:

Huh? Doesn't Vista include the .NET Framework? What the hell is this dialog talking about, anyway?

The problem here, which is actually a bug in the MSKLC setup, is that it is looking for the 1.0 or 1.1 version of the .NET Framework specifically. It is not thinking that the 2.0 version is the right one.

In my defense, I was using the built-in stuff in Visual Studio setup projects to build the dependency. Maybe I forgot a setting or something. :-)

In any case, if you click Yes on that dialog, you will be taken to the SDKs, Redistributables, & Service Packs page of the .NET Framework Developer Center. This too is all of the built in stuff from Visual Studio setups, which I have to admit is fairly cool.

Unfortunately, that link contains only 2.0 versions of the .NET Framework for download. And if that were good enough, then you wouldn't have had the problem in the first place!

Many people actually try installing that 2.0 downloiad, only to be told that it is already installed. And now they are really confused....

(By the way, that seems like a bug in the page content to me; if Visual Studio 2003's setup projects create links to a download page, then that page ought to at least have a link to that version's .NET Framework!)

In any case, the original page that comes up after you download MSKLC includes a What Others are Downloading list. Today that list looks like:

Others who downloaded Microsoft Keyboard Layout Creator (MSKLC) Version 1.3.4073 also downloaded:

1. Microsoft .NET Framework Version 2.0 Redistributable Package (x86)
2. Microsoft .NET Framework Version 1.1 Redistributable Package
3. MDAC Utility: Component Checker
4. Microsoft .NET Framework 1.1 Service Pack 1
5. IntelliType Pro 5.5 Keyboard Software for Windows

Note item #2 on that list (the fact that it is #2 is because of that bug in the redist page content I mentioned above) and the fact that even if the 1.1 download were on the page there is no clear indication that it is what you need.

Hopefully this blog post will help turn it around a little bit? :-)

Interestingly enough, after you install the 1.1 .NET Framework and install MSKLC, you can go to Add or Remove ProgramsPrograms and Features in the Control Panel and uninstall the 1.1 .NET Framework. But Microsoft Keyboard Layout Creator will still run just fine. It is just a silly, setup-time restriction that has nothing to do with the actual functionality of MSKLC....

This post brought to you by ෛ (U+0dd8, a.k.a. SINHALA VOWEL SIGN KOMBU DEKA)

People have been noticing that attempting install an MSKLC-generated keyboard layout on Vista has some problems. Say you take your handy dandy custom Bulgarian keyboard created by MSKLC:

You try to double click on the Bulg.msi file, and you may or may not get the LUA dialog:

However, whether you get it or not, and no matter which option you choose if you do, the final result is unfortunately the same. :-(

Now it may or may not be a bug that if you are running a recent enough build to get the LUA dialog that clicking "Allow" doesn't allow the install to proceed. We'll figure that out later.

For now, the easy workaround!

Just run from an elevated command prompt, by right clicking on the Comand Prompt option on the Start Menu and choosing to Run as administrator:

After saying Continue when that User Account Control dialog comes up:

you will then have a command prompt that you can run Bulg.msi from, and it will install just fine!

Don't you love happy endings?

Now LUA (Least-privileged User Account) or UAC (User Account Control) is serving an important purpose here -- to make sure that you are okay with the changes this installer tries to make to your system -- a file in your system32 directory, and registry keys under KEY_LOCAL_MACHINE. So it is a pretty invasive little bugger, an MSKLC generated keyboard layout install. The key is making sure that you give it the right permissions to get its work done. :-)

Now the other question, the one I tabled earlier -- whether, if you say "Allow" to a LUA dialog should it still fail? I assume this would be a bug (the same thing happens if I run msiexec /i Bulg.msi from an unelevated command prompt). Probably someone ought to fix that. :-)

This post brought to you by K (U+004b, a.k.a. LATIN CAPITAL LETTER K)

You know the drill, if you have seen any of these other previous posts: