October, 2008

The title has nothing to do with the blog!

In this crucial time where we need to reduce, renew, reuse, recycle, and in general preserve our precious natural resources, blogs like this one:

Dead Celebrities Made Out Of Fonts

Isn't it great in these trying economic times that we have a way to train fonts to work in this exciting new career? :-)

Just wait until we start getting the other language fonts installed!

(Thanks for the pointer, Zach!)

So it was yesterday morning¹ when Rae Dawn Chong^2,3 put in her facebook status:

Of course there are some interesting language problems here that occurred to me, despite the fact that I knew what she meant, some of which are caused by the underlying definitions of the words involved.

Starting with the obvious, of course....

Since in this country you must be registered in order to vote, one could just as easily make the statement that I did in the title -- there is a certain logical analogue there. :-)

So, just like back in But is it mnemonical?, hints of Mark Liberman's Those who are not authorized are not authorized are there.

Now in order to be registered, there are specific requirements. From Wikipedia's article on the subject:

Under the United States Constitution, states may not restrict voting rights on the basis of race (Fifteenth Amendment) or sex (Nineteenth Amendment). The Twenty-sixth Amendment prohibits the federal government and the states from forbidding any citizen of age 18 or greater to vote simply because of their age.

While the federal government has jurisdiction over federal elections, most election laws are decided at the state level and the true authority to interpret and enforce those laws comes at the local level. Because of this, the administration of elections can vary widely across jurisdictions.

Registering to vote is the responsibility of individuals in the United States. Voters are not automatically registered to vote once they reach the age of 18. Every state except North Dakota requires that citizens who wish to vote be registered.

Traditionally, voters had to register at state offices to vote, but in the mid-1990s efforts were made by the federal government to make registering easier, in an attempt to increase turnout. The National Voter Registration Act of 1993 (the "Motor Voter" law) forced state governments to make the voter registration process easier by providing uniform registration services through drivers' license registration centers, disability centers, schools, libraries, and mail-in registration. Some states allow citizens to register to vote on the same day of the election, known as Election Day Registration. States with same-day registration are exempt from Motor Voter, namely: Idaho, Minnesota, New Hampshire, North Dakota, Wisconsin, and Wyoming.Voters may register at the local election office (which is usually at city or town hall) or, one may call the election department and request a voter registration form through the mail. Voter registration forms may be found at public libraries and registries of motor vehicles. These forms must be filled out and mailed to the local election department. Also, one may register at a voter registration drive. The only states with online voter registration are Arizona and Washington, though legislation has been introduced in other states.

Some states prohibit individuals convicted of a felony from voting, known as felony disenfranchisement. One may register wherever one has an address, regardless of its permanence- for example, a college student living away from home may register to vote in the college's city, even if that is not a permanent address. In most states, one must register, usually 30 days before a given election, in order to vote in it. Six states, Idaho, Maine, Minnesota, New Hampshire, Wisconsin and Wyoming, allow for Election Day Registration.

In some states, when registering to vote, one may declare an affiliation with a political party. This declaration of affiliation does not cost any money, and it is not the same as being a dues-paying member of a party; for example, a party cannot prevent anybody from declaring his or her affiliation with them, but it can refuse requests for full membership. Some states, including Michigan and Virginia do not have party affiliation with registration.

In general elections, a voter may choose to vote for all of a particular party's candidates (straight-ticket voting) or to vote for candidates from different parties for different offices (Party X's candidate for President, Party Y's candidate for Senator, Party Z's candidate for Governor). In a general election, one's political party affiliation does not determine which party's candidates one may vote for.

The obvious deeper issue here that this description makes clear is that the requirements vary -- there are even states⁴ where no registration is required, as well as ones that have not hit a deadline in terms of lack of registration meaning it is too late to vote -- they can register on the very same day!

I happen to live in a state (WA) that I heard in 2006 had something like over 85% of the vote handled by mail, which is the same thing as absentee ballots, though many of them arrive in before election day. I was reading an article last week that claimed that by 2010 WA would be 100% mail-in for its ballots. The registration deadline still exist, though registration (if not already done, as it was for me) can be done online, which of course makes it even easier (though not easier than already done, of course!).

Then of course there is the fact that Rae Dawn Chong (of NH) and I (of WA) are both in Blue states, and thus even though both us are presumably going to vote, our actual votes will not really have influence over the election itself, even if you multiplied each of us by many times in our respective states.

I have a colleague who really does not like Obama much, so he has stated that he doesn't plan to vote for him. But he has the same problem, one step worse -- trapped in a Blue state, the region will carry the person who he really doesn't want to win.

In theory if we wanted our votes to "count" in the sense of having influence in a state that is "in play", all three of us would be sure to reside/be registered in a state like Ohio or somewhere that the results really depend on heavy voter turnout of whoever we want to win. If we brought enough of our friends with us, we might not even mind so much living on the wrong color state. But this is really not such a practical strategy, so one whole I think we are stuck with not mattering....

A largish chunk of my family actually does live in Ohio. In a generally Northern and bluer area (sans Kentucky twang type accent easier to find as one heads towards Columbus and areas further south) though at least one of them is virtually⁵ guaranteed to vote Republican. Which leads to the other question -- does it show one to be more patriotic if they work⁶ harder to make sure their vote counts by living somewhere that it will?

1 - Well, she lives in New Hampshire so technically I think it was afternoon for her?
2 - The actress, and a person whose commentary on life and such I am finding to pretty interesting these days.
3 - Yes, as far as I know, it is actually her. Though there are many profiles on facebook that are not real so the skepticism is understood.
4 - Well, one state.
5 - Though in 2004, the choice was reportedly made to vote Democrat rather than Republican.
6 - After having lived in various parts of Ohio for 12 years of my life, I feel qualified to say that living in Ohio can be hard work, sometimes!

Unicode characters are decidedly non-partisan and sponsoring this post would have been unethical.

So back in Unicode 4.1, Unicode added ஶ (U+0bb6, aka TAMIL LETTER SHA).

Then there was a {ahem} brief delay, after which Vista shipped.

For the record, the Latha font was updated prior to ship.

.

Two points for Microsoft Typography!

Originally I thought that was the point of this blog. Pointing this fact out.

Though this does require one to have >= Vista.

In the words of someone with the nickname shrii on the Microsoft VOLT User's Community:

...I did figure out that ETTamilNew font works fine for sha on XP as also e-tamil OT font - see http://sanskritintamilunicode.blogspot.com/2008/09/which-font-to-choose.html

I also checked the font version on my Vista machine and found out that the versons are different - http://sanskritintamilunicode.blogspot.com/2008/09/on-vista-you-can-use.html

Microsoft marketing people may not agree, but I would strongly suggest that updated versions of indic fonts, uniscribe and richedit, if required, should be made available freely with IE updates so that the Indic script webpages display correctly.

While I agree that it would be nice for everyone to upgrade to Vista or newer OSes, a large proportion of users in India are on older versions of windows and don't/can't update the PC hardware/software as often as users in US.

People would expect that if they download IE8, it would display Indic scripts display correctly, regardless of the platform they are on. If it doesn't but the competition does, then they will switch.

my 2cents :-)

Now this is the argument that Office uses to include updated fonts based on the latest version of Windows, and also the argument they use to carry their own locale support to use on those platforms that do not yet contain the newest locales they support.

So Microsoft does find that argument to be convincing in some cases. :-)

I then thought that might have been the point of this blog.

Unfortunately, the other relevant piece of this is something I described in The fonts are no charge, but the OS they are in has to cost something -- the fact that in many cases, support does come at some price (a pretty small price, all things considered; but a non-zero price).

Perhaps the point of this blog was somewhere in there?

But then if you look at the keyboard included in Vista and at the four conjuncts included in it:

You will notice (after looking at these four images and the rest of the keyboard) that:

• TAMI LETTER SHA is not on the keyboard;
• Specifically on the SHRII conjunct, it is actually more of a SRII than a SHRII;
• There is plenty of room to add whatever is needed in that shift state;
• One could make the argument that the rule about never modifying keyboards may not apply to the SRII --> SHRII update, since the conceptual purpose was ideal rendering of the specific conjunct. You might lose the argument, but it might at least be worthwhile to have the conversation.

I realized that any one of those bullet points might also be the point of the blog.

Then I thought about this whole issue with the keyboard missing the character that was added to Unicode in large part to better represent the SHRII and I realized that the NLS folks had a bug to solve, one way or another. In Vista it is included in the sort so you have no problem using it since it has weight, and it is in font so you have no problem seeing it, but you can't get it there in the first place....

And I decided that was the point of this blog. :-)

This blog brought to you by ஶ (U+0bb6, aka TAMIL LETTER SHA)

Happy Halloween, everybody!

Over on Suzanne Vega's Blog yesterday, in her blog je suis pret!, she describes an interesting phenomenon about language:

One thing that has been a growing phenomenon on this tour is that at some point in the show, usually before Calypso, I say to Gerry¹, “Are you ready?” And he answers “Je suis pret!²” (I am ready!) At the beginning of the tour the audience chuckled appreciatively, but for some reason every night, in each town, the audience carries this one step further than it was the night before. So lately we have had gasps of admiration sweeping through room, and applause, sometimes stamping of feet.  Last night the audience demanded an encore, and I came onstage saying “Encore? Avec plaisir!⁴” The audience cheered, and I thought guiltily, oh now I have squashed Gerry’s moment, but when I said “Gerry, are you ready?” And he said “Je suis pret!”, it seemed to raise the audience to new heights, as they laughed out loud, yelling, then cheered even louder in a kind of roar. Whew. Wonder what will happen tonight?

Now you honestly may need to be a celebrity to get people to cheer about it.

But there are many times when taking the extra time to learn a few phrases in a language can really please people, a lot.

In the past I have often gone so far in presentations in Amsterdan, Japan, Sweden, and France to have bilingual slides, and to try to say a few phrases in the language.

This has the additional fun benefit where I have gotten typically subdued audiences in places like Japan to smile and laugh, given my pronunciation deficiencies. In its own way the effort is still very much appreciated, in a manner that they might not express if my pronunciation was perfect.

I have heard many theories about why this might be the case, and even in the most cynical light (explicit reminded that the natives know the language best) the appreciation is there.

I was reminded of a bit from an episode of the TV show Angel:

GWEN: And if that means we do it the hard way… {She starts to remove her gloves}
GUNN: Uh-uh. Keep your mittens on, sparky.
GUNN: {calls out} Morimoto-san!
{Morimoto looks over and acknowledges him and Gunn bows and greets him in Japanese. Surprised, Morimoto rattles off a rapid-paced Japanese response.}
GUNN: Sorry, didn’t follow that last bit. Shot my entire Japanese vocabulary when I said hello.
MORIMOTO: The gesture is still appreciated.

And it really is. A simple gesture can go a long way here - even if it is just to greet and say good evening.

or to say Je suis prêt....

Now the missing accent is something I'll talk about another time. Stay tuned. :-)

1 - The Gerry she is referring to is Gerry Leonard, who is touring with her right now in Europe.
2 - French for "I'm ready!" ³
3 - Actually it technically should be "Je suis prêt!", I think. But you know we aren't so much with the diacritics here in the US.
4 - French for "With Pleasure!"

This blg broiught to you by ê (U+00ea, aka LATIN SMALL LETTER E WITH CIRCUMFLEX)

I thought regular and semi-regular readers might like an update on life with the IBOT, after I blogged about getting one in Cogito ergo cathedra... (I think, therefore IBOT...) a few days ago.

If you're not interested, then you can leave now. :-)

This may or may not be a series, it depends on how things go, like whether one seems needed.

So I went in ready -- like taking the English instructions to baggage handler types on dealing with the IBOT:

and the Spanish instructions:

printing them both out, then getting them laminated as a nice double-sided sheet -- which I will include with the IBOT, any time I travel.

Who says Kinko's can't be fun? :-)

After several instances of baggage handlers at destinations screwing up the scooter in both large and small ways, I decided it would be best not to take chances with a unit that costs more than 25 times as much as the scooter did!

But anyway, about the transition.

Obviously we are dealing with two entirely different steering systems:

• The scooter has a front wheel that can be turned to the left or right via a bicycle-style handlebar and two levers (one to be pushed by each thumb) -- the right lever makes the unit go forward, the left lever makes it go backwards;
• The IBOT has a joystick with back and forward going back and forward (obviously) and left and right doing zero turning radius turns, counterclockwise and clockwise, respectively.

Now the differences here are fairly obvious, for the most part.

In fact, when going forward, a smooth turn under power in the scooter handles a lot like a smooth diagonal push of the joystick.

Though obviously the combination of the zero turning radius side movement with forward movement can make the joystick handle with a bit more roughness when the movement is not as smooth, something that doesn't happen with the scooter.

Just something that will get better with practice -- the fact that I no longer have to back into elevators and offices is by itself so cool that I don't miss the finer points of scooter navigation. :-)

Going backwards is a bit harder though, since the nature of the movements when using the scooter versus the IBOT, which so closely tie together in forward movement, are pretty much reversed when going backward since the nature of the zero turning radius movement is unchanged.

THUS, going backwards in the scooter while turning the handlebars to the right will move the scooter back to the right, while going backwards in the scooter while pushing the joystick toward the right will actually move the chair to the left.

At some point I will draw this out or find someone else's drawings, as I don't find the above description to be very satisfying, and not everyone will want to take the descriptions a\and work out the physics for themselves. But in the meantime, just imagine combining a clockwise turn with a backwards movement and think about what way the scooter will be pointing.

You might see what I mean. :-)

Or if you have been steering a boat before you will understand this idea. That might be where I get pictures from.

Tonight I am going to head to the Stumbling Monk for a chess night, in the IBOT. In fact by the time you are reading this if you were waiting anxiously for the 12:01 AM post, I will be on my way home, perhaps after having gone backwards twice to get on the bus, which I am really happy that I will be doing at uncrowded times as I fighting over a year of muscle memory when I do it and I'd just as soon have fewer witnesses, if you know what I mean.

I can also talk about hills, living in a city whose downtown isn't exactly San Francisco but isn't exactly the flattest part of Kansas City, Kansas either.

With the scooter, they recommend not more than a 20° incline but to be honest I have done 30° with no trouble. But I have tripped the circuit breaker trying with higher inclines, some of which exist in Seattle. When it happens there is nothing to do but wait 30 seconds, hit the circuit breaker reset, and try again. I have had occasional hills that required doing that 4-5 times while I cursed choosing the wrong street to go on.

But with the IBOT, the incline is not really so much the issue -- it can handle even steeper inclines. Just take the hill head-on rather than at an angle, an even during training around the streets outside Swedish Medical Center I had steeper inclines. That is a change I'm much happier about, as I'm sure you can imagine. :-)

I'll probably talk about all of this later.

For now, I'll move on.

Let's talk about height.

When I graduated from high school in 1988 in Philadelphia PA, I stood tall and proud at 5'8".

A few years later, with the MS I suddenly found standing for long periods of time was harder, yet I did manage to gain a couple more inches so I was able to stand a proud 5'10", when I stood. Perhaps a lare growth spurt, perhaps it was marrying young. The spurt stopped after the divorce, so maybe she was the inspiration. :-)

Eventually, in the scooter, I found myself looking up to those I used to gaze upon at eye level, in my more diminished 4'1". I even tried to change the height to 4'1" at the DMV, though they said no dice. Aparently even if you are paralyzed driving with hand controls they want a real height, I guess measured lying down. Or estimated. It's not like they weighed me.

Anyway, now with the IBOT in balance mode and full verticalk extension I am once again 5'8".

I feel like I just graduated from high school again! In more ways than one. Or something like that.

Well crap, I could talk about being able to hit 5" curbs and climb over them without blinking or not being blocked by steps. And all of that is cool too.

Either way, I swear I'd like to find the original doctor who signed the initial NOT MEDICALLY NECESSARY rejection letter -- flying to him if that is what it takes -- wheel up to him in my IBOT in balance mode, and tell him that human dignity may not be medically necessary.

But health insurance that can't pay for human dignity for those who are sick is stupid, lame-ass health insurance, and it was a pleasure to go over his head to find people who understand that.

After which I'll break my safety rules and "accidentally" run over his foot. :-)

This post brought to you by ♿ (U+267f, a.k.a. WHEELCHAIR SYMBOL) 

In Perhaps we can call them live keys? :-) earlier today, there was an interesting bit of ambiguity I did not mention but probably should have.

Looking at the art again:

 Whether you look at as

I wish the English language had more interesting characters.

or

I wîsh the Énġlišh långuãge hađ mørē iñteŕêßţing çharäčtèrş.

 or even if you aded the color, there are still lots of ways to read/interpret the sentence.

One could interpret it with the emphasis kind of like this:

I wîsh the Énġlišh långuãge hađ mørē iñteŕêßţing çharäčtèrş.

which is to imply that there are some interesting characters now but the problem is just with the quantity.

Or alternately one could instead interpret it kind of this other way:

I wîsh the Énġlišh långuãge hađ mørē iñteŕêßţing çharäčtèrş.

Now if you looked only at Michael Ciancio's site and the text he had associated with the art ("Basically this was a conclusion I came to at the end of my one-year stay in Europe.") then you could in theory read it either way, but given the (for the most part) real lack of any diacritics on letters used in English other the the occasional loan word, and even there it is usually considered optional, the second interpretation seems more likely even without the lack of typographical hints to that effect.

However, if you look at the letters chosen, there is an interesting alternate interpretation, one that adds even perhaps a bit of irony to the statement!

Let's look at them as code points and letters:

0049 0020 0077 00ee 0073 0068

0074 0068 0065 0020 00c9 006e 0121 006c 0069 0161 0068

006c 00e5 006e 0067 0075 00e3 0067 0065

0068 0061 0111 0020 006d 00f8 0072 0113

0069 00f1 0074 0065 0155 00ea 00df 0163 0069 006e 0067

00e7 0068 0061 0072 00e4 010d 0074 00e8 0072 015f 002e

The first thing you might notice is that there is not a single letter that is outside of Basic Latin, Latin-1 Supplement, and a precious few from Latin Extended-A.

Well, that was the first thing I noticed, at least.

I suspect that I might be cooler as a person had I not realized this before I looked at the actual code points. :-)

Perhaps this is meant to be another level of meaning to the sentence? You know, like in contrast to all of the characters that might randomly show up as sponsors on blogs here, the "interesting" characters are really only interesting at the very first level of interestingosity, with a whole bunch of other Lain Extension range characters and lookalike characters from other scripts not being used or even considered.

You know, a meta-comment on how even the word "interesting" doesn't get much of a workout, these days!

Though in truth I know that is probably not it.

If you take Andrew West's comment to the blog, which talks bout the forensic typography examination to identify the font:

It's Gill Sans of some sort. Of course English used to have lots of interesting letters, such as thorn þ, eth ð, wynn ƿ, yogh ȝ, ash æ, long s ſ, r rotunda ꝛ, etc.

And then you look at a copy of Gil Sans in Character Map:

It is clear that the most likely reason for the limited range of potential "interesting" character is the limited range of characters in the font used.

Ah well, I guess the only real irony is the unintended one -- how the font choice so sharply limits the pool of interesting characters such that the example cannot show most of them....

This blog brought to you by ⱡ (U+2c61, aka LATIN SMALL LETTER L WITH DOUBLE BAR, a charcter in the Latin Extended-C block)

The question was an interesting one:

My customer has a tree control with nodes sorted using CString::Compare(). In another part of the application he has the same list of names in a combo box that has the CBS_SORT style set. He wants names to appear in both controls in the same order. Currently they do not. We've tried changing the sort algorithm used in the tree to CString::Collate() and tried a few variations of ::CompareString(), but none of those match the ordering used internally with CBS_SORT set (particularly when names have hyphens or numbers).

The customer does not want to make the combo box owner-draw to override sorting because it will lose accessibility functionality based on this logic:

• Overriding Sort algorithm requires to do “Owner-Drawn” (http://msdn.microsoft.com/library/bb775921.aspx)
• Exposing Owner-Drawn Combo Box Items with Accessibility requires CBS_HASSTRINGS style. (http://msdn.microsoft.com/library/ms695697.aspx).
• Having CBS_HASSTRINGS style causes that the combo box will ONLY use the internal implementation of “Sort” if CBS_SORT is specified, or does not sort the list at all if CBS_SORT is not specified.
• Cause the control to completely ignore our own implementation of sort.

Thus, to achieve the task, he expects to know the algorithm used by CBS_SORT inside Combo Box. Is it allowed? If it’s not, is there any other workarounds that can help the customer?

Many thanks in advance

This question is interesting on multiple levels, actually.

First of all, it is about collation, which is always interesting to me. :-)

Then second it is interesting for the solution, which is of course to take the already sorted list and add it to the Combo Box without the CBS_SORT style, so that it will just be in that order. This way they match and there is nothing else to worry about.

well, other than the fact that the CString:::Compare (actually CStringT::Compare) order is actually the _tcscmp order, which is basically a binary type if order that is not terribly useful to give to users. They'd almost certainly be happier with just about any kind of linguistic result and probably don't need or want to be exposed to an internal ordering that wouldn't be helpful for them.

Think of the precedent we have in file names -- the order which NTFS uses to do sorting internally is nothing to do with the order that Explorer uses to sort them when they are being viewed. That order would just confuse any normal, sane person! :-)

Then of course the question is fascinating for the uestion in the title -- what the hell is the sort order of a Com Box with the CBS_SORT style on it?

The documentation in this case claims:

Automatically sorts strings entered into the list box.

which is not terribly helpful for us, is it?

There is a good hint in the CB_SETLOCALE message documentation:

An application sends a CB_SETLOCALE message to set the current locale of the combo box. If the combo box has the CBS_SORT style and strings are added using CB_ADDSTRING, the locale of a combo box affects how list items are sorted.

Now we're getting somewhere!

If the order is locale-based, it really suggests a CompareString type thing, doesn't it?

Of course StrCmpLogical is CompareString-based (ref: What is up with number sorting?), but neither function's ordering matches the CBS_SORT ordering.

The key is know what is the difference in this case.

The answer was provided by Jay Krell, who I have mentioned before in the same context (tracking down problems!):

Reading the code suggests it is roughly equivalent to CompareStringW(ignore case), except that it treats initial left square brackets differently (seriously)....
 
But if it isn't documented, don't depend on it, of course. (unless you can convince the owners to document it..)

And no, this is unlikely to ever be documented explicitly -- and no, writing about it here doesn't count as documentation! :-)

Probably not worth duplicating, but the logic, which I suspect is quite old, has an illuminating comment at the top of the function that describes the thinking:

* This is a version of lstrcmpi() specifically used for listboxes
* This gives more weight to '[' characters than alpha-numerics;
* The US version of lstrcmpi() and lstrcmp() are similar as far as
* non-alphanumerals are concerned; All non-alphanumerals get sorted
* before alphanumerals; This means that subdirectory strings that start
* with '[' will get sorted before; But we don't want that; So, this
* function takes care of it;

And then as we all know, lstrcmpi is just a nice thin wrapper around CompareString. And there we go.

Not such a bad order, really. Though to be honest it looks like a real corner case to me (if one cares about one symbol -- even negatively -- one probably cares about others, so a simple NORM_IGNORESYMBOL probably would have made more sense in the end. But then that's just me. :-)

The final answer -- you can user CBS_SORT , or choose an ordering you like better. And just go from there....

This blog brought to you by [ (U+005b, aka LEFT SQUARE BRACKET)

Now ordinarily I would apologize to René Descartes for appropriating and re-purposing his quote, but he was not, in the end, all that fond of the particular phrasing of the idea, so....

I have talked about Dean Kamen's IBOT several times in the past, like in 'It's an iBOT,' I would say. 'Yes you bought it, but what is it?' she would answer.

And then back in the beginning of the year in NOT MEDICALLY NECESSARY per, by their own description, non-medical people, I talked about my fledgling first attempt to get an IBOT. after it became clear that this was not just a luxury item -- this was something that many people who are currently in powered wheelchairs or scooters really ought to be allowed to consider.

The news?

Well, as of today, a mere 10 months after the initial denial, I now have an IBOT!

Just got it today after being approved in a pass/fail test of my driving ability. Reminded me of that bit from the movie Sahara:

Rudi: I'm going with you guys, right?
Dirk: No, you gotta tell Sandecker.
Rudi: Hold on, how do I cross the border? I never took survival training.
Dirk: Consider this your course, Rudi.
Al: Kinda pass/fail, which I always thought was easier.

Anyway, I passed (luckily enough!).

Getting back to insurance company adventures, it only had to go through one appeal, which means it was decided at the Premera Blue Cross level (with no help or review by Microsoft, though the fact that it was a Microsoft account might have had some influence, a theory about which I have no supporting data).

The language of the approval letter made it very clear what was on their mind. I won't quote it here except to say that they wanted to make it clear that this was not to be a precedent that could be used to obtain future approvals for others who needed an IBOT -- the focus was on the unique facts reviewed in the course of the appeal.

Fair enough.

But did you ever see the movie The Shawshank Redemption? In particular, this scene:

ANDY: What is all this?
HADLEY: You tell me, fuck-stick! They're addressed to you, every damn one!
{Wiley thrusts an envelope at Andy. Andy just stares at it.}
WILEY: Well, take it.
{Andy takes the envelope, pulls out a letter}
ANDY {reading}: "Dear Mr. Dufresne. In response to your repeated inquiries, the State Senate has allocated the enclosed funds for your library project... "
{stunned, examines check}
ANDY: This is two hundred dollars.
{Wiley grins. Hadley glares at him. The grin vanishes.}
ANDY {reading}: In addition, the Library District has generously responded with a charitable donation of used books and sundries. We trust this will fill your needs. We now consider the matter closed. Please stop sending us letters. Yours truly, the State Comptroller's Office.
{Andy gazes around at the boxes. The riches of the world lay at his feet. His eyes mist with emotion at the sight.}
HADLEY: I want all this cleared out before the warden gets back, I shit you not.
{Hadley exits. Andy touches the boxes like a love-struck man touching a beautiful woman. Wiley grins.}
WILEY: Good for you, Andy.
ANDY: Only took six years.
{pause}
ANDY: From now on, I send two letters a week instead of one.
{Wiley laughs, shakes his head}

So if there are people out there who are in the same situation vis-a-vis Blue Cross and an IBOT denial, the folks at Independence Technology (the subsidiary of Johnson and Johnson that produces the IBOT) now have more information on the cases that may have a better basis for an appeal.

There are other insurance company plans (e,g, Aetna's) that specifically disallow the IBOT as a matter of policy, which is eerily ironic since employees of Independence Technology are actually covered by Aetna. I wonder how they would choose to word THAT denial, and how they turn around and justify the policy to the company that makes the units.

Talk about your very sensitive communications!

Though luckily, if an employee there should actually need one -- not an unreasonable supposition when one considers how medical "miracles" can sometimes tend inspire people to work for the companies that provide them! -- Independence Technology will self-insure for the unit. Insurance company policy be damned! :-)

But anyway, I have an IBOT now.

If you a serf or a Seattle barnights/anti-freeze/chess night/karaoke/relative-o-mine type, you may even see it at some point. I can't give rides to anyone who weighs more than 100 pounds or who is uncomfortable sitting on my lap, but there are rules about people who have never taken the training, sorry....

This post brought to you by ♿ (U+267f, a.k.a. WHEELCHAIR SYMBOL)

Not a blog about VanVelzen, sorry to disappint any fans of his!

Just in case you have been too busy following the presidential race in the USA to notice, Microsoft did just release Silverlight 2.0.

Which is pretty cool in most respects.

One piece of the new version of Silverlight that I think is pretty cool that has not gotten a lot of public attention, I thought I'd talk about now.

Shawn Steele and Poornima Priyadarshini did a presentation at the recent Internationalization and Unicode conference entitled Globalization in Silverlight, which you can see the slides for if you go here on Shawn's blog.

Now if you download the slides and skip to slide 12 (when they started talking about globalization support) you'll see where the support is.

You will see something very interesting, I think! :-)

Remember almost a year ago in When yesterday's workaround becomes tomorrow's potential solution... where I talked about how the "WindowsOnly" culture, the one that does not rely on the .NET Framework's cache of data, can solve many of the interesting problems when .NET and its underlying host differ in results?

Well, Silverlight 2.0 takes the whole idea of "synthetic" globalization support to  whole new level, by simply not carting data around -- it relies completely on the host for the information....

As yet another cool bonus, the download got to be smaller because of this!

Now I don't want to be like mini and treat every example that seems to go in my direction as proof that I'm right -- Microsoft as a company is complex enough that on any given day one could probably find just as much to prove that the company agrees with me as that it does not. This is not mixed messages, it is just a huge number of people we're talking about!

But I am very happy about this direction, and in fact the only thing that is sadly weird about it would be the naming scheme given to the feature ("WindowsOnly") in light of the fact that it really is now not just Windows now. Luckily with all the data coming from the underlying platform that "appendix" is not something we have to pay too much attention to. :-)

I also loved the topic that started on slide 15 and continued for a lot of the rest of the deck, which mentions the differences between the OS X  view on a lot of these matters -- more examples were given in the talk and explaining how Silverlight worked to wrap things in a way to properly balance these two (not always consistent) viewpoints of the world I so frequently occupy. When it was going on I was one of several who was consulted about specific decisions and plans, and the issues were often fascinating.

I'd love to see a presentation that talked about nothing else, as this is a fascinating area. Maybe I'll cover a few of these issues here, under the premise that talk about the stuff I find interesting. :-)

And of course slide 19, which talks about sorting differences, will get my special attention. :-)

Here it is:

I like the principle behind this example myself, because it does show very directly how results coming from these two completely different operating systems might tend to vary.

Though I probably would have been happier if actual code points were also included, and maybe also the code used to return the results.

Looking at the strings themselves, the font did not give me the hints to understand where all of them came from.

I mean, clearly we have some hiragana and some katakana and some kanji and some mixes thereof, and people who don't know Japanese could get a little lost trying to understand the basis for the differences (and even people who know Japanese could have trouble here -- this just makes us all look random, right? <grin implied>).

To be honest, that is actually just me nit-picking about my favorite topic, I doubt most people would care about those kinds of details -- and that presentation would not have been effective had they dug in that way s it would have failed as an overview. Maybe I'll jump in there some time soon a there are some really interesting details here, too.

But anyway, take a look at the presentation as it does highlight the way globalization support happens there.

Sometime soon I'll talk about the next big question that is likely to be on many people's minds after they look at the slides. :-)

This blog brought to you by サチコ (U+30b5 U+30c1 U+30b3, aka KATAKANA LETTER SA, TI, and KO)

Prior posts in the series:

• 0 (You have to start somewhere)
• 1 (Starting with a dictionary simple in every way)
• 2 (Three ways to convert!)
• 3 (Where do keystrokes come in?)
• 4 (Specifying when to modify, if not how)
• 5 (All about the language, baby!)
• 6 (it is sufficient to be iconic, aka DaYi, DaYi Aynu!)
• 7 (Configuration 'junk in the trunk', part 1)
• 8 (Configuration 'junk in the trunk', part 2)
• 9 (Will you be content if I tell you how some content can be defined?)
• 10 ("Even Jove nods," Atlas shrugged, "so we'll convert their old files, now.")
• 11 (The knights who say நீ, redux, #1)
• 12 (The knights who say நீ, redux, #2)
• 13 (Sinhalification proclamation!)
  

Now often the consequences of combining all this information together can get tricky.

I first mentioned the particular problem in Part 1 -- when I pointed out

Unfortunately, you can also select a candidate by typing the number on the left. I say unfortunately given the strange interaction between typing numbers and also typing the number used to identify items in a list.

This is a lesson that was learned the hard way with the Amharic IME in Vista. :-)

Now I have talked about Ethiopic numbers in the past (like in Why that is positively Ethiopic!).

If you just type the numbers e.g. 0123456789, you just get the number you type.

But if you type ', you get into (for lack of an official term) Fidel number mode, defined in TableTextServiceAmharic.txt like:

"0" = "0"
"1" = "1"
"2" = "2"
"3" = "3"
"4" = "4"
"5" = "5"
"6" = "6"
"7" = "7"
"8" = "8"
"9" = "9"

"'1" = "፩"
"'10" = "፲"
"'100" = "፻"
"'1000" = "፲፻"
"'10000" = "፼"

"'2" = "፪"
"'20" = "፳"
"'200" = "፪፻"
"'2000" = "፳፻"
"'20000" = "፪፼"

"'3" = "፫"
"'30" = "፴"
"'300" = "፫፻"
"'3000" = "፴፻"
"'30000" = "፫፼"

"'4" = "፬"
"'40" = "፵"
"'400" = "፬፻"
"'4000" = "፵፻"
"'40000" = "፬፼"

"'5" = "፭"
"'50" = "፶"
"'500" = "፭፻"
"'5000" = "፶፻"
"'50000" = "፭፼"

"'6" = "፮"
"'60" = "፷"
"'600" = "፮፻"
"'6000" = "፷፻"
"'60000" = "፮፼"

"'7" = "፯"
"'70" = "፸"
"'700" = "፯፻"
"'7000" = "፸፻"
"'70000" = "፯፼"

"'8" = "፰"
"'80" = "፹"
"'800" = "፰፻"
"'8000" = "፹፻"
"'80000" = "፰፼"

"'9" = "፱"
"'90" = "፺"
"'900" = "፱፻"
"'9000" = "፺፻"
"'90000" = "፱፼"

which will look like this:

Now although the idea is kinda cool to let you type the European number and have all of the zero forms of the Fidel numbers just show up, the fact that what you are typing and the index numbers for the list are the same thing mean you can't ever type what you are being given the hints for.

So to use the candidate list in that case, you have to scroll to pick the item or click on it or something like that.

Plus, when typing numbers the need to constantly accept candidates is also kind of a pain, and it would in retrospect have been better to consider including all of the numbers rather than just the ones with zeroes in them.

But we were learning; in some ways we still are. :-)

Now the options for dealing with this are limited, but I'll talk about them soon, though in the end the current definition in the Amharic IME has some real limitations....

Now to whether there is a bug in the underlying technology that gives hints that you can't use? I may be a philosopher but code clearly isn't, and the code underneath here is just doing what it is told to. So the underlying bug can mainly be in the definition.

And the moral of the story (and a moral that my friend Cathy could really get behind!) is not to expect too much from numbers!

This blog brought to you by ' (U+0027, aka APOSTROPHE)

Previous blogs in this series of blogs on this Blog:

• Part 0: The intro, sans content
• Part 1: Getting the obvious out of the way
• Part 2: A&P of a 'linguistic character'
• Part 3: It starts with cursor movement (where MS simultaneously gets better and worse)

If you have been following this series, you might be wondering what comes next.

Perhaps it would be about selection and all of the interesting issues there (especially the weirdnesses with selecting partial elements in some cases and more interestingly the way we would think it weird if partial elements were not selected in other cases. It may be impossible to separate our intuitive expectations coming fresh into computers from our intuitive behavior based on generations of typewriters and then computers.

But it is really kind of old hat, minus an issue or two like the ones summarized in More on cursor support: the rest of the answer and the earlier blogs it references, plus a blog or two where I mention how weird it is to select half a surrogate pair or half a composite (decomposed) character -- like the cases I mention in More on cursor movement.

And when you get down to it these issues really are a natural extension of the ones involving cursor movement -- since selection is often just moving the cursor with shift key held down (and has top behave the same way even when it's not).

Or I could go down the road of how the more destructive operations (e.g. the BACKSPACE key and the DELETE key) come into play here, though they are two just natural extensions of cursor movement and selection (the one major difference being the exception I describe in BACKSPACE vs. DELETE and What do you get when you combine a base character with a buttload of diacritics? and really I think the behavior difference between BACKSPACE and DELETE is pretty sensible, and pretty defensible.

If you want to get into the one that is harder to suss out, I would say it is probably easier to get confused about the difference between the BACKSPACE key and the BACK ARROW key. The fact that these two do not behave the same way is explainable but may be even less intuitive....

And of course none of it should happen for surrogate pairs in any case -- thy should always be whole units and never split out, as the blogs above mention.

But in the end what is left to talk about?

The main thing that is left to describe in this series is explaining what support a product should add to a product if it is Unicode but thoroughly "USC-2" based using the definitions in this series, and one wants to move it up to be "UTF-16" based.

In other words, the obvious remaining practical question is what work needs to be done?

And that is a very good question.

Now for my example I will use a product that simultaneously

1. much more UCS-2 based than any product ought to be, and
2. knows much more about this whole idea of clusters of characters than most products ever might,

And that product type is databases.

The first criteria is met by the fact that they have a firm base in the middle of allocation issues -- whether it is column lengths or whatever -- and also "character" based parameters/syntax in SQL.

And the second is met by the fact that if you look at the categories from earlier in this series the second and third categories (and if you include SQL Server 2008, the first category as well) is well handled by the collation support used by the engine for almost all of its comparison operations, even the ones it should not!

This schizophrenic behavior is one I have mentioned in the past in blogs like Wild[card] thing, You make my CHAR sing and the follow-up to it, with the other side of the equation somewhere in the realm of blogs like the Freudianly-themed Sometimes a WCHAR really *is* just a character.....

Perhaps with With SQL Server (and SQL itself) comes the illogic of 'trailing spaces' (and the myth of fixed width) thrown in for good measure....

But here and now we run into a bigger problem --to do a full job here, up to three things would be needed:

• For functions and such that currently take one Unicode character, a way to pass more than one would be needed -- somewhat like the way .NET took the Char.IsLetter method and its one IsLetter(Char) signature and added an IsLetter(String, Int32) overload;
• For things like wildcards (as discussed in Wild[card] thing, You make my CHAR sing), a new wildcard would be needed to specify as linguistic character; something like the StringInfo class would be needed.
  
• For things like linguistic string lengths
  

Now obviously these three items while being simple enough conceptually could lead to a ton of actual work, so how much would actually be needed would have to be triaged.

And ideally, if one can get away with just extending existing support, that would be much better (as would handling all three categories of linguistic character) rather than just some if them).

Next time, I'll talk about some of the triage rules.

This blog brought to you by ດ (U+0e94, aka LAO LETTER DO)

(This is a post about Limonata)

or, to use the newer can style in my art:

In the middle of last year when I mentioned There is no more Limonata at Trader Joe's and the fact that Trader Joe's had chosen to stop carrying San Pellegrino Limonata.

Anyway, the situation is getting worse again.

Larry's in Redmond has been out of it for several weeks now. And they have no word on when they will have more -- they are on the verge of taking the shelf space away, apparently.

I negotiated a temporary discount at Whole Foods while I look into a more permanent episodic discount at Costco or Sams or Whole Foods or somewhere.

If all else fails, I guess I'll have to move to Italy.

Anyone in Italia looking for someone who knows a lot about globalization and drinks Limonata like it was water except he enjoys it more? :-)

To Larry's, I say: FEH. Perhaps it is because I do so much with an operating system, but I need a much more reliable uptime....

This post brought top you by ف (U+0641, aka ARABIC LETTER FEH)

Previous blogs in this series of blogs on this Blog:

• Part 0: The intro, sans content
• Part 1: Getting the obvious out of the way
• Part 2: A&P of a 'linguistic character'
• Part 3: It starts with cursor movement (where MS simultaneously gets better and worse)
• Part 4: Talking about the ask
  

Now going through the previous parts, there are two conclusions that developers and architects can come to:

1. The basic problem is much simpler that it seemed at first;
2. The full problem is not only more complicated than it seemed, but there is currently no full, query-able support for it on the platform.

It was ever about UCS-2; it was never about UTF-16. It was about character identity, and the desire to not destroy meaning....

But it is important to look at the full scope of the kinds of things that users might expect here.

The other day a mail thread involving the "delete" aspects of the problem inherent in how to safely truncate a string talked about several aspects of the problem.

In that thread, Peter Constable suggested:

SCRIPT_LOGATTR::fCharStop returned by Uniscribe’s ScriptBreak() API ought to tell you valid truncation points – including not to break between surrogate pairs, between a base character and a combining mark, or within Indic clusters.

This gives a practical suggestion to handle many of the problems pointed out earlier, problems that encompass the first two of the three major categories of linguistic characters I defined earlier in the series.

Peter then went on to mention:

One case that won’t tell you, though: In Thai and Lao, it’s best not to break between {U+0E40..U+0E44, U+0EC0..U+0EC4} and a following consonant letter {U+0E01..U+0E2E, U+0EC1..U+0EAE}.

Now these two cases actually also happen to fall under the third category which I named as sort elements, since these are the kinds of things that would also tend to impact collation operations.

Though the fact that a "collation dude" like me would call them sort elements while a "font dude" like Peter would not suggests that perhaps a name that encompasses a wider description of what these items would be might be in order. Otherwise we are each being partially functionally descriptive but not in any kind of robust, complete way.

In this case the result is the same, and plus there is no function to get all of the information, so naming it might be less important at the moment. :-)

There are also another aspect of truncation that was discussed in the thread, such as:

Also additional support for Arabic [could be] added since when breaking in the middle of a word, the character shape would change from the medial or initial form to the final form... ...inject a ZWJ to keep the form from changing so the user has context that additional text follows and wouldn’t change the meaning/context.

Note that this is a new idea that has not been discussed before in this series, or indeed this blog. I am still wrapping my had around the validity of the notion since to a native reader of the language the different forms really are the same letter, and I am not convinced that the confusion of truncating in the middle of a word would be mitigated appropriately by such an approach.

Another interesting issues related to the Thai case that Peter mentioned was that while a user might occasionally expect cursor movement and/or selection to respect those category-three sort element boundaries, they would for the most part actually expect the cursor to be able to put in the middle of the sort element, and most might never have such an expectation.

This is similar to the way (to use the earlier example for sort elements) a Hungarian user might expect dzs to be handled.

At this point it may be a challenge to find a user who is sophisticated enough to ask about expectations who does not have decades of experiences with typewriters and/or computers that have already built up expectations of proper behavior. and for every case where a user might expect multiple characters to be treated as one for movement/selection/deletion due to their experiences with dead keys, there can be just as many cases where the fact that the underlying entry/storage mechanism has multiple characters does not even allow for the possibility of treating the cluster as a single character.

Thus we have a design flaw in the platform where the underlying entry/storage mechanisms are guiding "expected" behavior, rather than the other way around.

Then we can look at yet another aspect of the problem: the way .NET does things.

You know how I quoted Peter Constable's ScriptBreak comments earlier?

This is an idea I mentioned before, in blogs like Stick a fork in GetCharacterPlacement and Sometimes you need more than StringInfo.

Now that blog prophetically talks about how StringInfo can't do it all here (in fact it only handles most of those first two categories of linguistic character boundaries without giving any of the word boundary information).

Add to it the fact that all of the methods that used to take just a System.Char -- for example Char.IsLetter(Char) -- now have a second overload that takes a string and a length -- for example Char.IsLetter(String, Int32). These have only one purpose -- handling surrogate pair handling. Thus all of the other times when a user might think of it as a character or even Unicode would think it is a grapheme cluster or a character via canonical equivalence are ignored.

And this addition of dozens of overloads to .NET >= 2.0 is a partial bandage over a perceived limitation -- the inability of functions to handle surrogate pairs properly -- that ignores all of the other scenarios here.

And .NET has no built-in support for the kind of things ScriptBreak can tell you, which are largely based in what Unicode and its character properties reveal (even though lots of the underlying properties are available via the CharUnicodeInfo class).

I guess this is what happens when the typography and rendering folks are not involved in design discussions that the core NLS folks work on. :-)

But you can see how lots of the pieces are available here, and can be used. Note that if in truncation you use the ScriptBreak-provided word break opportunities for your truncation then you will be able to support all three levels of linguistic character since none of then would survive a word-breaking opportunity!

This blog brought to you by එ (U+0d91, aka SINHALA LETTER EYANNA)

Regular readers, as you might imagine, tend to come and go.

Sometimes, like in the case of people like Suzanne E. McCarthy, they seem to drop off the face of the planet and you never hear from them again. Even their Blogs are inactive.

Since indeed Suzanne's went inactive after she was talking about Myanmar, it must be allowed that it is technically possible that militant Burmese abducted her from her home in Vancouver, though under the theory of the "when you hear hoof beats except horses not zebras" theory of forming conclusions, such possibilities rarely deserve the kind of attention that the first half of this very sentence, in its own reckless and self-serving way, has chosen to do.

And other times, they are still around!

Take for example regular reader Jan Kučera, who just asked over in the Suggestion Box:

One simple question I'm afraid I know the answer to.... If you have more languages installed, but you use some of them only rarely, I would like to select those which will be in the "Left Alt+Shift cycle", and keep the rest only for explicit selection or custom keyboard shortcuts... any way to do that?

Thanks! Jan

Unfortunately, there is no direct way to force only a subset of the installed keyboard layouts to be used in the LEFT ALT+SHIFT cycle.

One could, if one were drinking with geeks, hypothesize turning off LEFT ALT+SHIFT handling, adding a global hotkey, and attaching to it a handler that would use the configured list.

And then, if one were the kind of geek to take those kinds of ideas and try them out (thought experiment solutions almost begging to actually achieve the state of implementation and all), one could try to make it possible.

Though these days I am not quite that kind of geek, if you know what I mean. It takes a certain kind of commitment that I just don't have.

Not that I'm not still involved in the thought experiment type situations and scenarios.

But it is like the difference between bacon and eggs.

With bacon and eggs, the pig is committed; the chicken is merely involved....

Sorry Jan! Though you did say you kind of knew the answer going in. :-)

This blog brought to you by 豕 (U+2f97, aka KANGXI RADICAL PIG)

Over in the Suggestion Box, Gene Sorensen asked:

Dear Michael,

Thank you for an incredibly helpful tool in the MSKLC, and for your helpful blog. I have two questions related to the MSKLC:

1. Will the MSKLC install into 64-bit Vista? We are using it in 32-bit Vista, and I know it creates keyboards for 64-bit environments, but will the tool itself install and work in 64-bit Vista? (We need to buy a new computer.)

2. When installed into 32-bit XP MSKLC warns we will not have access to all features unless we install it into Vista. I cannot find any features missing except access to the new languages and keyboards available in Vista. Are there any other features we are "missing" by using it in XP?

Thanks again,

Gene

For the first question, eve the old version of MSKLC (version 1.3) which could only build x86 keyboard layout DLLs, could be installed and run on 64-bit Windows. and the new version (1.4) can install on 64-bit and build 64-bit keyboard layouts as well. So there should be no problems there at all, whether you author keyboard layouts (a) on 64-bit, (b) for 64-bit, or (c) both on and for 64-bit, you will be just fine with the latest version of MSKLC -- version 1.4.

For the second question, the warning will indeed show up in these cases:

and it will show up every time unless you both check the Don't show me this again checkbox and don't uncheck the View|Options Remember Settings After Shutdown checkbox (if you don't remember settings then MSKLC won't remember the first checkbox!).

Actually, this feature existed in the earlier version of MSKLC as well -- if you run it on Windows 2000 you'll get the warning about how if you don't run it on >= XP you won't get all the features....

Now the new platform (anything >= Vista) gives you several advantages:

1. As you mentioned, they provide a larger list of keyboards in the File|Load Existing Keyboard list;
2. They have a larger number of fonts installed for the support of different languages and scripts;
3. They have a later version of Uniscribe and GDI+ on them, which will again support a larger number of languages an scripts;
4. The Locale Builder tool, one of two ways to get custom cultures on your machine, only will install its locale on >= Vista, and these items are included in the locale list o MSKLC;
   
5. The helpful screen shots in the included CHM file showing how to use MSKLC 1.4 were mostly done using Vista with glass so they will look more like what you are using.

But if you look at this list you will notice that all of these capabilities (which can make your job easier in authoring layouts) are optional and you can definitely proceed without them if you like.

There is even a help topic that talks about how you can developer keyboard layouts even if you can't see the characters, by using the Unicode code points (it also shows how you can explicitly choose some other font you have installed that will then help the characters show up properly!).

You can even use the tool to download later versions of the Unicode standard to get a fuller list of character names, any time you want -- and that can be done on any platform, as long as you have an Internet connection....

But none of these features are required and I have authored many a layout even on Windows 2000 and then later installed it on Server 2008 with no problems.

So you miss features, sure. But not ones that have to block you. :-)

This blog brought to you by ሴ (U+1234, aka ETHIOPIC SYLLABLE SEE)