June, 2010

That's right, now available, it's the Urdu Langauge Interface Pack for Windows 7!

Available for 32-bit only, and the system must have English fallback resources. You can download it from right here.

The Urdu Windows 7 LIP is produced as part of the Local Language Program sponsored by Public Sector.

And now a few facts about Urdu:

NUMBER OF SPEAKERS:

~60 million native speakers, ~40 milion second language speakers

NAME IN THE LANGUAGE ITSELF:

 اردو 

Urdu is the national language of Pakistan. Although English (the second official language) is widely used in elite circles and Punjabi has more native speakers, Urdu is promoted and expected by the administration to eventually prevail in Pakistan. It is taught as a compulsory language up to high school and has started becoming the lingua franca.

From a linguistic perspective, Urdu and Hindi are two standard forms of one language rather than two different languages (This phenomenon is referred to as a "diasystem" and known from language groups like Persian/Dari/Tajik, Bosnian/Croatian/Serbian (formerly known as Serbo-Croatian), Danish/Norwegian or Malay/Indonesian. The issue is, of course, politically highly charged, and speakers of these languages normally consider them as distinct languages). Urdu has been borrowing more from Persian, while Hindi has been using Sanskrit to extend its vocabulary, and both are written in different scripts.

FUN FACTS:

• There are large numbers of Urdu speakers outside of South Asia in countries with Urdu workers or immigrants: The United States, Saudi Arabia and the United Kingdom each have between 350,000 and 400,000 speakers.
• The language name Urdu is related to the English word horde: Urdu started to develop under the Muslim courts which ruled South Asia from the early 13th century. The rulers, the Moghuls, originally spoke Turkish. Their word ordu, meaning tent or army, was eventually used for the language of the army - which became modern Urdu.
• Standard Hindi and Urdu are nearly identical in grammar and share a basic common vocabulary but differ in literary conventions and specialised vocabulary with Urdu retaining strong Persian, Arabic and Turkic influences, and Hindi relying heavily on Sanskrit.
• Before the Partition of British India, the terms Hindustani, Urdu and Hindi were synonymous; all covered what would be called Urdu and Hindi today.
• Urdu is one of the 22 scheduled languages on India and one of the official languages in five of the states within India: Uttar Pradesh, Bihar, Andhra Pradesh, Chattisgarh and the national capital (New Delhi).
• Urdu is one of the three of the major languages used in the disputed region of Jammu and Kashmir -- the others being Kashmiri and Dogri.

CLASSIFICATION:

Urdu is an Indo-European language.

SCRIPT:

Urdu is written with Perso-Arabic, an extended Persian script (which is an extended Arabic alphabet).

Microsoft-specific:

Although Urdu is generally expected to be rendered with the Nastaliq calligraphy style of the Perso-Arabic script, the development of properly validated OpenType Nastaliq fonts continues to be a challenge (yet remains high on the list of items in Microsoft's lists of things it wants!).

You can find out more about the Urdu language here.

Enjoy!

So, the other day I was asked a strange question about behavior.

In sorting, not my personal behavior!

As I have mentioned, most people forget that I haven’t owned the collation code in Windows for years now. Though I suppose the fact that I keep answering the questions anyway kind of means it’s my fault….

In my personal behavior I get people questioning it all the time (the most recent one was just days prior to this blog being posted) but that isn’t something I am blogging about now. Besides, that was just flirting and she started it (plus she suggested the complainer join the current century so I’m probably okay on this one).

Anyway, back to the topic.

It had to do with the fact that in Kazakh the following pairs of letters were not being considered equal in a case-insensitive comparison:

Ё (U+0401, aka CYRILLIC CAPITAL LETTER IO)
ё (U+0451, aka CYRILLIC SMALL LETTER IO)

and

Е (U+0415, aka CYRILLIC CAPITAL LETTER IE)
е (U+0435, aka CYRILLIC SMALL LETTER IE)

Intrigued, I looked into it a bit.

And I found the following Kazakh exception table entries, used in every version of Windows since XP:

0x0435  16  25   2   2 ;Cyrillic Small Ie (The small versions of Ie come after the capital as unique AW)
0x04bd  16  25   5   2 ;Cyrillic Small Ie Hook
0x0451  16  25  19   2 ;Cyrillic Small Io
0x04bf  16  25  23   2 ;Cyrillic Small Ie Hook Ogonek

with the following relevant default table entries (overridden entries marked appropriately):

0x0435  16  24   2   2 ;Cyrillic Small Ie
0x0415  16  24   2  18 ;Cyrillic Capital Ie
0x04bd  16  24   5   2 ;Cyrillic Small Ie Hook
0x04bc  16  24   5  18 ;Cyrillic Capital Ie Hook
0x0451  16  24  19   2 ;Cyrillic Small Io
0x0401  16  24  19  18 ;Cyrillic Capital Io
0x04bf  16  24  23   2 ;Cyrillic Small Ie Hook Ogonek
0x04be  16  24  23  18 ;Cyrillic Capital Ie Hook Ogonek

Note the clear case pair relationship in the default table that is essentially removed in the Kazakh exception table, by making each of the four lowercase letters a single new letter with a unique alphabetic weight (interestingly ignoring the secondary/diacritic weight – NORM_IGNORENONSPACE – will cause all four lowercase characters to be treated as equal and all four uppercase characters to be treated as equal!).

In any case, the data and the comment explains the behavior that was being seen, though not the reason for it.

I mean

(The small versions of Ie come after the capital as unique AW)

is a strange comment by pretty much most definitions of strangeness if you ask me, since it does not really tell you the source of the information or its veracity, its reliability. How to tell why such a thing would be expected without the source becomes rather complicated (I have been unable to verify it as of yet, though I have only looked for the same amount of time that a journalist claims someone was not available for comment and I never believe they tried hard enough so I probably haven’t tried hard enough yet either!

I often talk about how CASE != COLLATION (or CASE <> COLLATION for you VB types!) but it is usually in the other direction; these EIGHT characters may be the only four case pairs whose relationship exists in the casing table but not in the collation table (only for Kazakh). If it is really true maybe there should be a Kazakh-specific entry in the linguistic casing tables too!

The entries exist in the RTM version of XP and later, which will of course narrow down the “who” but that won’t really help either. I mean like from 10 years ago? Who are we kidding?

But is it inaccurate? Or is there really something in Kazakh that has been intending to be doing this all along – as farfetched as that may seem?

I mean no one has complained up until now in the last ten years, sure. But how often would the problem really get noticed if it is in fact a problem?

So, any Kazakh speakers have light to shed here on whether there is a reason for this?

I am probably going to be writing blogs on various topics related to things I learned at the World Classical Tamil Conference and the co-located Tamil Internet 2010 for a while now. I just learned so much, it will take a while to get through it all.

So think of the as the first of many different blogs on the topic.

This one is more about me than future ones will be. :-)

First of all, regular reader Jan Kučera should text me ASAP at the phone number I mailed to your matfyz.cz address because I foolishly did not collect a phone number and want to not miss you in Bangalore!

Okay, so on to some observations.

It is really really cool when Ministers and the Chief Minister know your name, shake your hand, and thank you for what you have been doing (and then saying enough about the things that it is clear that mean it!). This may seem like an obvious point but I had never really thought about it before, and certainly have never even met anyone as important before (I technically had met one of them before but no such interaction took place then).

I am really humbled by so much of that – the importance that language has for them and everyone I talked to, and their genuine pleasure about so many of the things that have been happening for Tamil.

It wasn’t that I didn’t think I was working on something important – I did. But I guess I never expected really important people to think so.

When they were on stage and I was sitting in the front row (where I was asked to sit as a “Special Invitee” (சிறப்பு ௮ழைப்பாளா்), thanks to Venkat, the INFITT chairman!), they’d periodically look at me and smile. I think I smiled back, hopefully no too goofily...

I was surprised how much of their words I followed, but then I’ve been in the space for a while.

The “Celebrity” aspects of my visit actually started sooner than that -- from when I first landed in Coimbatore and a பொன்னாடை (ponnadai) was put around my shoulders. I can’t really describe it and I only learned the word because I embarrassingly asked Muthu what it was – I hope I spelled it right. Basically it is a shawl but when given to someone of importance up on their arrival its a பொன்னாடை. Here it is later, in my room:

The look on my face was probably much better than after I saw myself in the mirror. So unworthy!

And being a சிறப்பு ௮ழைப்பாளா் had its burdens, I’ll admit. I had a security guard and an attendant not too far from me at all times, and a driver on call at all times even though he was needed usually only two times a day (and one day, four times) at most. It is weird having people do things for me that I’d rather do myself, but even weirder when I see how my insistence that I could do it was taken as not trusting them, not thinking them worthy. Sri suggested I just let them do what they wanted to do, and I finally gave in and did so, mostly. I think they liked the “celebrity” aspect of their protectee/subject a little bit, too

I don’t know if my security was armed or not, but he was never required to brandish it. He may have been there to protect others from me running over their feet, or something.

It was no trouble at all staying interested and engaged with the plethora of adults and children and young adults who kept walking up to me, so interested in me and why I was there and how the chair balanced. It was a lot of fun and amusing when someone would come up to me right after and tell me that the last group of kids were the Hon. Minister’s and I would be glad that I wasn’t getting tired or cynical about it!

Their enthusiasm actually kept me going the whole time. I came back to the hotel exhausted but while at the conference I never felt it.

The volunteers were also amazing and I felt bad that so many of them were terrified of my accent, and I will forever be grateful to Sri Lakshmi for being the one they always came to, and with a graceful smile she’d unfailingly have the answer to the question. After the first day I would test that understanding and tease her a little with stern faces coupled with phrases like “you are continuing to fail to displease me” which would at first horrify her and the when she parsed it again would realize what the words were actually saying would smile.

I have almost none of the pictures of myself and people who wanted pictures other than the ones I saw in the newspapers, but if anyone sends me links they may become Facebook profile pictures in the future. Hundreds of them were taken, maybe almost a thousand.

And to the young girls who would come up, pose for a picture with me that a friend was taking and then run skitter without so much as introducing themselves, I will forever be a little bit baffled, but this week I learned that you don’t need to have a pale face like mine to be recognized for blushing!

And so many noticed when I shaved for Saturday (when the keynote and the main presentation were both happening). Everyone approved, and several people even wanted to take pictures again.

I am going to need to learn Tamil now, for real. It is all fun and cute to be a recognized expert in something one knows nothing about. But after ten years it is a little embarrassing. It is time to move past வணக்கம் and நன்றி and being able to spot my name (மைக்கேல் எஸ். கப்லன் or மைக்கேல் எஸ். கப்லான், as both were there!) in the program so I would know where to go…

And that song, the one that some of the popular Tamil singers worked on? It is still stuck in my head, but I don’t mind. I still love it. And I want to be able to know what all the lyrics mean!

Anyway, there is more but I just wanted to get this first blog out there. :-)

In the spirit of BUG SPOTTING answers, today's blog will talk about the problems in Bugs hidden in plain sight, and commented that way too. :-)

There are several problems here, in fact.

Here is the email and comment inside a code snippet for your perusal if you don't want to click to look:

The current console output behavior for <redacted> and <redacted> was painfully established through years of international testing.
That’s not to say there are no issues – just that we need to be cautious about making changes.

// if redirected to a pipe or a file, don't use WriteConsole;
// it drops redirected output on the floor
// if going to a file, we should not use console codepage, we
// should use ANSI codepage OR write unicode & a unicode
// filemarker at beginning of file.
// Since we don't know if this is the first thing in the file,
// let's go the ANSI route.
WideCharToMultiByte(GetACP())
WriteFile(hStdOut)

The reminder here is that this is from code that actually has shipped before in Windows.

Yes that part needs to be fixed too. :-)

Now it is hard to know what kind of "international testing" was done here, though one thing is for certain: the tester either never strayed outside of ASCII, never tested with chcp (which would try to change the console's code page and have limited success given the conversion this code will do), never tried both in the console/redirection given the unusual differences the code would cause, or perhaps didn't do anything once it was working for ASCII and the "redirection does nothing" bug was reported and the fix was verified.

But let's give then credit and assume, like I did in Conventional wisdom is retarded, aka What the @#%&* is _O_U16TEXT? and The real problem(s) with all of these console "fallback" discussions and Anyone who says the console can't do Unicode isn't as smart as they think they are and Cunningly conquering communicated console caveats. Comprende, mon Capitán?, that the console vs. redirection case was being detected and this block is inside code that detects that we are talking about redirection.

After all, once I didn't know how to do it right, either...

That makes the first two lines of the comment piece correct!

The next three lines don't give their justification; that change just makes it harder for test because now they have two different code paths to test. But a code comment does not a spec make so the lack of info may be okay here.

That last part is the problem.

That is, if whack has gone back to meaning something bad again.

Because I mean this bit:

// Since we don't know if this is the first thing in the file,
// let's go the ANSI route.

It is just plain wrong.

First of all, what's all this about not knowing if this is the first thing in the file? We can know that exactly!

A simple call to GetFileSizeEx will tell you that, immediately! Just pass in that stdout pointer that you have already determined is a redirected file, and then you will know by the size of the file if they redirected with a > or a >>.

Second of all, you can find out the encoding, too. A quick call to GetFinalPathNameByHandle to get the path and you can look at the contents and see what is in there and make the appropriate decision (if you need it to be pre-Vista you cab use code like this).

What's so hard about that?

Remember that the file is guaranteed to be opened to you or else all of your write operations would fail. So you are the one person with access (that's how you were able to get the size in the aforementioned First of all.

And third of all, and this also goes to algorithm flaws too, what makes a person think that if choosing between

• appending Unicode data to a non-Unicode file, or
• appending non-Unicode data to a Unicode file

that either one of those options does not completely suck?!?

Either way a bunch of your file will be unreadable.

So if you assume both of those options suck, and you are trying to ship a quality tool, then you have two choices. Either

• use the above method I mentioned in Second of all to get the actual answer of what the encoding is, or
• make a smart decision based on te assumption that if they used >> they are appending to a file that either you wrote in an earlier incarnation or that an equally smart developer's tool wrote. Either way you would have written Unicode if you didn't want to fail to support the languages that Windows does!

In the end, the comment, and the subsequent code that implements it, is dead wrong and should be fixed to not make such misstatements since people reading such a comment and thinking they learned something new about Wndows are probably the reason why this code was wrong in the first place. Inacurate information perpetuates!

With all of these console blogs, jncluding this one, I have shown you the way so that if you want to produce a tool that will do its best to fully support all languages, that you can. That can really be worth the effort, after a decade of beng just plain wrong and broken (which pretty much everything has been up until now)....

So.

It was several weeks ago, in BUG SPOTTING: Identify 1) what the code does, and 2) what they wanted it to do, that I challenged people to explain something about some code, as indicated by the title.

This code:

whcar_t* newParams = new wchar[MAX_PATH + MAX_PATH]; whcar_t* p = newParams; while (*params) {
    *p++ = *params;
    params = ::CharNext(params); 
}
*p = 0;
return newParams; 

in fact.

I actually cheated a little here. :-)

When I suggested The torrents of breaking CharNext/CharPrev as a refresher what I didn't say is that the bug in some code that caused Raymond Chen to in passing notice this other bug in NLS code looked almost identical to the current bug here. :-)

In essence the intent of this code was to be some kind of really smart allocating wmemcpy or wcscpy in C++.

But unfortunately, it was being so clever in its iteration that it had to use CharNextW rather than ++ for iterating the string, even though it clearly thought they were the same thing.

And they are not, whether there is a bug like the one in The torrents of breaking CharNext/CharPrev or not!

The reasoning for the separate function is unclear though, as is the need to allocate something of size MAX_PATH + MAX_PATH.

That is either twice the path you need or not even close to the amount of path you need, depending on who is using the function.

And none of which would perform better than an inlined wmemcpy of a function anyway. :-)

It was just over 17 months ago in the blog titled My name is Michael Kaplan and I work for Корпорация Майкрософт that I described the very interesting exception to the rule that Microsoft has throughout the world where the company's name is left in English even if everything around it is translated.

The exception? Well, Microsoft is transliterated to Майкрософт (CYRILLIC EM A I KA ER O ES O EF TE, and yes the Unicode names for Cyrillic letters are an upcoming topic as well!).

And there was one comment in the blog from Azarien that pointed out something else interesting here:

...it's only Microsoft which gets transliterated, the Corporation part is translated. [emphasis mine]

And indeed, that is interesting (since in so many other languages (though perhaps not all) the full Microsoft Corporation is kept in English. I think they kinda snuck this one past the lawyers all those years ago, maybe. But from my point of view lawyers speak Greek, not Russian! :-)

Anyway, I wanted to highlight that point, as an interesting issue was bandied about the other day.

I mean Russian, which uses Майкрософт (Microsoft to you English speaking types, by which I mean us English speaking types), has many Language Interface Packs that use the Cyrillic script.

Like Ukrainian. And Kazakh. And Tajik. And so on.

They can all have Russian as their base language. And English I believe....

And now for the questions, about the cat that is placed among the pigeons by all this:

1) Should these other languages transliterate Microsoft into Cyrillic as well, or should it be left in English?

2) If it is transliterated and the target language has slightly different transliteration conventions, should the slightly different conventions be followed?:

3) Should they translate the word Corporation akin to the way Russian does, or should it be transliterated, or should it be left in English?

4) Remembering that LIPs do not localize more than the top level UI, that means there will be many instances of the company name that may be in either English or Russian on one of these LIPs as the UI language. Knowing that the words in the target language can't dynamically change depending on the base language, does this change the answer to questions one, two, or three?

5) This has happened already in some previous LIPs too. So should each language just do whatever it did last time, whatever that may be?

6) Should they try and get metics (if they exist) on the most common base language and go with that one?

It seems like no matter what is done, there is going to be some inconsistencies here, for at least some of the time.

But which inconsistency is most reasonable and...well...consistent, do you think?

And over in the Suggestion Box again, this time it is CFynn asking:

Suggested Topic: Getting MSKLC working properly on Windows 7!

MSKLC 1.4 installs and runs - but when I load an existing keyboard, all I get is a blank screen in the middle (no keyboard layout displayed).

Searching around for a solution it appears others have the same problem - but I cannot find a solution.

I wonder, is CFynn Chris Fynn? :-)

Well, either way, this looks like A picture that can't be easily described with words all over again, and clearly it still can't be described any easier with words.

It isn't just a Windows 7 issue, or even just an MSKLC 1.4 issue (it has been repro'd on XP with 1.3), in fact.

I'll add the picture again:

Now once again the problem is a known WinForms issue that as far as I know has never been fixed.

This bug was found for MSKLC during the 1.4 beta in our very last RC.

I coded a fix for it, and we tested the fix.

And then we had a meeting.

There had to be a very serious conversation about whether this was a recall class bug.

We knew it could be worked around by futzing with DPI settings, we knew that it was rare (half a million downloads, just a few reports of the problem).

In the end we decided not to hold up the release.

In retrospect that decision was probably a mistake (given that the MSKLC project was moved to another team that has never shipped an updated version and as far as I know does not have such a release scheduled -- they have a whole bunch of other work they are doing and none of the scenarios I listed in my internal-eyes-only vision document about v.Next MSKLC and v.Next MSLB are priorities for them).

So we should have just fixed the bug.

But no one knew it was going to be moved and buried, so it isn't like anyone knew how it would turn out....

Anyway, to work around the problem, just futz with the per-application and overall DPI settings, and eventually it will work.

I wish I had a better answer; this is the one serious problem that really remains in MSKLC that I think a fix is truly needed for (there are a few other less common issues with specific layout problems that aren't MSKLC bugs so much as KBDUTOOL/KBDTOOL bugs that no one wants to touch for stability reasons).

About a week ago, I found out that one of the presentations they want me to do in Coimbatore at the World Classical Tamil Conference‎ is a keynote.

Quite an honor, I accepted....

I realized that I needed a new presentation for that.

Something shorter, with fewer slides.

With a few extra days in Chennai at my disposal, I talked to the hotel duty manager (who was always unfailingly polite and who I almost felt bad for not needing more since she was always so willing to be helpful) about whether there was someone I could talk to if I wanted a few of my slides translated to Tamil.

Really just two slides with a lot on them and a few with just a word or two.

Under 100 words in English, total....

She asked me to send the text and she would see what she could do (although a fluent Tamil speaker, she like many working at the hotel are actually from the north and as a group they speak Tamil quite well but their reading and writing is not a good).

There ended up being a two-part process.

First she got translations from a contact she had who admitted to being baffled by some of the content though did their best to do it.

Let me leave no doubt here; it was a quality translation.

Then, she introduced me to another duty manager who was a native Tamil speaker. He sat with me in my room for the two hours before checkout time and asked me questions and then the translation ended up being heavily reworked.

We used the Tamil version of the Microsoft Indic Language Input Tool, and although there were a few points where the transliteration he was most used to had to be adjusted (once adjustments were made it was -- with just 1-2 exceptions -- the first candidate on every word), in short order we had a much better quality translation based on an actual understanding of the content.

Let me leave no doubt here; it was a quality localization.

Something I learned: it was also really wonderful to see the Indic Language Input Tool able to be so quickly used by someone who had never either seen it nor heard of it. He was also able to quickly make adjustments as he saw the way it was generating candidates (I suspect he would have preferred to be able to modify the tool's scheme rather than modify his intuitive transliteration instincts as a native speaker, though he never complained about it!).

As someone who I suspect spends more time in other languages these days, it was a fascinating field test of the tool and definitely akin to a more complete and formal positive experience than my own with Bengali.

Something else I learned:

Tamil localized text takes up more space than German localized text.

We usually recommend that dialogs should not be overpacked and that for any technology that doesn't autosize the dialogs (like Win32) developers should leave room for expansion -- about 30-40%.

Tamil seems to need more in many cases.

Although I had heard the claim made once by a tester a few months ago, seeing it firsthand makes me want to look at some of the LIP content and try to get formal numbers on the percentages.

I don't know, maybe Tamil would be a great pilot language in some cases!

Now there are deeper lessons here related to the issues that both the original translator and the Le Meridien "localizer" had with trying to move the original text from English to Tamil that I will also talk about.

But that will have to wait for some future blog. For now I have to see what I can do to try to capture some of what I learned in the original presentation I'm doing (and the one for Unicode!), because I feel like I understand the viewpoint a lot better than I did a week ago.

A monh ago.

A year ago.

And ten years ago, when I first talked with Dr. Om Vikas at a Unicode Technical Committee meeting about what Tamil was looking for out of Unicode, and not really getting.

How little we all knew then; how much we can know tomorrow....

Running through my head right now is that Lionel Richie song "Hello", which has no connection for me other than some long ago boyfriend of my sister's like ~25 years ago; it was their song. But somehow HELLo would be better than Hello for that song, and not just because I have Steve Dallasian/Bill the Catsian feelings about Mr. Richie. Man, why would anyone date anyone with that as the song?

"I don't have to talk to you about...scuba diving... so... that'll save some time." - Emo Phillips

Being right about something when people have a different (conflicting) plan they are set on can be a terrible burden, sometimes.

I mean at the time they just write you off as a crotchety old man, a crank who is more interested in theoretical backcompat issues than trying to solve actual customer-felt problems (I had an ex who, before meeting me, assumed I was in my 50s based on how people in such situations at work descibed me).

Then later on when it turns out you were right, you're that guy.

The one who says "I told you so".

Or even worse, the one who doesn't but everyone knows you'd be within rights to do so. Can it get worse?

Of course it can. Involve a blog somehow. When there is a blog involved...such people can be truly insufferable.

Use your imagination this one. Or just trust me.

Now sometimes it's a theoretical exercise -- like the small number of developers or testers who like to brag at conferences that they did not sign off on Vista, even though the features covered by the signoff passed.

I've seen 'em do it.

The bragging rights about their nullified signoff vote really doesn't matter all that much in the scheme of things. And in at least one case I know of it wasn't actually true. Which is like making up what you were doing when Kennedy was shot so you have a better story at parties.

I wasn't alive yet, which I figure gives me an alibi, at least.

Anyhow....

Other times, the impact is more marked on projects that are out there, that are almost about to ship, etc.

Like the problem I discussed in I know I'll Never say Never... again, at least with the change in the casing table with ς, aka U+03c2 (GREEK SMALL LETTER FINAL SIGMA).

As predicted, nothing happened. A cow made a satisfied grunt somewhere after chewing its cud, and everyone just kind of walked slowly away.

And, as also predicted, someone ran into problems.

They are going to work around it (I'll explain how in a moment!), though one amusing note: the guy I'm helping here is someone who I helped a few years back on another problem (that time was to do with Korean). He remembered me as being very helpful last time too, which is always nice. In fact that is the best part of having a good reputation with someone: they are impressed in advance. :-)

Anyway, if you are having problems with the GREEK SMALL LETTER FINAL SIGMA in Windows 7 then of course there are workarounds (like replacing it with σ (U+03c3, a.k.a. GREEK SMALL LETTER SIGMA) before doing whatever operation was causing problems.

Or just avoiding the naming issue by not creating files in Windows 7 that would be considered duplicates on downlevel platforms.

Or don't uppercase prior to building whatever index or storage token you have (since even without this bug there are always new characters out there).

Or I could just stop being right; that would save a lot of time, too. For everyone, including me....

Between that and skipping the scuba diving conversation, I have time to get out and enjoy Chennai!

Blazingly non-technical, you know the drill....

Today (June 21st) is a day that I will probably always remember.

It was because of June 22, 1985.

I'll explain....

It was one of those days at the Beechmont Country Club caddy shack when there was just enough of a stumble in the weather that over half the caddies were stuck in the shack.

Like many of the others, I played Tonk pretty furiously in the 'shack, and we all played for money. Even though I was still at the early time in my Tonk career (such as it was). Because you see, on

• rainy days when not enough people wanted to hire a caddy or play, and
• weekday mornings when the ladies who wouldn't pay more than $12 a bag and neither three holes in one nor CPR revival would get you much of a tip

a four hour stint of Tonk while waiting for the afternoon loops could easily make more money than the $0-$24 the above would net. And in those early days I was good enough to be regularly winning but still able to make it look like I didn't know what I was doing and ascribe it all to luck (the easiest way to do that was to hang on to a very low hand rather than going down when I could and catching whoever went down; then it was never an overt act on my part that made me win -- I just looked like someone afraid to play boldly!).

I had a very innocent face at the age of 14, but was already fairly well into an evil phase, I think.

Before the summer was over folks caught on to me, and it proved to be harder to fleece caddies of their "extraneous" cash through acting innocent -- so I just played straight like the oldtimers did. But that still paid, and caddying still paid well on its own, too. You could easily do 36 holes if you showed up early enough to guarantee a morning loop, and on a really wild day you could do 45 holes and that last crazy dude who wanted a quick nine at the end of the day would often pay as much for his one bag as the two guys paid for both bags just 4 hours prior....

Anyway, where was I before that massive digression?

Oh yeah, June 22, 1985. It was a Saturday, I probably should have mentioned that.

Although now, over 25 years later, I am wondering if it was Sunday (the 23rd). It was that weekend, for sure.

I hadn't sat down to play Tonk yet.

I was waiting for one of the guys to be out who was about to be down to his bus fare home (he must have started light, no one plays higher stakes that early in the morning than 25¢/50¢ and it has to be a weekend afternoon with a bunch of highrollers off $50-100 loops from the morning to play $1/$2 or higher stakes games -- I can't believe that Wikipedia article suggested stakes were usually 5¢/10¢, who the hell would waste their time?!?).

He was about to be out and although I was anxious to get in the game (and make up for no morning loop since if memory serves my dad was playing and had a late tee time and I was too lazy to walk to thye bus if I could get a ride). I was by then at least smart enough to not act too eager; I was just killing time, looking at an extra bit of the paper someone had brought in.

And then I saw it. In The Plain Dealer.

Ettore "Hector" Boiardi, who you may know as Chef Boyardee, had passed away on Friday. He had been living in Parma.

I suppose I could look in The Plain Dealer archives to find out the exact date, now that I think about it. But there isn't time right now, and this blog is time sensitive!

I called this fact (his passing away) out to the remaining caddies in the shack, and we all had kind of a moment of silence.

We had some conversations about the merits of the Chef's stuff vs. SpaghettiOs and so on. Some people liked one better than the other, and so on. And I finally got to play some Tonk.

Then in the afternoon when it got a little sunnier and we got loops, I asked Jerry (the caddymaster) if we could fly the flag (which was just being raised since the rain had stopped) at half-mast. For the Chef.

I will never forget the look he gave me. He had long before worked out with my parents and the guy on the committe in charge of caddying that I would be allowed to caddy even though my family belonged but I could never ask for special treatment and that he should never treat me different from the other caddies. And I never did (I just like the money better there than Oakwood and the distance better than Canturbury and the golfers/money better than Shaker). But I don't know if he truly believed it, because he still did give me what looked like preferential treatment sometimes.

And at the very least, Jerry did fly the flag at half mast that day, after I asked.... :-)

When I went out on my loop, walking down the long fairway of the 5th hole (a legitimate Par 5 with an out of bounds on the left that even I had hooked into once myself) kind of with the whole group since their balls were fairly close together on the fairway and one of the guys in the foursome who I was carrying for (his bag was on my right shoulder) asked if I knew what was up with the half-mast flag (he had seen Jerry putting it up on the way over to the practice green earlier).

I explained about how we were just showing some respect for Chef Boyardee, who had just passed away. You know, how maybe someday it would be fancy food we'd be into but I was 14 and the Chef was all I needed these days....

They were all kind of surprised but I was very earnest in my little soliloqouy and the other caddy nodded reverently, so no one laughed. A few nice things were said about his pasta, but obviously things were pretty reverent.

It all lightened back up by the 7th hole when a rather crazy slice became as point of focus, and we moved back into cheerier interaction. I remember being relieved that we weren't going to act like the whole loop was a funeral march, and the other caddy was also relieved (no want wants depressed golfers when the time to get paid rolls around!).

At the end of the loop (light bags, good golfers -- one of mine shot a 77 and the other a 79 -- and the day was not too hot, all and all it was minimal effort required) I got an extra hundred dollar billover the money I expected for carrying the bags including tip (between $30 and $50, and it was $50) -- so at $150 this was the single largest amount I had ever received up to that date and the larget amount ever outside of a big money game or tournament game, with no real reason given beyond a few kind words he said to me about the little speech on the fifth hole fairway. And that golfer had never tipped me that much, before or since. The $50 would be much more typical for this particular golfer....

Back at the caddy shack I told the tale and showed the money, and some people figured it was because I seemed so mature about the death thing.

Others though it was to cheer me up.

And then some others thought it was an accident and that the bill was supposed to be a $20, but it was on top so I doubted that. I didn't go back to offer him the $100 back; he was at least a two-comma guy with one of those expensive cars in the lot and I figured he could spare it.

Perhaps you recall me mentioning I was kind of evil at the time, though I didn't look it!

I made a bit more playing $2/$4 Tonk for a few hours (some old sharks were hoping to relieve me of that tip but they failed in their quest watching a few good streaks of mine and gave up, deciding I was going to stay lucky that day!), and then I went home.

It was a very profitable day, considering I had easy golfers. And between Tonk before and after (the high stakes game especially!) and the 18 and the tip I went home with nearly $200, all tax-free.

Not bad for an easy day as a 14 year old. Minimum wage was like $3.35 then, so if I had been working that long at most jobs I'd have no cash to take home and a measly $20 extra in my paycheck to show for it all the next week.

When is the last time you tipped someone at Chick-Fil-A? :-)

That day, my thoughts were not on the money though, they were still on Ettore "Hector" Boiardi, who I had never met.

Not the first time I encountered death, but he was the first person I had some kind of connection with (his pasta!) but died before I ever really found out anything about them. Too late to change that, I consoled myself with the fact that I would not have known how to go meet him if I magically knew he was about to pass on. So there was no way it would have gone any differently.

I don't really remember if I mentioned the Chef Boyardee when I got home or not. I mean I vaguely recall my mom having one of those "where did that come from" looks on her face but I was inspiring that look often enough that I can't be sure it happened on that date in particular.

I am pretty sure I didn't talk about the money. I didn't want to miss out on getting money for stuff like movies because they figured I could use my own!

Anyway, June 21st was quite a day in the life of me.

I remember it a lot more vividly than I do a lot of my caddying days (there are a handful I remember very well, this is one of them).

So when I saw it come up on the calendar while looking at the blogs, I pushed aside the blog that was going up (it will be there tomorrow, never fear!), and wrote this, instead.

I hope I didn't bore anyone too much.... :-)

There are times that I am very happy for some of the eccentricities of the way I look at code.

And the way they keep me from certain kinds of bugs.

Like a few hours ago when I happened to spot Matthew Wilson's memset() Considered Harmful - especially to those who (think they) know what they're doing! blog.

Just the title at first.

And two things immediately came to mind.

The first thing?

I know the problem he ran into.

Did you see it too? Before you clicked on the link, I mean. :-)

The second thing that came to mind?

Too bad he didn't use wmemset; it would have saved some time here!

The truth is that one of the biggest sources of bugs if one moves a lot between Unicode and non-Unicode programming is byte/character count problems. In fact that is one of the great things about wmemset, the way it (and its ilk) takes that particular variable out of the equation even if one doesn't happen to have std::fill_n() at one's disposal. :-)

My trial by fire for all of this was MSLU; the constant need on a per-function basis to be thinking about the Unicode and the non-Unicode kept me on my toes here and really drilled the issues into me to think very very carefully about buffer sizes. And also to feel smugly superior to all the Win9x code that tended to pop up with "bugs" occasionally related to non-Unicode buffers that were twice the size they needed to be except on the CJK versions where those were the expected buffers (and no they were not thinking ahead brilliantly, they were just messing up byte/character counts in WideCharToMultiByte calls!).

Now there are some flaws in the docs for wmemset and its ilk that I just noticed, like the security warning that really is two different problems between memset() and wmemset() and therefore deserves a bit of wordsmithing beyond a generic pointer to warnings about avoiding buffer overruns.

And the suggestion that the .Net Framework equivalent for memset() and wmemset() is System::Buffer::SetByte?

That's a keeper, for sure. I mean what better way to introduce byte/character mismatches into the .Net world so elegantly than that method and a cast or two? :-)

Now of course the viewpoint doesn't make me invulnerable to all bugs, it just makes a certain class a bug a lot less likely....

Another Multiple Sclerosis/iBot blog, with a tiny bit of iBot....

So some of you will have read Arising from one's own ashes. Like [up to 80% of ]a phoenix.... or Not everyone wants me to do The Right Thing™ + Ampyra update/irony...or one of the comments thereof, and are interested in how I am finding life with Ampyra now that I have been on it for a bit, taking 10 mg p.o. BID, which is to say one of the tablets twice a day.

This blog is dedicated to Karen, who asked in a comment to that first blog:

so,, have you tried it? has it helped you???

Good question!

Now first some background.

My symptom that Ampyra would help is disequilibrium, which I described in On the avoidance of being too tired to move, and the iBot as:

Third, I have disequilibrium. This is not like vertigo as I feel nothing spinning -- I just tend to see the ground coming toward me or falling away from me, without warning. I always have it somewhat and sometimes it can be extreme. I have had this symptom for years and in fact it is the one I originally got a cane for (I'd rather that people who saw me fall in the daytime assume I am a gimp, not a drunk!), the one that can at times be the most difficult and debilitating. Summary: I can't always tell where I am in relation to the ground.

Now on that same blog I also mentioned:

First, I've lost proprioception (joint position sense) in my legs. This means that I can put them somewhere but I can't tell where they are by any other means. So I have to trust that they will be where I put them and of course the corollary to that is that you have to remember where you put them. Sometimes I also get the symptom in my arms too, though this is not as worrisome. Summary: I have to trust where I put my legs and feet.

Listing them separately was intentional, although the sort of problems with the balance system of the body that these two symptoms indicate might have the same cause.

It is just that my disequilibrium's cause was never formally established; it was just experienced by me.

It is unlikely that it is vestibular in origin since that is the kind of problem that leads to the "spinning feelings" associated with dizziness or vertigo or too much water in an ENG test or Ménière's disease. I've had such issues in the past on occasion, and this is definitely not that issue.

Now previously (in blogs like On that contrived notion of ideal kissing height, and a mildly interesting and/or ironic language thought or two, as well) I have mentioned my positive Romberg's Sign, which let me assure you is a very negative thing to have.

As that article explains:

Maintaining balance while standing in the stationary position relies on intact sensory pathways, sensorimotor integration centers and motor pathways.

The main sensory inputs are:

1.Joint position sense (proprioception), carried in the dorsal columns of the spinal cord;
2.Vision
3.Vestibular apparatus

Crucially, the brain can obtain sufficient information to maintain balance if any two of the three systems are intact.

In my case, I know I have problems with #1, and although #3 is unlikely to cause disequilibrium without vertigo, I do have a positive BAER (Brainstem/Auditory Evoked Response), so it could not be entirely ruled out.

The meaning of the test result is also described in the Romberg's Sign article:

...In the second stage, the visual pathway is removed by closing the eyes, known as a "sharpened Romberg". If the proprioceptive and vestibular pathways are intact, balance will be maintained. But if proprioception is defective, two of the sensory inputs will be absent and the patient will sway then fall.

Now Ampyra itself, as it's Wikipedia article cites, only describes itself saying Ampyra...

...works as a potassium channel blocker. Electrophysiologic studies of demyelinated axons show that augmented potassium currents increase extracellular potassium ion concentration which decreases action potential duration and amplitude which may cause conduction failure. Potassium channel blockade reverses this effect. However, a recent study has shown that 4-AP is a potent calcium channel activator and can improve synaptic and neuromuscular function by directly acting on the calcium channel beta subunit

.And talking to the people at the drug company they won't even speculate that far, and have said that they are not sponsoring further studies in this area, or tests beyond the "walking speed improvement" test that they used to prove its usefulness.

Still with me? :-)

Okay.

Since I have been taking Ampyra, my very very positive Romberg's Sign is no longer positive. Yet my disequilibrium is still present.

This indicates several likely things:

• My disequilbrium and my proprioception loss are most likely the same problem manifesting in different ways, and
• The Ampyra does have an effect on at least one of my multiple sclerosis symptoms, and
• That effect is not currently detectable by me in regard to noticeable disequilibrium improvement, indicating that either the improvement is too small for me to detect, or the now negative Romberg's Sign has another cause, and
• My balance does feel a little different on the drug, though I cannot separate enough factors to describe how, precisely.

I got one refill of the Ampyra (so I would not run out in India) though after I get back I will probably discuss the situation with my neurologist.

If nothing changes, it probably won't be worth staying on a drug that costs over $1000 a month if even the known, provable benefits do not improve the symptoms I was hoping they would.

So I may be back to my iBot (which I haven't actually left, I mean justthe iBot), unless I see major differences with the heat while I'm in India that the Ampyra helps with.

All of this points to a flaw in the way that we imprecisely discuss balance/dizziness in general and the way the pharmaceutical company tested Ampyra in particular.

In short, when we talk about balance and balance problems, we are talking about a whole bunch of different possible underlying things.

Only some of them are potentially helped by Ampyra.

And in some people like me, they can help in only subtle ways that I myself might never have detected if Dr. Sherry Apple wasn't my friend all those years back teaching me about neurology.

The drug company is doing none of those work to figure out the types of balance problems the drugs can help, a problem that

• Causes some people to perhaps pay for a month or two of a drug that is unlikely to help them, and
• Hurts the reputation of the drug's effectiveness (a problem that is clearly happening, and causing many reports).

 It would have been harder for the drug company to assess the problem more specifically than they did, but they could have saved people a lot of money if they did.

And avoided the seizure risk that overdoses of the drug can cause.

Perhaps it is just me being cynical, but I suspect the two problems are related: the drug company suspects there will be a seizure scare due to people doubling up the dose leading too a huge clampdown on prescriptions of the drug. And they want to make their money will they can before such a potential lockdown, and would rather make the money while they can (being a pharmaceutical can be quite lucrative!).

If I had the ability I would want to more formally study all of this, but I don't. And the drug company isn't interested anyway -- they got the $2000 from me, which is probably all they were looking for....

In my time working in windows, I have seen a lot of different code.

As begets any project of this size written over this much time by this many developers, there is a lot there.

Some parts of it are ordinary.

Other parts of it are quite brilliant.

Still other parts are just wrong.

Every once in a while the wrongness even includes a comment that vmakes the wrongness so entirely clear to anyone who read the comment and went so far as to think about what it said that one has to conclude that no one was reading the comment and thinking about whether it might be true.

An example of this phenomenon can be seen in blogs of mine like The Bug(s) Spotted, aka Design flaws are worse than bugs, just to give one example.

Today I am going to talk about another!

Now before I get into that, let me backpedal a bit and mention a small point of obviation, or at least of mitigation.

When code has been around for a while (like multiple versions) it is treated as if it is a bit more stable. More likely to be right.

As a consequence, comments associated with that code are sometimes given that same bit of extra authority. And people who see it in passing who are not thinking about bugs or code reviews can easily let that authority stand, subconsciously.

It is of course still just as wrong, mind you.

But otherwise smart people can go so far as to quote the code with its comment when making a tangantial point, not even seeing the problem in either one!

Such as this bit from a piece of mail the other day, one that came up in a conversation about console applications (I have been involved in several such email threads/meetings after blogs like Conventional wisdom is retarded, aka What the @#%&* is _O_U16TEXT? and The real problem(s) with all of these console "fallback" discussions and Anyone who says the console can't do Unicode isn't as smart as they think they are and Cunningly conquering communicated console caveats. Comprende, mon Capitán?, as you might expect).

Product details removed for hopefully obvious reasons:

The current console output behavior for <redacted> and <redacted> was painfully established through years of international testing.
That’s not to say there are no issues – just that we need to be cautious about making changes.

// if redirected to a pipe or a file, don't use WriteConsole;
// it drops redirected output on the floor
// if going to a file, we should not use console codepage, we
// should use ANSI codepage OR write unicode & a unicode
// filemarker at beginning of file.
// Since we don't know if this is the first thing in the file,
// let's go the ANSI route.
WideCharToMultiByte(GetACP())
WriteFile(hStdOut)

See the problems?

 Let's make it interesting, in a Tosh.0 sort of way....

How many problems do you see, in any of the following:

• The email text (above, in blue);
• The comment (above, in green);
• The algorithm implied by the pseudo code (above, in black).

Make Daniel proud....how many problems can you find in 20 seconds?

Ready? Set? Go!

I remember back in the middle of 2005 (when I was getting into blogging and following various bloggers) at some Unicode meeting telling Rick McGowan of Unicode that they should have an RSS feed for announcements.

He thought bit was intertesting, but didn't see a real benefit beyond the announcement distribution lists that had been there for years.

Which is true.

I dropped the matter; I'm no zealot. :-)

Anyway, regular reader Jean just pointed out to me (in response to Unicode 6.0.0 in beta! that I should have linked to The Unicode Blog in the announcement.

Well shucks.

I didn't even know it existed; no one ever told me!

But there it is:

http://unicode-inc.blogspot.com/

I'll razz Rick about that later, of course.

For now I'll put up this blog, and some time before I get on the plane I'll add the link to the side....

After I wrote I have not, generally speaking, been a "flasher" (for the last few years at least!) about the flash.globalization namespace, I had people sending me mail.

They really wanted more information on Microsoft technologies as well.

They figured Flash was all well and good, but what kind of stuff was available in other (non-Flash) cases.

Fair enough.

Because Cathy, in her new role, pointed out something to me that is really interesting on that front. :-)

You may remember JScript's ultimate globalization hero: VBScript, where I pointed out:

JScript and JavaScript (aka ECMAScript) all have in common this particular lack of the kind of support for internationalization that people like me consider to be pretty important.

This kind of says it all....

Well, it did then.

But now, in jQuery Globalization Plugin from Microsoft, MS VP Scott Guthrie points out a new open source library that seeks to provide a much better answer here!

In his words:

Today, we released a prototype of a new jQuery Globalization Plugin that enables you to add globalization support to your JavaScript applications. This plugin includes globalization information for over 350 cultures ranging from Scottish Gaelic, Frisian, Hungarian, Japanese, to Canadian English.  We will be releasing this plugin to the community as open-source

That kind of says it all, doesn't it? :-)

Scott also includes some samples of how the plugin works and where to download it from and so on.

You can read more about it on Port 25. And this is a great thing on the Open Source and interoperability fronts, and is a great example of a case where Microsoft needed a problem solved but looked at the strategic advantages to an interoperable solution and delivered on it....

This is really very cool and should fill a long-empty hole in web development for a whole lot more more people than just me. :-)

In fact, in my own work then there might be times we can take JScript off the "challenging technologies from a globalization point of view" standpoint, which is not such a great list to be on....

I imagine VBScript will be able to get over no longer being JScript's "Globalization hero" though I admit I can't get the imag of the rejected VBScript in some takeoff on the Swiffer "Who's That Lady?" commercials or something. Makes me wonder who VBScript will be dancing with -- WSH maybe?

Anyway, enjoy. And let me know what you think of the JQuery Globalization Plugin!

(or if you decide to do that video, I think that would be hilarious!)