Search Form
Syndication
Administrivia

What to do with the Vietnamese keyboard on Windows?

Sorting it all Out
Michael Kaplan's random stuff of dubious value
Be sure to read the disclaimer here first!
MSDN Blogs > Sorting it all Out > What to do with the Vietnamese keyboard on Windows?

What to do with the Vietnamese keyboard on Windows?

11 Nov 2005 4:01 AM
  • Comments 31

By the way, I am not implying we must do anything at all. It is probably important to start with that disclaimer. :-)

If you think back to previous posts I have done on Vietnamese (here and especially here), it is clear that the implementation has been inspired largely and maybe originally guided by Windows code page 1258.

And although

  • the issues with some characters not sorting properly is/will be resolved in Vista;
  • using Unicode instead of the code page will take care of the rest of the limitations

There is still the problem of the keyboard.

We'll start by looking at it. Here is the base state:

and here is the shifted state:

If you look at the top row, it contains five combining characters. It is not a coincidence that those five characters are the same ones as are in code page 1258:

VK_5    CC = U+0300 : COMBINING GRAVE ACCENT
VK_6    D2 = U+0309 : COMBINING HOOK ABOVE
VK_7    DE = U+0303 : COMBINING TILDE
VK_8    EC = U+0301 : COMBINING ACUTE ACCENT
VK_9    F2 = U+0323 : COMBINING DOT BELOW

Okay, so the model here is (on this keyboard) to allow and encourage text that is not really in any particular Unicode normalization form. Instead, it makes extensive use of intermediate forms that are neither Form C or Form D (most of the keyboards that Microsoft ships are in Form C).

Vista does/will fix the issue with the various equivalent forms, so it will be less noticeable. But it does make things a little odd for any time the data is being transferred around, or if the Win32 NLS API is not used to get comparisons done.

Of course it does nothing for the fact that not all of the characters that have been added to the sorting table for Vietnamese can be typed in with the Vietnamese keyboard.

And it does not allow for a more sophisticated model that supports chained dead keys so that one could type the two dead keys and the letter to get the various precomposed characters used in Vietnamese.

Such a keyboard would not be as useful in legacy non-Unicode applications due to all the problems in and lack of compatibility with code page 1258, but it would be better for Unicode support, for the language, and for consistency with the other keyboards on Windows.

With all that said, I have fallen into the trap of coming up with a whole plan for a technical solution without knowing whether there are actual users who are running into problems with the existing support. If there are any native speakers of Vietnamese who have thoughts here, it would help prioritize this issue as opposed to other features that future versions might include. Including features like support for chained dead keys in MSKLC to make creating such keyboards easier!

What would people who need to type Vietnamese like to see?

 

This post brought to you by "ệ" (U+1ec7, a.k.a. LATIN SMALL LETTER E WITH CIRCUMFLEX AND DOT BELOW)

, , , ,
Comment on the blather
Leave a Comment
  • Please add 1 and 1 and type the answer here:
  • Post
Blog - Comment List
Comments
  • Mihai
    11 Nov 2005 12:24 PM
    I could query some Vietnamese translators.

    We constantly get from them MS Word files with NFC form (and then we have problems saving in 1258 :-), although the standard keyboard does not allow to input them.

    I suspect they use a 3rd party program or modified keyboard (maybe even with MSKLC :-)
    Since a translator would probably not care if a character is NFC or intermediate form, I guess that application/keyboard has some other attractive features.
  • 11 Nov 2005 5:40 PM
    That would be cool. I'd love to know more about what we need to do. :-)
  • Suzanne McCarthy
    11 Nov 2005 7:48 PM
    Hi Mike,

    As usual I have no idea what I am doing. However, I am taking an other attempt at explaining why I gave up on the MS Vietnamese keyboard. Not enough google hits to fill a thimble.

    http://abecedaria.blogspot.com/2005/11/vietnamese-revisited.html

    Suz
  • 11 Nov 2005 8:20 PM
    Hi Suzanne,

    Well, it is not your fault if the combination of a bad keyboard and lots of bad user data is out there -- it is a no-win situation....
  • Suzanne McCarthy
    11 Nov 2005 9:24 PM
    Surely the user data from the VietDic dictionary is from another aand more acceptable encoding standard. Why doesn't the MS keyboard enable one to keyboard this encoding? Obviously VietDic is the one with the most hits and MS keyboard encoding has the fewest hits. It indicates to me that the MS keyboard is not being used much because it doesn't match the standard encoding. So why not change the keyboard to match the standard? Have I missed something? I find the MS Vietnemese keyboard unusable. I know it is not my fault - why would it be?
  • 12 Nov 2005 12:00 AM
    I was referring to where you talked about not knowing what you are doing.... When we talk about encodings, anything short of Unicode will be inadequate at representing text fully. That includes *all* of the other possible options, and cp1258, too. But in any case, individual standards that cannot represent other languages are an evolutionary blind alley -- as is deciding the best encoding for a language by measuring google hits! :-)
  • Suzanne McCarthy
    12 Nov 2005 12:39 AM
    Mike,

    These are *all* unicode - what do you mean by 'short of Unicode'? What else would I be using?

    The main difference for example 1 and 2 is this.

    1. uses one character for the 'e' - U+1EC3 : LATIN SMALL LETTER E WITH CIRCUMFLEX AND HOOK ABOVE

    2. uses two characters in sequence - U+00EA : LATIN SMALL LETTER E WITH CIRCUMFLEX and U+0309 : COMBINING HOOK ABOVE

    Aren't they both unicode? The other options are visibly different so they don't really count except to show the complexity involved.

    Is there supposed to be one standard and who would decide that? The google hits are just to show that they are different really - I have no idea which encoding is best, just wondering if there is a process or entity which deals with this discrepancy?
  • Mihai
    12 Nov 2005 1:43 AM
    Ok, got it!

    Sounds like something serious. The thing is available free from the "Vietnamese Professionals Society" (http://www.vps.org)

    Click DOWNLOADS, the software is VPSKeys 4.3.
    You should click on "(latest version)", if you click on the software title nothing happens.

    The description is "Vietnamese Keyboard Driver for Win95-98, ME, NT4, Win2000 and Win XP" (impressive coverage), and and English manual is available (great, because the UI is Vietnamese :-)
  • Andrew Cunningham
    12 Nov 2005 3:44 AM
    Hi Michael,

    In theory, applications and web services should normalize their data. The problem of the character sequences generated by the Windows Vietnamese layout vs third party solutions is more related to this lack of normalization. The users end up in a mess.

    examples:
    1) using a third party IME in MS Word (and using precomposed characters) and having the Vietnamese proffing tools installed ... spell checking doesn't work, the dic assumes the character sequences the Windows layout uses.

    2) searching Google. Google doesn't normalize. Search results using Microsoft's keyboard will yield different results than a third party solution.

    3) Searching BBC's Vietnamese site with the Windows layout will find more recent articles, and using a third party solution will find older articles. Assuming they haven't updated/fixed that since my last search.

    Third party solutions tend to be more flexible about the location of teh diacrtic. Some softwrae allows you to either type the diacritics after the base vowel or at the end of the word, dpeending on your preferences. (seems to mimic handwrighting styles in this sense).

    A similar problem is beginning to surface with African languages, as more keyboard layouts are generated using MSKLC. There have been a number of dicussions in various fora regarding Nigerian languages when typing tones.

    The ability to create keyboard layouts, using MSKLC, that generate NFC or NFD character sequences would be useful.

    But in the long run, normalization issues aren't particularly a keyboard layout issue.

    If this makes sense?

    Andrew
  • Gabe
    12 Nov 2005 4:25 AM
    I don't know much about Vietnamese, so maybe I'm wrong to be surprised here. But why on earth could they possibly use backslash often enough to warrant two keys for it on opposite sides of the keyboard?
  • 12 Nov 2005 9:06 AM
    Hi Gabe --

    That one way over on the left is the VK_OEM102 key, which is not present on 101-key keyboards and is thus by convention usually duplicated elsewhere.

    MSKLC even issues a warning in the validation phase if you do not have an assignment there duplicated elsewhere, for that very reason....
  • 12 Nov 2005 9:10 AM
    Hi Andrew --

    Yep, makes sense. One of the changes in Vista is to pick up the different normalization forms (and all of the 'intermediate forms', as well) which will at least make comparison technologies make use of them.

    Over time, this may well improve search for MSN, though probably it will not do much for Google's search. :-)
  • 12 Nov 2005 9:12 AM
    Suzanne --

    This is a terminology difference between us -- Unicode is a single encoding. When you talk about encodings, I assume you are talking about the endlerss attempts at 8-bit encodings for Vietnamese, from cp1258 to all of the VNI code pages, and so on.
  • 12 Nov 2005 9:23 AM
    Hi Mihai --

    Thanks for the link, though it does look like they are hosting the Windows core fonts for download. :-(

    But the Online help looks interesting, and it does seem to support many of the different 8-bit encodings as well as Unicode...
  • Suz
    12 Nov 2005 3:56 PM
    Hi Mike,

    Sorry about my terminology. I have learned something about how to view, identify, and display characters since last June but not talk about them properly. Just so you know, I have never knowingly used or talked about anything other than Unicode. Anyhow - lots of interesting ideas from this.
Page 1 of 3 (31 items) 123
MSDN Blogs > Sorting it all Out > What to do with the Vietnamese keyboard on Windows?