Blog - Title

July, 2009

Sorting it all Out
Michael Kaplan's random stuff of dubious value
Be sure to read the disclaimer here first!
  • Sorting it all Out

    I know I'll Never say Never... again, at least

    • 4 Comments

    So, thinking about consequences of the CASING piece of Every character has a story #33: U+1e9e (CAPITAL SHARP S, Microsoft edition - Part 2).

    And ignoring that we didn't follow my recommendation, since I've covered that, for now.

    Let's look at what we did do.

    We essentially took a letter whose primary and for all practical purposes only need is to handle the German word with a Sharp S that is fully capitalized, and make sure that no process that will ever capitalize it.

    Ever.

    Have a cup of irony, will ya? :-)

    Yesterday I did talk about another issue I noticed.

    Regular readers know I've discussed U+03c2 (GREEK SMALL LETTER FINAL SIGMA) before.

    This letter, which has had an intentional one-way mapping to the regular uppercase function since NT 3.1, had this mapping moved to the linguistic tables activated by LCMAP_LINGUISTIC_CASING (for all locales).

    This changes the fundamental behavior of a character in the filesystem that has been in existence since the early 90's if you count the beta versions.

    As discussed in The last word on the FINAL SIGMA:

    If you are a native speaker of Greek, then you know that both ς (U+03c2, a.k.a. GREEK SMALL LETTER FINAL SIGMA) and σ (U+03c3, a.k.a. GREEK SMALL LETTER SIGMA) do indeed uppercase to Σ (U+03a3, a.k.a. GREEK CAPITAL LETTER SIGMA). But if we added this character to the linguistic table, then it suddenly ς would never work in the CharUpper/CharUpperBuff functions and would not work in the default call to LCMapString with the LCMapString function with the LCMAP_UPPERCASE flag.

    Obviously that would not be a good thing.

    Try to imagine how you would feel if attempting to uppercase the string hello would come out as HELLo. Wouldn't you consider it a bug? Especially is it used to come out with the HELLO you were expecting? You might be thinking about telling the platform GooDBYE, if you know what I mean.

    Now you don't have to imagine it -- you get to see it. If you deal with Greek at least.

    Relying on all caps style appearances in places like IE will see documents changed, too -- at least when they are looked at in the latest version.

    The easiest way to see it is to use Word's "Change Case" support -- prior versions will convert U+03c2 to U+03a3; Windows 7 will not...

  • Sorting it all Out

    Every character has a story #33: U+1e9e (CAPITAL SHARP S, Microsoft edition - Part 2)

    • 9 Comments

    You may want to start with Part 1 of this series....

    Okay, here we go. Last time some of the font stuff was covered.

    It is nice when letters are given the opportunity to look good.

    In the end, what can often be more important than how something looks is how it arrives, and how it behaves!

    I'll divide this up into a few parts.

    Part 1: INPUT

    This letter, LATIN CAPITAL LETTER SHARP S, was not added to the German keyboard.

    So if you wanted to input it, then you are on your own.

    Good luck with that for the average user. Oh wait, the average user kniws there us no such letter anyway. Never mind, the lack of a way to input the letter is okay....

    Part 2: COLLATION

    Well, if you go back to blogs like Dere are qvestions? In zat case... and the one that started it all What the %#$* is wrong with German sorting?, the following collation equivalences already existed:

    ß == ss

    Now note that now, just as was the case then, I was not stating any special kind of linguistic truth.

    I am simply point out that if you want ß to be treated as equivalent to SS when you ignore case that you have to recognize that ss is also equivalent to SS when you ignore case. So that this relationship which looks so unnatural to a native speaker allows for meaningful results to be returned.

    Now that we have a new uppercase form of the letter, let's see where it fits.

    As Peter Gibbons pointed out yesterday:

    But what's more important is that the collation algorithms seem to process "ẞ" right. At least in explorer with filenames.

    To make this happen, the following equivalences are needed:

    ß == ss

    ẞ == SS

    Now with these in place, everything behaves the way people expect them to, even if the above equivalences are like fingernails on the chalkboard to folks in Germany.

    Points given for intuitive results, right? :-)

    Anyway, there you have it.

    Though, as I pointed out way back in Collation != case, still, collation is not case.

    So now we get to part 3.

    Part 3: CASE

    The wheels are gonna come off the wagon a bit here, I will admit.

    Basically, it seems that the casing relationship between and ß was added, but only in the linguistic tables, and only to say that lowercases to ß.

    This means that two files that differ only by their use of one letter versus the other can co-exist in the same directory.

    This kind of sucks from an intuitive nature of results, so I am going to hope that the PART 1 issues with the inability to input the letter can be relied on. Though technically an obfuscationary solution, I won't think about the potential security and spoofing issues of these letters that look so similar in so many fonts and focus on the lack of built in input and the fact that it is technically someone else's problem now (as this ant in Alaska likes to frame the situation!).

    I am on the record (for what it's worth) as to how I felt the situation should have been handled here

    But on the other hand, the case table is used in order to enforce the case insensitivity in the NT object namespace and the file system. And one clear issue is that there is no good reason to allow one to put filenames differing only by the presence of U+00df and U+1e9e in the same directory. Users would either never try it or they would never expect it to work. So it is quite possible that in the next version of Windows (which only does simple casing) it may make the most sense to make the two characters case variants of each other -- to enforce reasonable use of both letters!

    There is still lots of time to decide, though at present I am leaning this way since it will give the most intuitive behavior for end users (even at the expensive of giving slightly unintuitive results for developers).

    Ah well, coulda woulda shoulda. Or whatever.

    I did accidentally discover an unrelated thing, a thing that I'll talk about tomorrow....

  • Sorting it all Out

    Every character has a story #32: U+1e9e (CAPITAL SHARP S, Microsoft edition - Part 1)

    • 12 Comments

     

    Previous blogs about this letter:

    Now once again, keep in mind that for most of the German speaking world this still isn't a letter....

    So Windows 7, which once again (just like in Vista) was made to be updated to the most recent version of Unicode it could, LATIN CAPITAL LETTER SHARP S needed to be integrated.

    Let's take a look at in WordPad, using the Segoe UI font:

    Ok, interesting. Obviously they couldn't make it much taller. So they made it a little wider and left it at that.

    I think that is probably what Unicode did too.

    Makes me wonder what happened in Character Map. Let's take a look:

     

    Hmm... Undefined? Oh I guess someone forgot to regenerate the list of names that Charmap uses. Luckily it can still display characters even when it doesn't know what they are.

    Any testers want to put that bug in? :-)

    You know I kind of wonder what they did for fonts that can't change the width.

    Let's take a look at Consolas:

    It's not there.

    Oh damn, let's look at some other fonts, too.

    Like Tahoma:

    and Microsoft Sans Serif:

    and the fixed width font that is in the font link chain, Courier New:

     

     And I am sincerely curious what the upper and lower case look like next to each other on that one. Let's take a look:

    Interesting!

    And it does meet the fixed width rules -- notices how the surrounding text lines up?

    Though it makes me wonder what might have changed from the old font's lowercase character. Just a little bit curious....

    Okay, so Courier New has it, yet Consolas does not.

    Uh oh -- is this a C* font thing?

    Let's look at Calibri, the default font in WordPad:

    Crap.

    Notice how RichEdit doesn't seem to be looking very hard for the substitute. Thank goodness Word is not this lazy!

    How about everyone's favorite uber-font, Arial Unicode MS?

    Double crap.

    Or maybe we'll get another 20 or 30 people who will agree with me that Arial Unicode MS effectively [bites|sucks|blows].

    Silver lining of a sort....

    On last font I want to check out though.

    Times New Roman:

      

    Wow, I think I like this one best -- this is on I can really tell the difference on. Much more than the others. Truly.

    Okay, let's move on, there is kind of a pattern and kind of a logic here. I'm happy. Well, as happy as I can be about a letter that doesn't really exist in the first place....

    But just wait until tomorrow when I do part 2 of this blog. :-)

  • Sorting it all Out

    Pretty damn closer to top of the line

    • 13 Comments

    You may have heard that Windows 7 RTM'd the other day....

    There were two machines of mine that I hadn't yet tried to upgrade as I was waiting for OEM support first.

    One of them was my Dell Precision Mobile Workstation, the one I mentioned in Pretty damn close to top of the line.

    But I had a spare drive so in the spirit of RTM I decided to try installing and clean and seeing what happened.

    Now although the 32bit install was pretty smooth when I put Vista on it, the 64bit install was tough since I had to find drivers myself (Dell wasn't supporting it yet so they had none on their site), asnd I figured it would be a good comparison to see if the story was any better this version.

    You want to know what? It was! All but six drivers were right in the win7 image ("inbox" drivers), five of the remaining six six were detected and found on Windows Online, and the sixth was just the Touchpad (the Vista driver worked perfectly). I was very quickly up and running!

    Even the modem was there, which last time wasn't available at all until long long after....

    The funny thing was the Windows Experience Rating!

    You may recall that the Vista one on this machine was 4.7:

    with a breakdown of:

    ComponentSubscore
    Processor5.2
    Memory (RAm)4.7
    Graphics5.9
    Gaming graphics5.8
    Primary hard disk5.0

    Well, the Windows 7 numbers were quite a shock!

    It was 5.1 for the Windows Experience Index:

    With the following breakdown:

    ComponentSubscore
    Processor5.8
    Memory (RAm)5.8
    Graphics5.9
    Gaming graphics5.8
    Primary hard disk5.1

    Wow -- identical hardware, 31 months later, and the scores are so much better!

    I went ahead and ordered another hard drive from Dell (both the original and the replacement were 5400 RPM drives so obviously that score would be pretty easy to raise!), but the improvements -- to the processor and the memory -- are kind of confusing to me.

    It was not running with or without Aero (in both cases the numbers were identical with and without Aero).

    Was it a flaw in the original tests?

    Was the slowdown due to Vista itself? Or the drivers on it?

    Or were the new drivers simply faster in some unanticipated way so that even on identical hardware the difference was visible?

    Or did Microsoft simply alter the nature of the tests themselves? It isn't like they guaranteed them to be identical tests across different versions.

    I'll let you know how things go after I get the drive, but does anyone have an opinion about the improvements to the two scores?

  • Sorting it all Out

    The Letter Police can EAT MY SHORTS!

    • 4 Comments

    Regular reader from way back (and perhaps still regular reader!) Jan Kučera asked over in the Suggestion Box:

    Hi again... I hope there are more people going to suggest a topic or place a question, otherwise I would feel a bit... alone. Maybe you should keep some fake entries in the suggestion list so that some bashful people (like me) don't feel annoying when...you know, suggesting a topic. :-)

    Though obviously I did that step, but this is only because I already know you are very kind and cheerful person... and thank you for all your help and extraordinary useful posts and advices!

    Well, I've just started learning Tamil at our university. Now I would like to create a document with the individual graphemes (or signs in the unicode naming terminology), like ௌ (0x0bcc) for example. I have no problems with this in notepad, wordpad, or WPF. However, it seems that this is not allowed in any of the Office 2007 applications and neither it is in the Visual Studio 2008 text editor, which require me to type the consonant first (yet pasting the vowel sign from clipboard just works).

    Besides the general background about this behavior and its intuitiveness, there are particularly two questions which I am interested in:

    As an user - is there any way how to type in the individual vowel signs in the Office applications besides the clipboard/custom keyboard/insert symbol way?  (the Alt+X trick does not work on standard Tamil keyboard either, which, however, does not surprise me at all :))

    And as a developer - any chance I could control (or find out) the current behavior? Like do not allow the signs in my WPF window (if I were that kind of person, of course)?

    Thanks and wish you lots of good and interesting questions!

    Very good question, that first one. The second one freaks me out a little though. :-)

    The problem is yet another example of one I have blogged about previously, in blogs such as Being smart, by not trying to be clever and You're not the one out of sequence, and that's the Word:

    Illegal Sequence Checking

    This Word-specific extension to Uniscribe's own forays into the same feature in Thai are something I often knock for two specific reasons:

    • some of its checks are reportedly incorrect for languages they try to cover;
    • some of its checks seem to break other languages that use the same script.

    I suppose we could add a third case to cover Jan's first question:

    • some of its checks keep instructional/educational materials from being publishable.

    Now I kind of indirectly discussed some of my own problems with this scenario in Why international test is an art (and why there are few fine artists), where I mentioned the challenges in supporting Thai in MSKLC due to the need to be able to display individual letters on keyboard keys that are for most scenarios illegal in isolation (an illegality enforced by Uniscribe).

    Personally I am opposed to this feature on any language for all of the reasons given, and am thankful for the times I can turn it off.

    (And of course the other option is to put in a  SPACE or as NO BREAK SPACE and use it as the "base" character" (this is how I solved the problem for Thai in MSKLC, since there is no way to turn off the feature in Uniscribe like there is for Word).

    This is also the workaround for the Visual Studio case or really any attempt at being the LETTER POLICE one may run across....

    And if I could plead with anyone wanting to implement either smart or "unsmart" versions of this feature? Please don't! :-)

  • Sorting it all Out

    No disassemble #5! Redux

    • 2 Comments

    This blog has its roots (obviously) in its predecessor (No disassemble #5!).

    In that blog I described the method I had been using to safely transport myself and the iBot all over the country. And by safely I mean safely for the iBot, circumventing many of the horror stories of air travel that so many people had been telling of baggage claim personnel had vistited upon their iBots as I have been able to get through so many slights unscathed....

    But then, in a comment to Confessions of an iBot riding fool...., Nancy Nelson was looking for a lot more details on this:

    When you get a chance, would you please send me step-by-step instructions of how you travel on an airplane with your iBOT.  I haven't traveled by air since my last exacerbation left me unable to walk more than a couple of feet, and I have never put the iBOT on an airplane.

    Do you travel alone or with someone?

    How do you deal with your luggage?

    Do you also travel with a manual chair or walker?  If so how do you deal with all the extra equipment?

    What do you mean by planeside?  Does the airline actually allow you onto the tarmac?

    How do you get to and from the restroom on an airplane?

    Once you get to your destination, how do you get your iBOT, luggage from baggage claim, etc. and then manage to get to ground transportation?

    The airlines have really screwed-up my manual chair in the past.  I have always checked my chair at the gate.  I can't imagine what I would do if the airlines severely damaged or destroyed my iBOT.  However, vacations are just not much fun without the BOT.

    Thanks -- Nancy

    Very good questions!

    I think I'll take the individual questions first....

    Do you travel alone or with someone?

    I have, generally speaking, been traveling alone.

    Most of the later questions will reflect this; if traveling with someone, many of the steps are easier.

    How do you deal with your luggage?

    For shorter trips, I have kept myself limited to two bags, one of them holding my laptop -- so that I could carry them both on the plane. I usually include my spare iBot charger in a third bag that I leave on the iBot itself, and have never yet had anyone question the need to carry this in a separate bag and have never been charged any penalty or had any other problems with this approach.

    Of course that is not Nancy's actual question; I have carefully but intentionally misunderstood her here so I could mention the above. Her actual question was a wider one related to how do I manage the luggage situation in general, from home to destination and back.

    For those shorter trips, I simply make sure that the two (or three) bags all fit on the iBot itself; two of them simply get put over the top of the big bar on the back so that I can either easily get them off myself while staying in the chair, or else easily direct someone from TSA to take the bag(s) off the back.

    As a rule I go through security in balance mode; this amuses and interests the TSA folk and I have seldom felt relegated to the hell of being ignored in a wheelchair which seems so easy to do when one is not in something that people can't help but stare at. People notice. And they help.

    This does require some extra care since it will almost certainly change the center of gravity of the iBot. The principal difference I have noted when the bags are heavy enough is that the transition to balance happens automatically when the chair tilts backward (usually I would have to throw the joystick forward as the final step). Although this felt unsettling the first few times it happened I got used to it quickly and have been doing a lot of training in various weight configurations so I can feel comfortable with how the iBot compensates.

    The habit of carrying people in my lap and on occasionally letting someone on roller skates catch a ride from me has also been helpful here -- it gets me used to a wider variety of situations to better understand how the iBot does its work under these different conditions.

    Catching a ride... 

    I feel like I have a much better understanding of my iBot than I ever would have had otherwise....

    On the rare occasions that I have had to take a bag to check that would not fit on the back of the iBot or rarely in my lap, I used a roller bag that I could handle using my non-joystick hand and whose handle I could reach easily. Now this part does require some extra coordination as it removes any way to carry anything else -- so if there was a door or an elevator button or whatever I have to be able to stop, let go of it, perform the action, and then grab the handle again. But generally the trip toward the airport has very little of those inconveniences, and the only real challenges are getting out of baggage claim and handling elevators once I have landed. I just work very deliberately, and the only time I get out of balance mode is waiting at the baggage carousel. This makes it easier for people to not miss seeing me....

    Do you also travel with a manual chair or walker?  If so how do you deal with all the extra equipment?

    I never travel with a manual chair or a walker, though I occasionally travel with a foldable cane.

    On one occasion for a Los Angeles trip I traveled with a foldable ramp, suitable for loading the iBot onto a van without a lift. Although this was a very heavy item, I carried it in my non-joystick hand and checked it, and then rented a Ryder panel van at the destination. Since the daily rental price of that vehicle is less than a single typical accessible taxi ride in LA, this was well worth the extra effort.

    Obviously if one is traveling with someone, such issues are easier to deal with!

    What do you mean by planeside?  Does the airline actually allow you onto the tarmac?

    I mean gate check, though they let me travel down the jetway to the edge of the plane and in places like the Burbank airport (where Alaska has no jetways) they let me roll to the side of the plane, on the tarmac. Though in those situations they let everyone out there during boarding....

    I take the legrests off and put them in the seat before folding the chairback down. Sometimes I stick the UCP in there too, sometimes I stick it in my bag, either way it is safely surrounded by lots of protective layers.

    How do you get to and from the restroom on an airplane?

    I do my best to try and get into first class and then use the cane to get there; on the occasions I could not do this I get into row 6 (the first coach row) and they let me use the restrooms in first class given the situation. I am always sure to mention it to a flight attendant beforehand so they know why but the cane pretty much makes it a non-issue and they have no trouble with it.

    Once you get to your destination, how do you get your iBOT, luggage from baggage claim, etc. and then manage to get to ground transportation?

    Okay, some of this I have covered already but the additional points are important....

    Since I checked it planeside it is delivered planeside. As soon as we land I identify to a flight attendant that I have a powered wheelchair checked planeside, and I make a special request.

    I ask them to tell the baggage handlers don't assemble the iBot AT ALL as I want to do it myself.

    The horror stories I hear have usually been about something broken by an overzealous baggage handler, so I always take this extra step.

    Note that once the chair arrives (usually after everyone is off the plane, this is the one inconvenient part about the iBot is that one wait), I put it together and get all my bags on it. Then I put myself in 4-wheel mode there at the side of the plane and into balance mode as soon as I am on level ground.

    Now I find that getting to ground transportation to be the one place where extra elevators and such are an issue, as I mentioned before.

    Ok, I guess that is it for now, though I'll be happy to take the next batch of questions in another blog. :-)

    Special thanks to Lexi Belle who served as the model in the art above; she told me afterward that the ride was a lot of fun!

  • Sorting it all Out

    Garbage in, garbage out -- and this means Ü!

    • 5 Comments

    So the other day a colleague over in C++-ville forwarded a bug report he was looking at, one he wanted my thoughts about.

    It went something like this:

    Repro Steps:
    + Change language in Control Panel under "Regional an Language Options" tab "Advanced" to Japanese
    + Make new c++ console project with MFC support
    + Change Character set to Multi-Byte (_MBCS)
    + Compile folowing Code (0xFC => "ü"):


        char* test= "\xfc";
        CString tt;
        tt = test;
        tt.MakeUpper();


    + Run or Debug the program => Crashes when MakeUpper() is called.

    The result of running this code, as the last step mentions, is a crash in the "Microsoft Visual C++ Debug Library".

    Obviously, the question was whether this was a bug....

    Now one of the cool things about the CRT, MFC, and ATL from the developer's point of view is that you don't have to take my word for it, you can look at the source if you don't believe me!

    In this case, CStringA's MakeUpper function calls an internal function StringUppercase that calls (when non-Unicode data is passed) CharUpperA or CharUpperBuffA -- in this case I think CharUpperA. These functions both call LCMapString eventually, after some convolutions of their own....

    But the important point is the

    0xFC => "ü"

    claim in the repro steps.

    This is very true in Windows code page 1252, but since the steps require changing the default system locale to be Japanese, the code is running with Windows code page 932.

    And on that code page, 0xFC is not LATIN SMALL LETTER U WITH DIAERESIS; it is a valid lead byte that is used for the following potential trail bytes.

    So now we see what is happening.

    Since a legal lead byte was found, the assumption is that a legal trail byte will follow. And when that attempt to access the non-existent trail byte happens, a crash occurs.

    So whose bug would it be?

    • In the Japanese Microsoft subsidiary, there is a definite preference for meaningful data, and a serious paucity of people doing non-Unicode data on any code page other than 932. So for them this a "garbage in, garbage out" bug and by design;
    • At the NLS level, the thoughts are similar, though there are nuances (skip two bullet points to see what I mean);
    • At the "USER" level, there is not much serious thought, mainly because even though NLS doesn't "own" those functions, they do own the functionality, something I talked about rears ago, and though there is some code here its bugs are almost invariably caused by either what they call or what calls them;
    • At the "ATLMFC" level, the StringUpperCase overload that calls CharUpperA has a deprecation assert on it since no buffer size is being passed and there is a good faith basis for functions below it to assume one more byte needs to be read under the circumstances, and the CharUpperBuffA-calling overload would be getting a buffer size that is either incorrect or that is being ignored by the NLS function. So either this is an ATLMFC bug for setting up a one byte buffer overrun or an NLS bug for doing a one byte buffer overrun (I suspect the former, fwiw, and not a "bug" but kind of the very reason the function is deprecated!);
    • At the customer level, it is a true "garbage in, garbage out" bug based on an incorrect assumption about code pages and definite proof of one of the many benefits of using Unicode!
    Sorting out what should be changed/fixed or if anything should be is something I'll leave to the owners to track down.... :-)
  • Sorting it all Out

    Anything still wrong is probably wrong for good....

    • 1 Comments

    Some complaints people have had about the doc story on some international issues have been around for a long time.

    Longer than this Blog in some cases.

    Now this Blog (and the constituent blogs thereof) has been around since the end of 2004. So if I point an issue that was first mentioned by readers here in a blog like What is my locale? Well, which locale do you mean? that was first published on February 1, 2005, then it is clear that this is an issue that been around for a bit.

    Maybe a whole byte's worth of bits. Or more....

    The simple truth:

    setlocale(LC_ALL, "")

    will set all aspects of the default CRT locale to be the same as the user default locale, aka LOCALE_USER_DEFAULT.

    You can tell me if that sounds confusing or not.

    I'm pretty sure it is less confusing than phrases like you can find in the various versions of the MSDN docs about setlocale, _wsetlocale.

    Generally, no one gets it right.

    Ben Bryant pointed out in a comment to that very blog about how the Delphi 7 SysUtils got this one wrong; not sure if they managed to fix this up or not.

    Now in theory you can try to make clever use of ".ACP" or ".OCP" to try and fix^H^H^Hwork around this, but if you want to be sure to completely get the correct locale, code like this will show you a properly set CRT locale:

    #define _UNICODE
    #define UNICODE
    #include <tchar.h>
    #include <stdio.h>
    #include <locale.h>
    #include <windows.h>

    #define SIZE(x) (sizeof(x) / sizeof(x[0]))

    void _tmain() {
        TCHAR language[80] = {0};
        TCHAR country[80] = {0};
        TCHAR codepage[5] = {0};
        TCHAR localestring[167] = {0};

        if(
            GetLocaleInfo(LOCALE_SYSTEM_DEFAULT, LOCALE_SENGLANGUAGE, language, SIZE(language)) &&
            GetLocaleInfo(LOCALE_SYSTEM_DEFAULT, LOCALE_SENGCOUNTRY, country, SIZE(country)) &&
            GetLocaleInfo(LOCALE_SYSTEM_DEFAULT, LOCALE_IDEFAULTANSICODEPAGE, codepage, SIZE(codepage)) && // use LOCALE_IDEFAULTCODEPAGE for console apps
            (_stprintf_s(localestring, SIZE(localestring), _T("%s_%s.%s"), language, country, codepage) >= 0)
          ) {
            _tprintf(_T("NULL     --> %s\n"), _tsetlocale(LC_ALL, NULL));
            _tprintf(_T("To set   --> %s\n"), localestring);
            _tprintf(_T("Set \"\"   --> %s\n"), _tsetlocale(LC_ALL, _T("")));
            _tprintf(_T("Set sys  --> %s\n"), _tsetlocale(LC_CTYPE, localestring));
            _tprintf(_T("NULL     --> %s\n"), _tsetlocale(LC_ALL, NULL));
        } else {
            _tprintf(_T("%s\n"), _T("Error!"));
        }
    }

    Note it also handles a few other problems like the fact that the setlocale, _wsetlocale docs make no mention to the _tsetlocale function that works quite well, thank you.

    The code basically sets the user locale for everything, then sets the system locale for LC_CTYPE -- which handles the character property stuff (that is actually locale independent, ref: ) and the codepage stuff so that all non-Unicode operations will use the same default that the system does.

    And although using ".ACP" or ".OCP" will return comparable results, any querying of the LC_CTYPE locale won't match the system, which is why I like this code better -- it gives correct answers. :-)

    Anyway, I have given up on trying to see this topic fixed in MSDN. It has slowly been getting better but to be honest way too slowly to be helpful. And no one wants to "fix" the function since people will have behavior differences to deal with if "" wrre to suddenly really mean "act like the system does", which is what the above settings would do.

    So I thought I'd write this blog....

  • Sorting it all Out

    Atlas? Jove? Where's my Wubi? (The latter shrugged, the former nodded)

    • 3 Comments

    So some people noticed that the Wubi IME that used to work on Windows wasn't there anymore.

    You may be familiar with the story -- it was there for them in XP but then for Vista (and Windows 7) it wasn't.

    The funny thing for me about the Wubi IME is that when I hear the name I have trouble getting the Mr. Mom visual of the small boy asking Michael Keaton Where's my Wubi? (this was a security blanket that his younger son carried around religiously for the first half of the movie; I'm fairly certain they spelled it woobie but they pronounced it kind of the same way).

    Anyway, everybody who had the problem insisted they had not installed any third party program yet they had it.

    So clearly it came from Microsoft, as far as they could tell.

    Now Microsoft is a big place, and sometimes it is not just a matter of the left hand knowing what the right hand is doing; sometimes it is actually the left hand knowing that there is a right hand.

    And eventually people found it.

    Probably they could gone to Wikipedia and found it much faster, in the article entitled Wubi 86 that gave instructions on how to enable it:

    In relation to Windows XP, Wubi 86 is part of the Chinese PRC language LangID 2052; it is a subset. It is found on Disc 4 of the MUI pack for Office XP.

    Its IME language code is e0100804.

    If you do not have the Office XP MUI installed for simplified Chinese, it will display Pinyin method instead of Wubi86 for the IME. If you need to use Wubi86 on a Windows XP system as an IME or default keyboard layout:

    • install the Simplified Chinese MUI for Office XP from Disc 4 of the 12 disc set.
    • Then just set the user's keyboard layout to e0100804.
      • The registry key is found at "HKCU\Keyboard Layout\Preload" - 1 is the first one, 2 is the second and so on.
      • Just create a string value with the name 2 and the data e0100804 to enable Wubi86 as an input option.

    This IME actually did come from a third party and the Office team arranged to ship it for the Simplified Chinese version of Office XP (and also, as the above makes clear, in the language pack for Office XP).

    I don't know about you, but here is what I was reminded of, here ist he bit from Douglas Adams:

    "But Mr Dent, the plans have been available in the local planning office for the last nine months."
    "Oh yes, well as soon as I heard I went straight round to see them, yesterday afternoon. You hadn't exactly gone out of your way to call attention to them, had you? I mean, like actually telling anybody or anything."
    "But the plans were on display ..."
    "On display? I eventually had to go down to the cellar to find them."
    "That's the display department."
    "With a flashlight."
    "Ah, well the lights had probably gone."
    "So had the stairs."
    "But look, you found the notice didn't you?"
    "Yes," said Arthur, "yes I did. It was on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying 'Beware of the Leopard'."

    We didn't go out of our way to make the Wubi easy to enable either, now did we? :-)

    Now this IME was generated by a third party as I mentioned, using Microsoft's ImeGen.exe and UImeTool.exe tools.

    These tools no longer work in Vista and Windows 7, though as I mentioned in Behold the Table Driven Text Service, Part 10 ("Even Jove nods," Atlas shrugged, "so we'll convert their old files, now.").

    And that blog has the instructions for the third party (the one with the source, which is what is required here) to move their data from the old format to the new one.

    It was funny, though. I wrote that blog but I had forgotten about it for several responses in the thread asking about ImeGen.exe and UImeTool.exe, so even though I was already recommending the use of TableTextService.dll I had initially forgotten that a way to ease that migration existed.

    But everything can be okay now:

    Jove will nod.

    Atlas will shrug.

    And there's a way (eventually) to see that people can (potentially) get their Wubi back....

  • Sorting it all Out

    Suddenly the chair looks smarter and I am the one with the control issues

    • 3 Comments

    Just about every day I learn new things about the iBot.

    Mostly it is just consequences stuff, like when I think through logically how it works I understand why this new understanding is something that just logically makes sense.

    And of course each day, as one becomes smarter, one realizes how dumb one was the day before....

    Like way back in From I SCOOT to IBOT, #2 of ??, where I mentioned

    I decide to slip down to standard mode for speed as soon as I hit the walk outside the apartment . It's faster (up to 6.8mph!) with a bit less control, but good for wide open spaces when the best thing you can do is move through the place and on to some place interesting.

    Now if you search on the web, you will find other people making the same observation about standard function in the iBot. People pretty routinely talk about how the control of the iBot in standard simply isn't as good as the other modes.

    Now fast forward to From I SCOOT to IBOT, #7 of ?? (Unintended movement of the joystick will result in unintended movement of the product), and the diagram from the manual.

    This page:

    http://www.trigeminal.com/images/ibot_steer01.jpg

    or more specifically, the following small part of that page:

    http://www.trigeminal.com/images/ibot_steer02.jpg

    Now speaking as someone who grew up accustomed to joysticks, I know how they work.

    I have logged countless hours in video arcades, with Atari 2600 and 5200 consoles, with games on the computer, and so on.

    Look really closely at that diagram, though.

    That is NOT how joysticks work in all of that rich history of mine.

    Mind you, for short distances, the behavior of a diagonal push forward (e.g. diagonally to the upper-left) will be almost what I expected -- what I wanted was a diagonal move and what I get is a move forward and and a counterclockwise turn that if I move back to center quickly will do pretty much what I wanted.

    Though that move back to center is me compensating for what looks like the chair being a little out if my control, doing something I didn't expect.

    Which in a way is unfair of me -- judging the device's behavior based on my own joystick expectations and prejudices.

    Now in 4-wheel mode a similar effect happens, but there is also a bit of a "shuffle" that the wheels do at the same time as they turn which makes the behavior feel more like that diagonal move. So I get the illusion of the joystick working more like I expected, and can thus claim that the control is better in 4-wheel mode.

    But really both situations are just my prejudices about the joystick and what I think it should be doing, and noticing it fall short.

    All of this implies a solution, of course -- just retrain myself on this altered usage if the joystick. And whether this means I

    • only ever do backward/forward OR left/right moves of the joystick but never combine them, or
    • combine them after carefully mentally calculating the effect of subtle moves toward the diagonal and away from it

    is really up to me and how comfortable I feel with solving such ballistics problems in my head as I am riding along.

    I suspect how tired and/or drunk and/or hungover I am might turn out to be crucial variables in that equation, but only time will tell on that one.

    In the meantime, I feel a lot smarter about all this today than I did last year. And a lot smarter than those other reviewers who dismissed the iBot's control in standard function and perhaps never had the chance to realize they were wrong.... :-)

  • Sorting it all Out

    Buying the cow vs. the cut of beef you like

    • 2 Comments

    The other day a question came from someone doing some work to support MUI in their application. Their question was simple enough:

    Which one is preferred way GetFileMUIPath VS. GetThreadPreferredUILanguages?

    Although the question seems simple, the answer is even simpler.

    This is the wrong question.

    These two functions have two different though related purposes:

    • GetFileMUIPath -- Retrieves the path to all language-specific resource files associated with the supplied LN file. The application must call this function repeatedly to get the path for each resource file.
    • GetThreadPreferredUILangauges -- Retrieves the thread preferred UI languages for the current thread.

    GetFileMUIPath is best thought of as a Windows-specific implementation of the resource logic -- it assumes that even if you are not built into Windows that you are following the same rules about directories and resources as if you were. And it returns information that your application may or may not even need, depending.

    Now obviously GetFileMUIPath works at a much higher level than GetThreadPreferredUILanguages -- in fact, GetFileMUIPath even takes a flag value that instructs the function to make use of the samje list provided by GetThreadPreferredUILanguages (MUI_USE_PREFERRED_UI_LANGAUGES).

    If you do anything even remotely different in how you have laid out the files or whatever then you have to do that extra work yourself, possibly based on the list provided by GetThreadPreferredUILanguages. Or not -- it is really up to you -- entirely based on how your app's resources work.

    So asking which one to use is kind of the equivalent of asking something like:

    Which is the preferred way -- buy the live cow, or the top round sirloin cut?

    (A particularly relevant metaphor given the way the MUI (EHM-YOU-EYE) team calls themselves the MUI (MOO-EEY) team -- cows mooing seems within their expected results!)

    Of course there is no best answer here either, except based on the situation.

    Perhaps you lack the facilities to slaughter the cow. Or perhaps you preferred the ribeye. Maybe you are a vegetarian and need to make alternate plans for dinner.

    But asking which one is "preferred" under those circumstances makes no sense....

  • Sorting it all Out

    Confessions of an iBot riding fool....

    • 3 Comments

    The following story is entirely¹ true.

    The iBot naturally raises questions in people's minds, questions beyond those that are answered in Google^H^H^H^H^H^HBing² searches.

    Like what happens if someone crashes into me? Or if I crash in to someone/something?

    Now no one was willing to do the former³,  but the latter seemed easy enough.

    There was not a whole lot of room in my office⁴, but I turned the iBot toward the wall,said watch this, and crashed into the wall.

    Now the wall was maybe a foot from me.

    The chair immediately moved back and dropped down into 4-wheel mode.

    Ronen, my neighbor, was a little bit freaked out, but I reassured him I was fine.

    And what is more, the wall was completely unharmed⁵.

    My neighbor was surprised and thought the end result was pretty cool. He understood that any time the chair was in a catastrophic situation where there was some potential danger, it would move itself to a safer mode to keep me from having any damage done to me.

    He asked me a few times to make sure I was fine since the mode change was pretty fast. But truly I was quite unharmed.

    We were being kind of loud about all this, so a few curious onlookers who were passing by stopped, curious.

    We tried to explain what was going on, but it was probably easier to just show it.

    Of course what with all the boxes, things were pretty cramped in my office, so we moved this little exhibition out into the hall.

    So here I was, out in the hall, about 6 or 7 feet from the target, and I was explaining what would happen.

    Even in advance everybody thought it sounded pretty cool. The fact that I had one witness to it already just made it more plausible, and to take everyone's mind off the pain in the ass of unpacking⁴, this little demonstration seemed like the perfect way to take everyone's mind off it all and take a break.

    Now remember that I was a bit further from the wall now then I was last time when I was all cramped in by the boxes.

    Anyway, I turned to the wall and proceeded to head right for it.

    You may be able to guess at the result.

    On one side (the right side) everything was fine.

    On the left side, the end of the footrest and the side piece of the footrest went right through the wall.

    Oops!

    Everyone turned to me first -- was I okay? I was. I was just fine, and the iBot had backed up and dropped down to 4-wheel mode as expected.

    Almost as one we all turned to look at the wall, not knowing quite what to say.

    We all knew about the stories about Ballmer and Cutler putting things through walls during emotional displays. But those guys probably had whole morale budgets that could absorb costs of such things.

    A hole in the wall of the shape of a piece of the iBot right across from the office of the guy with the iBot? You don't need the NCIS folks to figure this crime out; the Barney Fife level of investigation could discern the culprit here!

    Ah well, we decided. We'll just walk away. If anyone asks, we'll tell the truth, but maybe no one will ask.

    And you know what? No one did ask.

    Then the other day, we noticed someone had filled in the hole:

    The hole in the wall, filled in 

    I decided to come clean and write this blog -- if they send me a bill then so be it.

    It was kind of dumb, and it was all predictable -- in retrospect at least.

    Though if you ignore the part that was a mistake, the interesting part was how in both cases the iBot kept me from danger or problems -- it dealt with both situations in a way that kept me safe in case of potential danger. That part was pretty cool.

    And it caused me to be much more prepared for the later situation where someone crashed into me from the side at high speed³. :-)

     

    1 -Yes in the truth and nothing but the truth, though it may not be the whole truth since I am not including last names, and I don't hqve pictures of everything.
    2 - Does this one even need a footnote?
    3 - Someone accidentally did the crash into me whole they were not looking while I was in Los Angeles recently, and the result was identical -- they were injured and I was perfectly fine. Plus I got a date with the girl I was explaining it to who witnessed it.
    4 - We had just recently moved to building 9 from building 24 and the office was cluttered with boxes, lots of boxes.

    5 - I'll admit this surprised me a little bit; I expected the usual wear and tear that you  see on walls when you are in an office like a smll scrape or smudge but there was no mark whatsoever.

Page 1 of 1 (12 items)