Musings of a tester testing test tools! : i18n

My favorite engineering practice at Microsoft

anutthara — Wed, 25 Mar 2009 22:41:00 GMT

Here at MSFT, specifically at DevDiv, we have specific testing categories called "Exit Criteria" - basically gates that you need to pass through in order to ship a release at any milestone. These criteria represent different flavors of testing that your product has to go through in order to be pronounced "fit for release". For eg: localization, accessibility, performance. At DevDiv, we have 26 different exit criteria that we need to pass through before we ship!
Now, as is the case with any "process", this one has also been the subject of passionate debate among the MSFT denizens as to the value this really brings to our product quality. I have heard arguments around how this is a lot of overhead at a very busy time in the ship cycle. Well - I have to concede that these set of tests really do not bring in a lot of bugs. Especially, criteria like "branding and copyright" - where we just test to see if the product is carrying the right trademarks everywhere, the tool names are mentioned correctly etc. Also, when criteria like "configuration matrix" is in question - this is basically the gate where we ensure our products run ok on different operating systems, different versions of CLR, different versions of IE, different bitness...some of us testers tend to go berserk and many times, products get overtested. And of course, there is this eternal danger of manual testing (which is still the best source of the most important bugs) getting undermined and sacrificed at the altar of checking off items in a checklist.

In spite of all of this, I still opine that the exit criteria proc is one of the best engineering practices I have seen here at home. If you are a tester testing a product with a customer base numbering tens of thousands or more, I am sure you appreciate the sheer breadth of things you need to cover in your test planning. The number of dimensions is really mind boggling - security, performance, localization, globalization, backward compatibility, accessibility, stress, endurance, limit...it sure gets crazy. Exit criteria really bring a semblance of sanity in this whole universe of testing. I would term this the distilled essence of the different testing dimensions that go into testing a world class product. It serves as a map to help testers navigate the maze of testing requirements and builds confidence into the quality of the product step by step.
The Globalization EC for instance, is so very well thought out - we have different configurations like Italian, Turkish, Arabic; with each configuration defined by what is different in that config and what are the kind of bugs you can expect to hit. For instance, ITA is one of the "most localized" OSes, which help catch issues where we hardcode stuff like security group names. I remember Team Build had this bug in 2005, where we had hardcoded to check if user belongs to "Administrators"! The test passed on JPN, because the admin group was still "Administrators" on the JPN OS, but it failed on German OS, since the group was "Administrateren" on DEU! Turkish has the infamous sort problems with the quirky dotted/undotted I's. Testing on Arabic configs throw up specific problems rendering bidirectional text or right-to-left UI mirroring etc. To me, this is a treasure trove of info put together by several years of testing experience. Who wouldn't want to use this cheat sheet? :-)

OrdinalIgnoreCase is not a panacea!

anutthara — Tue, 06 Dec 2005 13:57:00 GMT

There can be too much of a good thing you know! Last night as I was scanning through some code, I noticed that all the String APIs were used in the culture safe overloads so thoughtfully provided by .NET. As I silently thanked FxCop for codifying this, I also came to the conclusion that I can safely bid goodbye to all intl bugs in that code.

After a quick run though, I noticed some of the sorting orders in the UI were wrong! Ouch! OrdinalIgnoreCase was the harmless looking deceiver. In a bid to stick to the .NET guidance rules the code was relentlessly smattered with OrdinalIgnoreCase all around. Hence, no matter what the current culture was, the sorts and compares all gave the same result each time.

Ok - another explanation. There are 2 kinds of data - input and output. Say, your input data is taken in and used for processing. When you want to compare your input data with some benchmark variables defined, you will want to use our favourite OrdinalIgnoreCase in the comparison. This is purely a non linguistic comparison. In cases where you want case sensitivity to be maintained, then you can use Ordinal in the compare API. Consider a case where you want to extract the contents of a project file. Project file names are unique and case insensitive. Hence, when comparing between your user input and the project file name, you can use the OrdinalIgnoreCase in your comparison and expect it to yield accurate results.
In cases where you would like to do a linguistic comparison, where the current culture does matter to you, use the CurrentCulture value in the StringComparison object. For instance, if you would like to create a sorted list of random user input values, you must use the CurrentCulture value. In the Turkish culture I is sorted before i whereas in the English culture i is sorted before I. But if I sort using the Ordinal value, I'll get a standard order that the values will appear in, no matter what the culture of the client is. But that is not what we want! Hence, your comparisons need to take the current culture into account. Usually the output data needs to be presented in the culture that is being used by the user while input data when being processed must be culture insensitive.

That brings us back to the original topic - using OrdinalIgnoreCase in ALL cases will not rid your code of intl bugs!

Avoiding the Turkish i issue

anutthara — Mon, 05 Dec 2005 10:11:00 GMT

There are 4 i's in the Turkish language - the small dotted i(which is the same as English), the small undotted ı, the capital dotted İ and the capital undotted I(which is the same as English). Capitalization works in this way: ı.ToUpper() will yield I and i.ToUpper() will yield İ. Of course, there has to be a sort order prescribed to these too - the sort order being ı<I<i<İ Note that I<i in the Turkish culture whereas i<I in English. Hmm - do you smell bugs here? In fact, the "turkish i" issue has earned enough notoreity to actually have dedicated testing with these characters on Turkish setups.

Why do these bugs arise? Comparison based on default methods is a common example. String.Compare("mine".ToUpper(), "MINE") will not yield a zero on a Turkish culture. Obviously because the small dotted i translates to İ and not I on capitalization. Using ToUpper(), ToLower() to normalize the string for comparison will always yield undesirable results on different cultures.

We are currently testing Team Build on a Turkish setup and we are trying out a lot of cases around the Turkish i, esp sorting. The turkish eye bugs that I have seen in the past while testing other products are invariably due to incorrect usage of String APIs. Here is a really cool paper on handling string APIs that gives a specific set of dos and donts wrt string APIs. I bet several people are surprised to know that using InvariantCulture can actually lead to a couple of issues even in places that seemed tailor made for using InvariantCulture! Of course, the new StringComparer is super cool and offers as much clarity as possible about what to use where. Checking these rules while doing static analysis is certainly going to save a lot of pain in i18n issues.

My first web chat session

anutthara — Thu, 01 Dec 2005 15:07:00 GMT

I just finished my web chat session. We had nearly 50 qs asked in an hour or so. I must say it brought back memories of exams...

As I went to the conf room with a borrowed laptop, I was nervous. Ok - I was a little more than nervous - bordering on jittery you can say. At least I was before time - I had a good 20 mins to go until the chat began. I would collect my thoughts until then I thought. As I put the network cable into my laptop, the machine just would not connect to corpnet. Troubled as I was, I had to drag my manager out of his room to fix my conn issues. It turned out I was connecting to a dead port and just had to switch the port! Now, I understood that nervousness was beginning to play on me. As I started up the machine, I realized I had not yet installed chat manager on this system. Ouch! I tried TS-ing into my dev machine, but for some reason, the perf was so goddamn slow that I could actually hear the letters being rendered as I typed! I tried installing chat manager on the laptop itself, but the msi being in Redmond , it took me a while before it was installed on the machine. In the middle of the installation, my laptop hibernated without a warning! Battery low! As I tried plugging in the power supply, I noticed that the laptop's plug had round pins whereas the sockets had flat pins. No adapter in sight. Determined to remain calm, I searched around a few offices and finally got an adapter that would interface with my plug and socket. Finally, I was ready to go no more than 20 seconds before the chat began.

As the moderator started introducing us, I felt a twang of sharp fear. Expert? He is actually introducing me as an expert; but what if I can't answer a q? What if I forget answers to basic qs like what is internationalization? That was ok - I could always msn my way through ( that's right - I did not say Google my way through :P) Yeah right! I was supposed to msn search? Like the participants can't do that themselves! Ok - if I forgot, I could reassign the q to the other experts in the room. They would not forget unlike me. But what if I can't answer any q at all? Yikes! This is exactly the way I felt before each exam! Why did I even sign up for this?

As I was lost in my paranoia, I realised that the first q had already been asked. That is simple - I can answer that, I thought. It was a straightforward q on what internationalization of s/w is. As I finished typing the answer to that one, I saw a couple of more qs that I wanted to answer - qs related to asp.net i18n. Those were interesting. As more and more qs kept coming, I did not have the time to re-experience my fears! I took some fairly nice qs and was pleasantly surprised at how interested the participants were in newer features of internationalization and all aspects of it! Qs on satellite dlls, localizing static content on web sites, using multiple cultures in a single page, differences between cultures, Unicode encoding schemes, .NET API support for the same - they all kept coming. A few tricky ones were asked too - some good gotchas in i18n. As the qs kept flowing, I did not realize that we had already spent an hour :) Sweet! I actually made through the entire chat session without being totally foxed by any q! Er - I don't want to make that sound like a miracle, but that was really the way I felt when it began. The last q was on the difference between localization and localizability. Wow! I wouldn't have missed a chance to answer that one (readers of my previous blogs know why ;-)

Now that my first time went off quite decently, I can venture to say that I really quite enjoyed it. I am looking forward to another web chat again...

I have an internationalization web chat tomorrow

anutthara — Wed, 30 Nov 2005 14:24:00 GMT

at http://www.microsoft.com/india/msdn/chat/default.aspx

Join the chat if you want to know about what internationalization is all about and how you can develop your apps for everyone around the world. :)

Localization and localizability - what's the difference?

anutthara — Fri, 25 Nov 2005 15:15:00 GMT

As Michael Kaplan, our very own i18n guru noted in my previous blog, localizability is not the same as localization.

Designing a software's code and resources such that they can be localized and presented in other languages thus making it usable in other languages is called localizability. This means, I need to design my product in a way that if tomorrow I want an international developer to use it, the software must be easily lendable to localization. Localization is the actual process of localizing the resources used in the software, thus making the product ready in a new language.

Umm...imagine I weave a yard of cloth. Suppose I want to make strips of different colors from the cloth. I will have to ensure that the cloth is actually ready to be dyed. I have to check if the cloth has been weaved with the right consistency and has an even surface that lends itself to dying. I might want to soak it in water and dry it once just to ensure it absorbs the dye well. Then when the cloth is ready to take color, I go ahead and dye the cloth in different colors. The process of making the cloth ready to take dye properly is analogous to localizability whereas the process of actually dyeing the cloth in different colors is analogous to localization. Suppose I had made the cloth out of a material that cannot be dyed, then I have chosen the wrong raw material. It means that in order to make a blue piece of cloth I have to weave a separate cloth with blue thread and a pink piece of cloth a separate cloth from pink thread! I cannot dye a blue cloth pink or a brown cloth blue. There is no common cloth that I can use to create different colored clothing at the end of weaving! This is indicative of an unlocalized product where the core code cannot remain constant while the presenting language changes.

Time for an illustration. Imagine a simple winforms app where I have a button and a textbox. The button has text called "Please Click" and then the string "Hello World" appear on the text box upon clicking the button. Suppose I hardcode the strings "Hello World" and "Please Click" in my win forms code. The app will work fine when you expect English strings. But what if I want a Japanese developer to use my app? I want to display the 2 strings in Japanese. What do I need to do? Go back to my code, change the hardcoded strings to Japanese strings and recompile my app. But wait, then this wont work for English developers. Oh! What a mess. Imagine if real world products like VS would have to do this with millions of strings hardcoded in their sources...

But hang on...I have a plan. What if I moved these strings to resource files? Then I just need to include separate strings for each language I want to localize my product in and selectively pick up the right resource while displaying the form. That way, I do not need to rebuild my app each time I want to see my app in a different language either. That sounds cool. That is one aspect of what localizability is. Designing your software such that localization of resources becomes very easy and does not require source code to change is what localizability is. Later, when you go and actually translate all of the strings that you use in your product to other languages and include those strings in resource files, you are actually doing localization of your product.

Obviously localization requires the localizer to be fluent in both the source and target languages. So, we cannot have the product team doing localization of their products for all given languages we want to ship in simply because we may not have folks on the team who are fluent in all of those languages. So, at MSFT we have a separate localization team that does the actual localization of resources. Albeit, it is the job of the product team to ensure that the product is localizable. We have to run localizability tests on the product to ensure that all localizable resources are moved to a separate resources file that is marked for localization and no localizable strings are hard coded in the product code. We also need to check if UI has been designed in a way that if larger fonts used in other languages or longer strings that might be the equivalent of shorter English strings can be properly accomodated and rendered.

I guess that draws a little clearer line between localizability and localization. :)

Globalization? Now, why would that figure on a tech blog?

anutthara — Thu, 24 Nov 2005 13:21:00 GMT

Some months back, I was talking to my brother about a presentation that I was about to give on globablization to my team and he asked me if I had read Friedman! Then it struck me that globalization may be a very common concept within Microsoft but it sure wasn't so common outside!

So, what is globalization? There are millions of people all over the world who do not know English or may not be comfortable speaking/writing it. Of course, several of them are very techie and computer savvy. Thanks to all the different languages that operating systems are available in, we have now democratized choice of language for developers all over the world. A Chinese developer need not write or even read a single word in English to develop any app on his system. Likewise a Spanish developer might choose to develop his apps in Spanish rather than in English. MS Windows is released in several languages - Japanese, Korean, French, Italian, German, Spanish, Traditional Chinese, Simplified Chinese, Arabic, Hindi and many more. Whew! That is a large list.

Ok - so I have an OS in Hindi. But what about my products? Of course I would prefer that my applications and products I use be in Hindi too, but it would not be very realistic to expect every single software in the world to be developed in all possible languages. So, sometimes developers do settle for English products, but install it on localized Operating Systems. Of course, I expect this to work. This leads us to the definition of globalization. Designing software for the input, display, and output of a defined set of supported language scripts and data relating to specific geographic areas is the process of globalization of software.

But, shouldn't software work by default on whatever your OS language is? How does it even matter whether I use Hindi or Japanese on the OS? Ah! How I wish this was the case...it would rid me of half my testing work! :) There are several things dependent on the OS - stuff like currency, date/time, number formatting, bidi awareness, mirroring, IME support, etc (which we will go into detail in later blogs) The product we build should respect settings for all of the above factors and should be flexible enough to behave in a way that the current OS expects it to behave. As a simple example, suppose I have a Japanese system, then dates are rendered in yyyy/mm/dd format whereas on normal English-US machines, this would be mm/dd/yyyy. If my product displays dates, it should display in the former format on a Japanese machine and the latter on an English machine. Suppose your product takes filenames as input. It should work fine even when I provide filenames in Hindi. This calls for internationally aware code to be written while developing the product. In my coming blogs, we will explore how .NET supports this kind of internationalization automatically in their APIs.

Right. I have a German OS on my machine. What use is this to me if I do not have my products in German too? Yes, of course we realize that and that's why we are shipping VS in German ;-) Coming to the definition of localizability then. Designing software code and resources such that said resources can be localized and presented in other languages thus making it usable in other languages is called localizibility of software.

Er - what was the difference between globalization and localization again? Let me cite an example. Suppose you have a Japanese OS installed on your machine. Now you install an English word processor software on your system. Note that the menu options and the word processor itself will be in English. If I can type a Japanese document and save the file and re-open appropriately, then the word processor software is globalized. Now, if I install a Japanese version of the word processor, where the menu options and the help are all in Japanese and obviously I can type, save and re-open a document in Japanese, then the processor software is said to be localized.

At Microsoft, we test the product we ship throughout the product cycle in several stages. We have concerted efforts on globalization testing and localization testing in every milestone. More on how we do that in coming blogs.

Till then, let me end my globalization blog with a localized goodbye :D

Au revoir...