Dr. International

When does 0 + 4 = 1?

2006-02-21T20:06:00Z

I received a good question today from a developer who asked:

Why should one not use pointer arithmetic for string length calculation, access to string elements, or string manipulation ?

The main reason pointer arithmetic is not a good idea is that not all characters are represented by the same number of bytes. For example in some code pages (like Japanese) some characters (a, b, c ... and half-size katakana) are represented by one byte, while kanji characters are represented by two bytes. Thus you can not compute where the fifth character in a string is by simply adding 4 to the beginning byte pointer of the string. The fifth character could be as much as 8 bytes offset from the beginning pointer (5 kanji characters - 2 bytes each)

Even using Unicode you can not assume that each character is 16 bits (or a word), because if you are using the UTF-16 encoding, then some characters (supplementary characters) are represented by surrogate pairs (two 16 bit words). Some single characters are represent as a base character (like "a") plus one or more combining characters (such as some diacritic "^'). And if you are using UTF-8, then a character can be 1, 2, 3 or 4 bytes long.

That is why it is better to use APIs that are aware of these differences to walk through a string than to use pointer arithmetic. (See: StringInfo Class) In globalization best practices, you never assume that all characters are the same byte size.

Because sometimes, 0 (beginning pointer to a string array) + 4 (a high surrogate and a low surrogate [4 bytes]) = 1 (a single Unicode character).

Transliteration Utility Released

2006-02-07T00:44:00Z

In the world today, several languages (like Serbian and Bosnian) use two or more different scripts (like Latin and Cyrillic) to represent their written languages. There are times when people would like to either type in one script but produce the other or be able to convert from one script to the other without retyping in everything. The free as-is Transliteration Utility (TU) tool was just released via a joint effort between the Local Windows Solutions and Programs (LWSP) and Speech and Natural Languages (SNL) groups to meet this need.

To use TU for instant transliteration, the Main Form is the appropriate user interface to see real-time transliterations of typed/pasted input text or text of an entire file. Before any transliteration can be applied to input text, you select a transliteration module, which best meets your needs (there are nine that ship with TU).

To transliterate a larger amount of data, spread over several files, a Command Line Interface is available to be used in batch-mode.

One fo the best features of this Utility is that it comes with its own Development Console that allows one to author, edit, and/or test transliteration modules, which uses a simple text format to create TU modules. Thus you are not held to only using the 9 modules that come with the tool, but open to create as many as you need.

To read more about the tool and to download it go to:
http://www.microsoft.com/globaldev/tools/translit.mspx

Make sure that you use the feedback pointer to give us suggestions on how to improve the tool.

Dr. Intl

Take me home (I really mean home where I live).

2005-12-05T22:00:00Z

Got an interesting questions from someone who bought there computer with Windows XP loaded on it in the United Kingdom. They ask, "My desktop PC was purchased in UK but now I live in US. However microsoft.com and google.com still treat me as having a GB locale. How do I force US locale?"

What is happening is that Internet Explorer is looking at the language preference variable found in IE's tool menu. Most likely it is set as English (United Kingdom) -- en-GB, instead of English (United States) -- en-US. Thus the server believes the computer is located in the UK instead of the US. The answer is quick simple. All they have to do is to remove English (United Kingdom) and replace it with English (United States). That way the servers will assume they are living in the US.

Actually they should be looking at the GeoID of the client which the location that is set in the Regional Tab of the Regional and Language Options Control Panel applet. There is a win32 API that can query the system for its GeoID. The call is GetGeoInfo.

English web-pages on a Japanese version of Windows

2005-11-17T23:57:00Z

Got an intersting question the other day on viewing English Pages on a Japanese version of Windows. It said, "I'm English speaking in Japan. My laptop has Windows XP Professional (Japanese). I've a major problem with Microsoft websites – most of the pages come in Japanese. For example - when I access Windows Live Ideas http://ideas.live.com/ I get Japanese page, and I have no idea how to view the English page. www.microsoft.com opens in English, but when I click on Support, it opens Japanese page. This is very difficult as I have very limited knowledge of Japanese. Help."

What was happening was because he was running the Japanese version of Windows, his IE default language setting was set to Japanese. Thus each time he went to a Microsoft web site, the web site ask the browser what language it wanted the content in and the broswer said "Japanese". To fix this I showed them how to set the default content language to English.

I am assumed they were using IE 6. So here is how I told them they would be able to see English pages

1. Open Internet Explorer.
2. Click Tools.
3. Click Internet Options.
4. Click the General tab, and then click Languages.
5. Click English, and then click Move Up to move English to the top of the list of languages.

This way, when asked which language to serve information in, the browser now tells the server English. You can do this for any langauge you want, but realize that some we sites may not ask this question nor if the do they may not have content in the language you set. Thus is will try to deliver the best it can.

Global Development and Deployment Conference 2006?

2005-11-14T23:16:00Z

In 2004, Microsoft held its first ever Global Development and Deployment Conference (GDDC 2004). It was dedicated to the development and deployment of international solutions using Microsoft technologies. It featured sessions on creating solutions that are localizable and easily adapted for sale and deployment all over the world. Besides the classes, the other feature we had at the conferences that went over well was we had many Microsoft people there that either dealt with these issues or were the developers of the technology.

With the release of many products over the next 12 months, we believe that it is time to hold another GDDC in 2006. Of course it will cover international development and deployment of products like Windows Vista, Visual studio 2005, SQL 2005, Office 12 and others, but to help meet your needs, we would like to hear from you on what subjects you would like to see at a conference like this.

Please send your input to my e-mail address at drintl@microsoft.com. I would appreciate it very much.

Dr. Intl

Name that Unattended Installed TimeZone

2005-10-28T22:07:00Z

I got a question today if it was possible to set the TimeZone of a computer via the Unattended installation process.

Well the answer is yes, but it is sometimes hard to find out how. So, I thought I would drop this here to help make it easier to find. Below you will see an example on how to do this. You will also find a table that the has the values listed. The example also mentions values for the RegionalSettings section of Unattended. You can find out what other international related values you can set and how to do it at my globaldev website.

Hope this helps.

EXAMPLE:
[GuiUnattended]
; Set the TimeZone. For example, to set the TimeZone for the
; Pacific Northwest, use a value of "004."

 Be sure to use the
; numeric value that represents your own time zone.

; To look up a numeric value, see the Deploy.chm file

; on the Windows XP Professional CD. The Deploy.cab file

; is in the \Support\Tools folder.

TimeZone = "YourTimeZone"

OemSkipWelcome = 1

; The OemSkipRegional key allows Unattended Installation

; to skip RegionalSettings when the final location of the

; computer is unknown. Also,If in the [Unattended] section

; you set OemPreinstall to Yes and provide values for the

; [RegionalSettings] section, set OEMSkipRegional = 1 to

; ensure that Setup completes without prompting the end

; user for regional information.

OemSkipRegional = 1

Time Zone Table

Index	Zone
000	International Dateline
001	Samoa
002	Hawaii
003	Alaskan
004	Pacific
010	Mountain (U.S. and Canada)
015	U.S. Mountain: Arizona
020	Central (U.S. and Canada)
025	Canada Central
030	Mexico
033	Central America
035	Eastern (U.S. and Canada)
040	U.S. Eastern: Indiana (East)
045	S.A. Pacific
050	Atlantic (Canada)
055	S.A. Western
056	Pacific S.A.
060	Newfoundland
065	E. South America
070	S.A. Eastern
073	Greenland
075	Mid-Atlantic
080	Azores
083	Cape Verde Islands
085	GMT (Greenwich Mean Time)
090	GMT Greenwich
095	Central Europe
100	Central European
105	Romance
110	W. Europe
113	W. Central Africa
115	E. Europe
120	Egypt
125	EET (Helsinki, Riga, Tallinn)
130	EET (Athens, Istanbul, Minsk)
135	Israel: Jerusalem
140	S. Africa: Harare, Pretoria
145	Russian
150	Arab
155	E. Africa
160	Iran
165	Arabian
170	Caucasus Pacific (U.S. and Canada)
175	Afghanistan
180	Russia Yekaterinburg
185	W. Asia
190	India
193	Nepal
195	Central Asia
200	Sri Lanka
201	N. Central Asia
203	Myanmar: Rangoon
205	S.E. Asia
207	N. Asia
210	China
215	Singapore
220	Taipei
225	W. Australia
227	N. Asia East
230	Korea: Seoul
235	Tokyo
240	Sakha Yakutsk
245	A.U.S. Central: Darwin
250	Central Australia
255	A.U.S. Eastern
260	E. Australia
265	Tasmania
270	Vladivostok
275	W. Pacific
280	Central Pacific
285	Fiji Islands
290	New Zealand
300	Tonga

www.microsoft.com/globaldev INPUT

2005-10-21T21:43:00Z

Most of you know that besides this blog, I have a website www.microsoft.com/globaldev dedicated to the dispersing of international information for developers, IT pros and end users. I would like to ask you the readers what you find positive about the site, what at the site really has no value for you and what could we add that would improve the site. Please send your comments via my feedback page. Looking forward to hearing from you.

"Help, I'm locked out of login by Thai keyboard"

2005-10-19T22:07:00Z

A while back I got the following plea for help:

My default keyboard language has been English for login. Some how it was switch to Thai and I do not have a Thai user login. I don't know how to switch back to an English language keyboard so I can log back in my desktop. I have tried using the alt + shift keys to switch the language but it didn't do anything. My OS is window XP Home.

The reference to "alt + shift" is about the function that allows you to switch between multiple input locales (language keyboards and IMEs). This is the default behavior every where in Windows XP (both Home and Pro) except for the login screen. There is a way to set up multiple input locales for the login screen (see: Alternate Language Keyboards for Login in my Dr. International columns), but this would not help the plea-for-help because they still needed to login in order to use this.

So I suggested that they use the old stand by that is commonly used for entering in accented characters from an English keyboard (i.e. using the <alt>+<keypad numbers>). What I mean is to enter say "Pat" as the user name, go to "User Name" field of the login screen. Then to enter in "P", while holding down the <alt>-key on the number-pad press "0", "0", "8", "0" then let up on the <alt> key. This should insert the upper case "P", then repeat using "0", "0", "9", "7" for lowercase "a" and "0", "1", "1", "6" for a lowercase "t". What you are doing is entering the 1252 codepage decimal value of each of these characters. I explained that they can use this method for both your user name and password.

After writing the explanation, it got me thinking that this may be an interesting way create a strong password. If you have the English keyboard as the default for login, you could add an accented character to your password, say É (<alt>+0201). Unless some one knows what that character is and how to input it using an English keyboard it would be hard to gustimate.

Also, this will work for any keyboard you use for your login screen default. The reason is no matter what keyboard you use for login, the <alt>+<keypad numbers> always inputs 1252 codepage characters. So you could use an obscure 1252 code page character that your keyboard does not have, thus making this a stronger password.

Dr. International's Friend at PDC '05

2005-09-14T00:50:00Z

Have questions about how globalization, localization and Multilingual User Interface (MUI) will look like in Windows Vista? Look up my friend Edward Ye who is in the Fundamentals area in the Track Lounges thoughout this week. Ed is a program manager for MUI features in Windows since Windows 2000 and can answer your questions about Windows Vista MUI and how globalization and localization are being handled in this upcoming version of Windows.

Orlando Day 4 (Thursday)

2005-06-09T07:13:00Z

Two more days to go in Orlando. Soon Dr. International will be back in the Pacific Northwest, just in time to miss Tropical Storm Arlene. Even though Arlene is not forecasted to hit Orlando, Dr. International still decided to play it safe, especially now that Dr. International became famous. The Pod Casting Crew came by and did a detailed interview at the cabana. It seems that the microphone was broken, but you can still understand most of it.

Tomorrow will be a huge day for Global Development & Deployment Virtual Track. We have DEVC19 - Advanced Localization Architecture (9AM, Cabana 8), DEVC21 - Custom Cultures in Whidbey (10:45AM, Cabana 6), MSG-369 Supporting East Asian Languages in Global Exchange Deployments (1PM, N230), BAP347 - MBS Azapta: a truly Global Business Application (1PM, N210B), and last but not least DBA319 - Databases for the World (2:45PM, N210B).

Orlando Day 3 (Wednesday)

2005-06-08T12:56:00Z

We have a winner! Yesterday afternoon Mickey Gousset came up to the Dr. International Cabana here at TechEd in Orlando. He had seen the challenge in the blog and collected the names of all the members of Dr. International. Mickey was happy to find out that he was the first person who got all the names right. He won a nice crystal globe award for this excellent detective work.

Our cabana is well visited and lots of people find out about Dr. International for the first time. We also took notes on what we need to add to our website - there will be lots of work for us in the next few months adding all the requested content. If you are not at TechEd, but you think we really should add some specific content, send us an e-mail.

Tomorrow we have two really cool talks lined up in our virtual track. Achim Ruopp and Matt Ayers will tell you everything you wanted to know about custom cultures in the .NET Framework. You don't even know what a custom culture is? I can tell you that much that it doesn't have anything to do with opera or classical music - it's all about locales... Anyway, if you know what a custom culture is, or if you want to find out the talk is DEV323 - .NET Framework: Think Global! Custom Cultures and International Data. The talk will be held on 6/09 at 1:30-2:45pm in S310G. The other talk for Thursday is the highly anticipated talk by Michele Leroux Bustamante on ASP.NET 2.0. Michele will show you all the great new features which make you international life so much easier. If you work with ASP.NET this talk is a must-see event. WEB326 - ASP.NET 2.0: Going Global Gets Easier! New Localization Features in ASP.NET 2.0 will be held on 6/09 at 3:15-4:30 pm.

Orlando Day 2 (Tuesday)

2005-06-08T08:08:00Z

Yong Rhee had some very good questions come at his Deploying Windows MUI (Multilingual User Interface) breakout session. One of the best questions, is one I get all the time. What is the difference between a regular version of Windows and the MUI version? Let it first be noted that Windows XP English (as well as any other version of Windows XP) has multilanguage support right out of the box. What I mean by this is that Windows XP has everything you need to display and create text in multiple languages (including Japanese, Chinese, Arabic, Thai, Hindi, as well as German, French, Russian, Spanish and over 100 other languages).

What the MUI technology adds is the capability of the user to choose what language to display the operating systems menus, dialogs, system messages and help. That means that you can have one machine that can behave as if the English, Japanese and German localized versions where installed on it at the same time. This allows large corporations to roll out the same worldwide image with a single install job. Local users can then select the user interface language or it can be set by Group Policy for Organizational Units. To find out more about MUI version of windows go see my FAQ at: http://www.microsoft.com/globaldev/DrIntl/faqs/lipfaq.mspx

Speaking of MUI, today, Wednesday June 8, Michael Jorden (no not Michael JordOn) is speaking about the MUI capability of Office and how to deploy this along with the Windows MUI technology to make management of the multilingual desktop in a global enterprise more cost effective. His session is DSK345 -- Deploying Office 2003 in a Multilingual Environment and it will presented Wednesday, June 8 from 3:45-5:00 pm in room N220 A. Hey maybe we will see you at the presentation.

Orlando Day 1 (Monday)

2005-06-07T09:02:00Z

Whoa! There are a lot people here and this place is big. I am going to get my exercise going from one break-out session to another. Today we had two breakout sessions in our International Virtual track. One was given by Michael Kapland on SQL Internationalization and then Houman Pournasseh and Russ Rolfe gave one on the process of Internationalizing your Product Cycle Model. I know we appreciated all of you that were able to come. We would like to see many more of you.

On Tuesday is a session at 1:30 given by Microsoft employee Yong Rhee on how to Deploy Windows XP in a International Environment (Breakout session DSK350 in room N 210 B). I think you will find this very helpful to all of you who are in the process of rolling out Windows in a global enterprise. Espically the ones that have to deal with many world locations and different languages besides.

Also don't forget to stop by the Dr. International booth in the Community Cabana. We have many experts there who can help you with your international questions. (Don't forget that there still is the contest going on about reveling who Dr. International is.)

See you in the Breakout sessions or at our Dr. International Clinic.

Orlando Day -1 (sunday)

2005-06-07T07:53:00Z

Well I have made it through the first day here at Tech-Ed 2005. Sunday was when all the Pre-Conference tutorial were given and Houman Pournasseh and Russ Rolfe gave a 5 hour overview on Developing International Software. One of the good questions that came out of that session was "So what does Microsoft mean when it uses the words "globalization", "localization" and "internationalization".

The answer given was the following graphic plus these definitions:

Internationalization: The process of creating a product that is world-ready and customized for a given language and regional market.

World-Readiness: A product that is properly globalized, simple to customize and easily localizable.

Globalization: Designing software for the input, display, and output of a defined set of Unicode supported language, scripts and data relating to specific locales and cultures.

Localizability: Designing software code and resources such that resources can be localized with no changes to the source code.

Customizability: Designing software that is componentized and extensible to allow for replacement, addition and/or subtraction of features necessary for a given market.

Localization: The process of customizing or translating the separated data and resources needed for a specific language-locale pair.

Market Customization: Tailoring a products functionality to meet the needs of a given regional market.

This I know is very generalized and I will see what I can do to add more clearificaiton on http://www.microsoft.com/globaldev when I get back from Tech-ed.

Which Unicode Version Is Supported by Windows XP and Windows 2003 Server?

2005-06-01T14:23:00Z

Hello Everyone. Recently I received this question from someone (you can send me mail too at drintl@microsoft.com). I thought I’d share my response here because, well, I guess that’s what bloggers do from time to time.

Hi Dr. International,

I need the following information on Windows XP and Windows 2003 servers.

a) Which version of Unicode is supported by Windows XP and Windows 2003 Server?
b) If it supports Unicode 3.1 and above does it support all the surrogate characters defined in Unicode 3.1 and above?

Hello,

I don’t mean to sound like a politician, but in order to answer your question, we need to define what you mean by “support”.

Currently Windows XP and Server 2003 support the Unicode encoding schema including surrogates. This means Windows XP and Server 2003 can process Unicode 3.0 supported characters correctly. It also means that it can process Unicode 4.0. What I mean by process is that Windows XP and Server 2003 both internally handle the encoding without changing or losing any of the text.

Processing characters and displaying characters are two different things. So, if your question is, “Can Windows XP or Server 2003 display all of the characters supported in Unicode 3.0 or 4.0?” then the answer is yes, if a font exists for the character. However, Windows XP and Server 2003 do not ship with fonts that contain glyphs for all of the characters in Unicode, but if someone develops a font to display those characters, the functionality to use that font is already built into Windows XP and Server 2003.

One other area when it comes to support is the existence of a shaping engine. In some scripts like Arabic and Thai, the character's shape changes depending on what characters either immediately precede or follow it. Thus you need a shaping engine to check the context of where the characters is used to make sure the right shape displays. There are some scripts that are part of Unicode that even if a font was developed, Windows XP does not have a shaping engine to figure out which shape to display and would only show the nominal shape.

So I come back to your original question. What version of Unicode does Windows XP or Server 2003 support Unicode? The answer is that it depends on what you mean by support. Will that part ever change? Yes, we are investigating what it takes to add new fonts and shaping engines to display scripts not currently supported. You may ask “Which scripts, and when will they be supported?” The answer is that we are looking at adding new scripts, but currently we have nothing definite we can announce.