Loneliness of the long-distance linguist. : locales

A to-do list.

KieranS — Sat, 29 Sep 2007 00:21:00 GMT

The last several months have been packed with planning our next set of features in the international space -- talking to customers about what they need and what they want, looking at the teams we have and seeing where our skills are up to the challenges ahead and where we need improvement, and looking at the initial Vista feedback as it begins to roll in. We're seeing some good stuff and some bad stuff. A few themes:

People want to use strings: We've been pleased at the initial adoption of our name-based NLS APIs both in Windows and in .NET. Most existing components still rely on LCID-based support, but new components are increasingly starting to rely on names, and even several existing components are starting to plan for a migration to strings. This is crucial as the rate of custom locale adoption will only increase over the next few years, and in order to benefit from the full range of locales that a user has installed, it will be vital that applications ask for international support using strings rather than LCIDs. Legacy LCIDs aren't going away any time soon, but there will come a time when new locales are identified with strings and strings only.

But strings have to be reliable: As we have seen more than a few times in the last several years, sometimes there are changes in the world situation that require changes to software. A geography that is united as one jurisdiction today may vote to become distinct nations in a month or a year or ten years. We have to make it easy for developers to call us for reliable globalization support for customers even when borders shift and the customers' country of residence is updated. We have done some things well here and other things not so well, so figuring out how to handle these situations at an infrastructure level is one of our key focuses.

Locale Builder is great, so where can I get it? We've gotten a lot of great questions about Locale Builder and we're seeing a number of customers interested in custom locale technology. We're seeing questions from customersacoss the spectrum: from OEMs, from application developers who need to deploy custom locales with their applications, and from users whose languages and regions do not have existing locale support. The initial beta of LB expired, but you can get the current version on the Microsoft Locale Builder download page. Please let us know what you think.

The globalization support that people have today is not enough. People are using what Globalization Services provides today, but all the time we're getting requests for more. In particular, people want more core linguistic support in the OS. This includes updates to our sorting functionality -- we have to follow through on the versioning story that we've begun building -- and also requirements for linguistic functionality that developers cannot get from the operating system at all today.

We have an ambitious list of stuff that we need to work on, but it's also been exciting to hear from customers what is and is not working today. All our work for improvements to existing functionality, or the introduction of new functionality, has come directly from the feedback that we've received. We want to know how you're using our stuff and ways that the experience could be better. We're also interested in whether we have any gaps in existing support that are blocking you in some way.

Information central.

KieranS — Mon, 05 Feb 2007 21:24:00 GMT

Just in time for the Vista street date, the Vista International Support Portal is live at www.microsoft.com/globaldev/vista/vistahome.mspx! It contains lots of information about the locales, language packs, LIPs, fonts, and keyboards that we ship, and it makes a handy reference to figure out which languages and regions have support in Vista. You can also visit the portal to find free downloads of several of the internationalization tools that we provide, including the recently updated Microsoft Keyboard Layout Creator, the LocaleBuilder, the Microsoft Transliteration Utility, and the RC1 version of the Microsoft Phonetic Input Tool. Check it out and let us know what you think!

What comes around goes around.

KieranS — Fri, 17 Nov 2006 22:20:00 GMT

Yes, it's been a while. Since I've last blogged, we've shipped Vista and I've gone on a long vacation to Hawaii. Personally I find both of these things very exciting. It's understandable if you care more about the first than the second. :)

But I'm back from vacation and into planning mode for our next set of requirements and projects, so if as you're using Vista you find stuff about our international story that's missing/broken/just plain wrong and you haven't already told us, please let us know.

You might also like to check out the article that Shawn Steele and I wrote for MSDN magazine, Locale Hero: Enable Global Apps With Locale Builder And Windows Vista, now available in the December issue. The article gives best practices and common pitfalls to watch out for in designing custom locales, and it also provides some guidance on how to develop globalized apps that are equipped to take advantage of the full range and variety of locale data that you could see. If you have comments or questions regarding the material in the article, Shawn and I would love to hear what you think.

Oh, the enumeration? Part two.

KieranS — Tue, 03 Oct 2006 23:56:00 GMT

A few days ago, in Oh, the enumeration!, I wrote about some changes to the enumeration of our Spanish Traditional sort locale that would be new for Vista. What I wrote is that callers to EnumSystemLocales would have to pass the LCID_ALTERNATE_SORTS flag to return 040a (Spanish Traditional sort) in Vista. Well, based on customer feedback, we have decided to revert that change. That is, EnumSystemLocales will continue to behave as it behaved in XP, returning 040a even without the LCID_ALTERNATE_SORTS flag.

However, the name-based API counterpart, EnumSystemLocalesEx, will require the LCID_ALTERNATE_SORTS flag in order to return es-ES_tradnl, which is the locale name identifier for Spanish traditional sort (040a). That means that the LCID-based API and the name-based API are not strictly in parity, with EnumSystemLocales respecting previous behavior and EnumSystemLocalesEx introducing the desired behavior -- in which Spanish Traditional sort is properly treated as an alternate sort locale -- instead.

From Afrikaans to Yoruba.

KieranS — Thu, 28 Sep 2006 03:09:00 GMT

Today someone asked me whether there is anywhere on the internet that customers can find the Vista locale list. Several of us have blogged about custom locales and the ways in which customers might use them, but I wanted to post the full list of built-in Windows locales here so that is is convenient for people to find. The following is the full list of locales that went into the recently available Vista release candidate, along with their LCIDs and string-based identifiers:

0436	af-ZA	Afrikaans	South Africa
041c	sq-AL	Albanian	Albania
0484	gsw-FR	Alsatian	France
045e	am-ET	Amharic	Ethiopia
0401	ar-SA	Arabic	Saudi Arabia
0801	ar-IQ	Arabic	Iraq
0c01	ar-EG	Arabic	Egypt
1001	ar-LY	Arabic	Libya
1401	ar-DZ	Arabic	Algeria
1801	ar-MA	Arabic	Morocco
1c01	ar-TN	Arabic	Tunisia
2001	ar-OM	Arabic	Oman
2401	ar-YE	Arabic	Yemen
2801	ar-SY	Arabic	Syria
2c01	ar-JO	Arabic	Jordan
3001	ar-LB	Arabic	Lebanon
3401	ar-KW	Arabic	Kuwait
3801	ar-AE	Arabic	U.A.E.
3c01	ar-BH	Arabic	Bahrain
4001	ar-QA	Arabic	Qatar
042b	hy-AM	Armenian	Armenia
044d	as-IN	Assamese	India
082c	az-Cyrl-AZ	Azeri (Cyrillic)	Azerbaijan
042c	az-Latn-AZ	Azeri (Latin)	Azerbaijan
046d	ba-RU	Bashkir	Russia
042d	eu-ES	Basque	Basque
0423	be-BY	Belarusian	Belarus
0445	bn-IN	Bengali	India
0845	bn-BD	Bengali	Bangladesh
201a	bs-Cyrl-BA	Bosnian (Cyrillic)	Bosnia and Herzegovina
141a	bs-Latn-BA	Bosnian (Latin)	Bosnia and Herzegovina
047e	br-FR	Breton	France
0402	bg-BG	Bulgarian	Bulgaria
0403	ca-ES	Catalan	Catalan
0404	zh-TW	Chinese	Taiwan
0804	zh-CN	Chinese	PRC
0c04	zh-HK	Chinese	Hong Kong SAR
1004	zh-SG	Chinese	Singapore
1404	zh-MO	Chinese	Macao SAR
0483	co-FR	Corsican	France
041a	hr-HR	Croatian	Croatia
101a	hr-BA	Croatian (Latin)	Bosnia and Herzegovina
0405	cs-CZ	Czech	Czech Republic
0406	da-DK	Danish	Denmark
048c	prs-AF	Dari	Afghanistan
0465	dv-MV	Divehi	Maldives
0813	nl-BE	Dutch	Belgium
0413	nl-NL	Dutch	Netherlands
1009	en-CA	English	Canada
2009	en-JM	English	Jamaica
2409	en-029	English	Caribbean
2809	en-BZ	English	Belize
2c09	en-TT	English	Trinidad
0809	en-GB	English	United Kingdom
1809	en-IE	English	Ireland
4009	en-IN	English	India
1c09	en-ZA	English	South Africa
3009	en-ZW	English	Zimbabwe
0c09	en-AU	English	Australia
1409	en-NZ	English	New Zealand
3409	en-PH	English	Philippines
0409	en-US	English	United States
4409	en-MY	English	Malaysia
4809	en-SG	English	Singapore
0425	et-EE	Estonian	Estonia
0438	fo-FO	Faroese	Faroe Islands
0464	fil-PH	Filipino	Philippines
040b	fi-FI	Finnish	Finland
0c0c	fr-CA	French	Canada
040c	fr-FR	French	France
180c	fr-MC	French	Monaco
100c	fr-CH	French	Switzerland
080c	fr-BE	French	Belgium
140c	fr-LU	French	Luxembourg
0462	fy-NL	Frisian	Netherlands
0456	gl-ES	Galician	Galician
0437	ka-GE	Georgian	Georgia
0407	de-DE	German	Germany
0807	de-CH	German	Switzerland
0c07	de-AT	German	Austria
1407	de-LI	German	Liechtenstein
1007	de-LU	German	Luxembourg
0408	el-GR	Greek	Greece
046f	kl-GL	Greenlandic	Greenland
0447	gu-IN	Gujarati	India
0468	ha-Latn-NG	Hausa	Nigeria
040d	he-IL	Hebrew	Israel
0439	hi-IN	Hindi	India
040e	hu-HU	Hungarian	Hungary
040f	is-IS	Icelandic	Iceland
0470	ig-NG	Igbo	Nigeria
0421	id-ID	Indonesian	Indonesia
045d	iu-Cans-CA	Inukitut (Syllabics)	Canada
085d	iu-Latn-CA	Inuktitut (Latin)	Canada
083c	ga-IE	Irish	Ireland
0434	xh-ZA	isiXhosa	South Africa
0435	zu-ZA	isiZulu	South Africa
0410	it-IT	Italian	Italy
0810	it-CH	Italian	Switzerland
0411	ja-JP	Japanese	Japan
044b	kn-IN	Kannada	India
043f	kk-KZ	Kazakh	Kazakhstan
0453	km-KH	Khmer	Cambodia
0486	qut-GT	K'iche	Guatemala
0487	rw-RW	Kinyarwanda	Rwanda
0441	sw-KE	Kiswahili	Kenya
0457	kok-IN	Konkani	India
0412	ko-KR	Korean	Korea
0440	ky-KG	Kyrgyz	Kyrgyzistan
0454	lo-LA	Lao	Lao P.D.R.
0426	lv-LV	Latvian	Latvia
0427	lt-LT	Lithuanian	Lithuania
082e	dsb-DE	Lower Sorbian	Germany
046e	lb-LU	Luxembourgish	Luxembourg
042f	mk-MK	Macedonian (FYROM)	Macedonia (FYROM)
043e	ms-MY	Malay	Malaysia
083e	ms-BN	Malay	Brunei Darussalam
044c	ml-IN	Malayalam	India
043a	mt-MT	Maltese	Malta
0481	mi-NZ	Maori	New Zealand
047a	arn-CL	Mapudungun	Chile
044e	mr-IN	Marathi	India
047c	moh-CA	Mohawk	Mohawk
0450	mn-MN	Mongolian (Cyrillic)	Mongolia
0850	mn-Mong-CN	Mongolian (Mongolian)	PRC
0461	ne-NP	Nepali	Nepal
0414	nb-NO	Norwegian (Bokmål)	Norway
0814	nn-NO	Norwegian (Nynorsk)	Norway
0482	oc-FR	Occitan	France
0448	or-IN	Oriya	India
0463	ps-AF	Pashto	Afghanistan
0429	fa-IR	Persian	Iran
0415	pl-PL	Polish	Poland
0416	pt-BR	Portuguese	Brazil
0816	pt-PT	Portuguese	Portugal
0446	pa-IN	Punjabi (Gurmukhi)	India
046b	quz-BO	Quechua	Bolivia
086b	quz-EC	Quechua	Ecuador
0c6b	quz-PE	Quechua	Peru
0418	ro-RO	Romanian	Romania
0417	rm-CH	Romansh	Switzerland
0419	ru-RU	Russian	Russia
243b	smn-FI	Sami, Inari	Finland
143b	smj-SE	Sami, Lule	Sweden
103b	smj-NO	Sami, Lule	Norway
043b	se-NO	Sami, Northern	Norway
083b	se-SE	Sami, Northern	Sweden
0c3b	se-FI	Sami, Northern	Finland
203b	sms-FI	Sami, Skolt	Finland
183b	sma-NO	Sami, Southern	Norway
1c3b	sma-SE	Sami, Southern	Sweden
044f	sa-IN	Sanskrit	India
0c1a	sr-Cyrl-CS	Serbian (Cyrillic)	Serbia
1c1a	sr-Cyrl-BA	Serbian (Cyrillic)	Bosnia and Herzegovina
081a	sr-Latn-CS	Serbian (Latin)	Serbia
181a	sr-Latn-BA	Serbian (Latin)	Bosnia and Herzegovina
046c	nso-ZA	Sesotho sa Leboa	South Africa
0432	tn-ZA	Setswana	South Africa
045b	si-LK	Sinhala	Sri Lanka
041b	sk-SK	Slovak	Slovakia
0424	sl-SI	Slovenian	Slovenia
080a	es-MX	Spanish	Mexico
100a	es-GT	Spanish	Guatemala
140a	es-CR	Spanish	Costa Rica
180a	es-PA	Spanish	Panama
1c0a	es-DO	Spanish	Dominican Republic
200a	es-VE	Spanish	Venezuela
240a	es-CO	Spanish	Colombia
280a	es-PE	Spanish	Peru
2c0a	es-AR	Spanish	Argentina
300a	es-EC	Spanish	Ecuador
340a	es-CL	Spanish	Chile
3c0a	es-PY	Spanish	Paraguay
400a	es-BO	Spanish	Bolivia
440a	es-SV	Spanish	El Salvador
480a	es-HN	Spanish	Honduras
4c0a	es-NI	Spanish	Nicaragua
500a	es-PR	Spanish	Puerto Rico
540a	es-US	Spanish	United States
380a	es-UY	Spanish	Uruguay
0c0a	es-ES	Spanish (International Sort)	Spain
040a	es-ES_tradnl	Spanish (Traditional Sort)	Spain
041d	sv-SE	Swedish	Sweden
081d	sv-FI	Swedish	Finland
045a	syr-SY	Syriac	Syria
0428	tg-Cyrl-TJ	Tajik	Tajikistan
085f	tzm-Latn-DZ	Tamazight (Latin)	Algeria
0449	ta-IN	Tamil	India
0444	tt-RU	Tatar	Russia
044a	te-IN	Telugu	India
041e	th-TH	Thai	Thailand
0451	bo-CN	Tibetan	PRC
041f	tr-TR	Turkish	Turkey
0442	tk-TM	Turkmen	Turkmenistan
0480	ug-CN	Uighur	PRC
0422	uk-UA	Ukrainian	Ukraine
042e	wen-DE	Upper Sorbian	Germany
0420	ur-PK	Urdu	Pakistan
0843	uz-Cyrl-UZ	Uzbek (Cyrillic)	Uzbekistan
0443	uz-Latn-UZ	Uzbek (Latin)	Uzbekistan
042a	vi-VN	Vietnamese	Vietnam
0452	cy-GB	Welsh	United Kingdom
0488	wo-SN	Wolof	Senegal
0485	sah-RU	Yakut	Russia
0478	ii-CN	Yi	PRC
046a	yo-NG	Yoruba	Nigeria

Oh, the enumeration!

KieranS — Tue, 26 Sep 2006 03:37:00 GMT

Note (10/3/06): This post contains outdated content. For the correction, please see the update in Oh, the enumeration? Part two.

If you use EnumSystemLocales(), one thing you may have noticed in Vista is that the Spanish Traditional sort locale (040a) is no longer included in our enumeration. The locale still exists, so you can ask for it specifically by LCID. But it is no longer enumerated in Vista. So if your code has some dependency on returning the locale in enumeration -- say, if you're enumerating all the locales on the system and relying on the returns from XP pushing forward onto Vista -- you'll want to be mindful of the change.

Note that you can still return this locale in our Vista enumeration function by passing the LCID_ALTERNATE_SORTS flag. If you pass this flag, you will return the Traditional sort Spanish locale as well as all other alternate sort locales available in Vista.

Why did we make this change? Good question.

When we first introduced the Traditional sort locale for Spanish, we made it identical to the locale for Spanish - Spain in all respects but two. First, the two locales used different sorting behavior, as their descriptions would indicate. At the time, we did not realize that we would find other locales that might require the support of multiple sorting standards or behaviors, so we introduced another key difference between the Spanish - Spain locale and the Traditional sort Spanish locale: the two locales share a primary lang ID, but they have different LCIDs.

In retrospect, in light of the many other alternate sort locales for which we have added support, this was not the right design call. Our model for other alternate sort locales is that they actually share all locale data programmatically with the locales on which they are based; the only difference is found in the sorting behavior itself.

Because the Traditional sort Spanish locale is more properly considered an alternate sort variant of the Spanish - Spain locale rather than a full-fledged locale with its own independently defined data, moving forward we would like to treat it like the other alternate sort locales that we support. We wanted to minimize the breaking nature of the change, so we are committed to continuing to support the locale if it is specifically asked for by LCID. However, we also wanted our Vista enumeration behavior to accurately reflect the true nature of the locale. Hence it is available when people enumerate asking for alternate sort locales, but it is not returned by a call to EnumSystemLocales without the LCID_ALTERNATE_SORTS flag.

What's in a name? Part two.

KieranS — Tue, 19 Sep 2006 00:39:00 GMT

Back in the day, in a post entitled What's in a name?, I blogged about some changes to NLS identifiers that we made in order to better meet the expectations of customers accustomed to the IETF standard. In that post, I wrote:

One thing to keep in mind regarding the highlighted entries above is that the CS identifier refers to the now-no-longer region entity Serbia and Montenegro, as Montenegro recently elected to separate from Serbia. The final set of identifiers for the new regions is not yet known, but as soon as the international standards community has reached a decision I will post a further update here.

With the RC1 release of Vista on its way to customers, I wanted to give people an update on the status of our region identifiers for Serbia and Montenegro.

Because we do not want to anticipate the new region identifiers before they are officially approved by the ISO, and because we don't want to cause a breaking change when people may have started to rely on the tags in the RC1 release of Vista, we have decided to continue to support the following tags for our Serbian locales:

sr-Latn-CS [Serbian (Latin, Serbia)]
sp-Cyrl-CS [Serbian (Cyrillic, Serbia)]

When new tags are officially introduced for Serbia and Montenegro, we will assist customers who require the new tags with custom locale solutions.

Please let me know if you have any questions!

Develop locally, think globally!

KieranS — Wed, 23 Aug 2006 03:33:00 GMT

So you want to develop a globalized application for Windows. You're up to speed on development best practices, you're going to be using Unicode, you know how to use Windows APIs and locale data to enhance the user experience you create. So how do you identify and support the right target market for launching your product?

Some applications strive to meet customer expectations across a pretty broad range of markets, since there are certain core computer tasks that most kinds of users in most locations require. Novice, intermediate, and advanced users all over the world are using the internet and communicating by email. Many of them are buying things online, writing documents, instant messaging, or managing their schedule. Some applications cast a wide net in order to capture the broadest range of customers who might be interested in using them.

Other products are intended for smaller, more highly focused markets, where the market might be defined by region, by customer segment, by language group, or by the environment in which the application is used. Datbase applications that are used in the workplace may be irrelevant at home, while educational software may be used at home or in schools but is not typically found on most people's machines at work. If you're writing an application geared to help Japanese speakers learn Italian, it's unlikely that you will design your product to accomodate the preferences of users in Russia. Even tasks that most computer users engage in, such as the examples in the paragraph just above, may require different user experiences for different kinds of customers. Depending on the scope of your application, the design decisions that you make in order to satsify the core scenarios of your core customers may vary quite a bit in the specificity of the user experience that they provide.

So here's the real question: Once you've identified the target market for your product's launch, what do you do if it's a market that Microsoft is only just beginning to support? Particularly if you've identified a business opportunity in a region that is newer to technology, there's a good chance that you'll find some gap between the internationalization support that you can count on the Windows platform to provide in general and the support that is available for the region you're targeting. So how can you develop the application in a scalable, globally extensible way that both supports your current business need and allows for your eventual growth into new markets without a lot of redundant development efforts on your part?

1. Know your requirements.What differentiates your market from other similar markets? What differentiates your application from other similar applications? The more specifically you have outlined your core requirements, the greater the chance that you'll be able to identify exactly which pieces of core support are missing. We may have support that you're not currently taking advantage of because you haven't needed to in the past.

2. Know your requirements, but think extensibly about them. Is your application useful for markets beyond the initial market where you've been focusing your efforts? Thinking up front about the desired growth path for your product helps ensure that you're making decisions that don't make it more difficult for you to expand later on. Hard-coding data that you could be using the Windows platform to store and retrieve is a good way to give yourself headaches when more customers are clamoring for your product later.

3. Include custom solutions for your users. Does your customer base require specialized locale data? Create and include a custom locale for their installation to enhance the experience they'll get using your application. If we can't include a locale on a timeline fast enough for you to jump into the market where you see your opportunity, use the tools and infrastructure available in Vista so that you don't need to wait.

4. Tell us. Finally, if you have a need to develop or extend an application for a market that we're not supporting, we want to know. If we're blocking you, we want to know. The Windows platform needs to support the requirements of developers who create applications for it. If we're not doing that, we need to hear about it and figure out how to get out of the way. Are there new languages or regions for which you're trying to develop applications that you can't rely on Windows to support? Are there kinds of information that you would like to be able to store and retrieve about your market's preferences that are not currently available in the Windows platform? If we don't know that you need it, we may never provide it.

Windows provides a great core infrastructure to work from today, but it's only by figuring out what's missing that we can continue to improve it to help developers provide a truly globalized experience for their applications.

Not all custom locales are created equal.

KieranS — Wed, 09 Aug 2006 20:12:00 GMT

If you're going to use the Locale Builder tool to create a custom locale, you have a couple of options: You can create a brand new locale for a language-region combination that is not currently supported by Microsoft, or you can replace a locale that exists on your machine with appropriately customized data. If you choose to add support for a language-region combination that's entirely new, you'll be creating a supplemental locale, designed to supplement the existing Windows list. If you choose to replace data for an existing locale, you'll be creating a replacement locale, designed to replace some locale that is already supported by Windows.

So how do you know which you really want?

A replacement locale is a good choice if you want to continue identifying the locale with its Windows name and LCID; the crucial property of a replacement locale that cannot be changed is its identifier (e.g. en-US, ii-CN, ja-JP, etc). This means that applications will be able to call our APIs with existing Windows identifiers and on your machine they will return the data that you have provided rather than the data that shipped with Windows. There are a few other properties that cannot be changed if you choose to create a replacement locale. One of the most crucial of these is code page assignments. We know that you wouldn't dream of developing using anything but Unicode :), but just in case you're still maintaining or running ANSI applications, we don't want those applications to break because code page assignments have changed.

Internally, we also try not to change sorting behavior for existing locales between major versions -- and if we discover that there is customer need for providing custom sorting behavior in the future, we'll have to think very hard about what to do here for replacement locales. When we change sorting behavior for existing locales -- say, if we discover that our collation isn't linguistically accurate -- applications that rely on consistent sorting results need to make changes to accomodate our updates (e.g. database applications that need to reindex whenever sorting behavior changes). So we want to keep this kind of update to a minimum. (We are introducing APIs that will let applications know which version of our sorting behavior is on a particular machine, which will help mitigate this, but that's fodder for another post.)

Supplemental locales require you to choose identifier names that do not overlap with the names of locales already on your machine. We recommend that you choose identifiers that conform as much as possible to the IETF standard; one reason that our names generally follow this model is that people recognize and use them. If you want your custom locale to be readily accessible to applications, using a standard identifier name is a good start. Once you've changed the name, you can customize code page assignments or anything else, since we don't run the same kind of a risk of breaking applications that are relying on long-standing settings.

In the end, whether you choose to create a supplemental locale or a replacement locale depends on your needs; it is just as easy to create, share, and install either of them.

So I want Windows to support my language. How does that happen, exactly?

KieranS — Tue, 08 Aug 2006 04:16:00 GMT

One of the disappointing things about the end of a product cycle is that it's too late to add into this version all the cool new customer requests that we get. (Of course, the flip side of this is that one of the great things about new product cycles is that we have lots of cool new customer requests to figure out how to add in the future!)

One type of request that comes from customers throughout the cycle is how they should go about getting support for their language into Windows. The extensibility tools we've been developing for keyboard layouts, locales, and transliteration are the beginning of this story, but people often want to know how they might go about getting on the roadmap for future releases, especially in terms of writing system and localization support.

In the end the single best way to let us know that you are advocating for support is to work with your local Microsoft subsidiaries. By engaging locally, we have a great opportunity to understand local markets and their particular needs and to help people in the region provide us with the technical information that we need to assess a support roadmap. Just a few of the factors we consider:

Is the writing system encoded in Unicode? If not, the first step towards support is working with the Unicode consortium to get the writing system encoded (including all relevant glyphs for your language).
Can we find accurate, reliable locale data? In many parts of the world that are newer to technology, finding consistent information can be a challenge. Locale data here encompass a pretty broad range of stuff:

Generally accepted standards for date/time formatting, calendar information, currency standards, and so on.
Generally accepted encoding of the writing system in the software space. Are there any particular challenges with getting the script to shape or render? Can we provide a font that the majority of speakers will be happy with?
Consistent standards for keyboard layouts and input methods. Is there some layout that users in the region favor? Are there multiple layouts? Does the language require an input method editor?
Consistent sorting behavior. Do literate speakers of the language agree on collation order? Do we have native speakers who can help us encode the sort order correctly?

Does the language have technical terminology? Would a Community Glossary effort help?
What do developers need? What kinds of applications are being developed for the language? Does the language require localization support? Are there other languages in the region that are commonly used?

And that's just a partial list!

Over time I'd like to use this space to address the requirements entailed in each of the above and more, plus let customers know how they can best collect and assemble this information and use our existing tools to provide solutions faster. One area where we've received feedback that we could do better is in clear, agile responses to this kind of request. As we've discussed, our extensibility tools are one part of that story; providing transparent information about the roadmap process is another important piece, and I want to use this space to answer some of the questions that we receive about this.

Lights, camera, action!

KieranS — Fri, 04 Aug 2006 01:44:00 GMT

For everything you ever wanted to know about Windows globalization and localization, or at least for one or two things, you might like to check out this Channel 9 interview. It features Michael Kaplan, whose blog you may know; Jeff Allen, our director of localization services; Alp Turkmen, who is driving our business direction in the Windows international space; Claus Juhl, one of our localization engineers; and me. Michael and I speak about globalization issues, with a heavy focus on the Locale Builder and associated ways that we'd like to open up our infrastructure to accomodate all kinds of community solutions, and we're about 45 minutes in. But don't skip! There's great information throughout.

If you watch, please let us know what you think! And yes -- there are closer to 7,000 languages in the world than 2,000, for those of you playing along at home.

What's in a name?

KieranS — Thu, 03 Aug 2006 08:28:00 GMT

Out with the LCIDs, in with the string-based identifiers! Well, mostly. You can still use LCIDs to retrieve data for Windows locales, as before. But as I've mentioned here previously, we've also introduced string-based identifiers and associated APIs that developers can use to retrieve locale data. One major advantage to using these identifiers is that they really open up the world of custom locales. You can now create custom locales with appropriately named identifiers and those identifiers will actually mean something.

The locale identifier strings that we're using follow the IETF standard as closely as possible, which means that we use ISO 639 and 3166 language and region identifiers wherever we can, along with appropriate script tags where relevant. So now you'll be able to use strings like en-US, fr-FR, iu-Cans-CA, etc. to retrieve locale data.

Unfortunately not all of the string identifiers introduced in the parallel APIs in .NET 2.0 follow the IETF standard. For instance, the .NET identifiers placed script name after region name rather than the other way around, so .NET 2.0 has e.g.

az-AZ-Latn

rather than e.g.

az-Latn-AZ.

In order to keep things consistent between Windows and .NET, we've decided to update these identifiers in the version of .NET that will be included with Windows Vista. We plan to release a patch with the new identifiers that will be available to .NET 2.0 installations on downlevel platforms as well so that we can make it easier for developers who are creating managed applications that need to work across Windows versions. Affected identifiers include:

az-AZ-Cyrl --> az-Cyrl-AZ
az-AZ-Latn --> az-Latn-AZ
bs-BA-Cyrl --> bs-Cyrl-BA
bs-BA-Latn --> bs-Latn-BA
zh-CHS --> zh-Hans
zh-CHT --> zh-Hant
div-MV --> dv-MV
en-CB --> en-029
iu-CA-Latn --> iu-Latn-CA
sr-BA-Cyrl --> sr-Cyrl-BA
sr-BA-Latn --> sr-Latn-BA
sr-SP-Cyrl --> sr-Cyrl-CS
sr-SP-Latn --> sr-Latn-CS
uz-UZ-Cyrl --> uz-Cyrl-UZ
uz-UZ-Latn --> uz-Latn-UZ

One thing to keep in mind regarding the highlighted entries above is that the CS identifier refers to the now-no-longer region entity Serbia and Montenegro, as Montenegro recently elected to separate from Serbia. The final set of identifiers for the new regions is not yet known, but as soon as the international standards community has reached a decision I will post a further update here.

So I'm ready to test-drive Windows Vista. What next?

KieranS — Tue, 01 Aug 2006 05:40:00 GMT

Many of our Windows Vista beta testers contact us excited to be test-driving the beta but wondering what to look for. I thought it might be nice to share here the test cases that we give interested beta testers. If you try these test cases and you have feedback for us on any aspect of our basic NLS/language enablement support, please let me know either in the comments section here or by sending me email directly.

***

NLS data testing for Beta partners

This is a collection of usage scenarios that beta testers might like to try in order to verify that their locale data is correct.

Fonts

If you do one thing: Look at sample representative strings and longer pieces of text in Notepad in Vista (or in XP with ClearType turned on) at 8pt (Office UI) and 9pt (Vista UI). Are the strings readable?

If you do more: For Latin, Cyrillic, and Greek, the font to look at is Segoe UI. Look at all four styles (the italic, bold and bold italic as well as the regular). Identify any bad character shapes, or unacceptable clipping of diacritics. If you can do more, also check out strings using Microsoft Sans Serif and MS UI Gothic.

In Vista, one good way to test the UI font is by creating/renaming folders with strings set in the relevant languages.

Keyboards

If you do one thing: Make sure that you can select the keyboards that are important for your locale. Type using the keyboard you have selected and make sure that the output matches what you expect from the layout.

If you do more: Verify that the keyboards for your locale match the standards that you expect to find.

Regional Options

If you do one thing: Open up Regional Options and change your user locale. Is your locale on the list? Is it in the form that you expect? Are the sample data displayed what you expect?

If you do more: Click customize in Regional Options. Are the default values what you expect?

Are you able to edit and save these values?

Change your user locale (Standards and Formats) setting and save settings. Does your formatting change as you expect it to (e.g. in Windows calendar)?

Are the date formats that you expect available? Are there any errors in month or day day names?

If you speak a language that uses genitive forms for month names, are the month names displayed correctly in date strings?

Open up your Windows calendar. Does it look like you expect it to?

Sorting

If you do one thing: Create several filenames using characters from your language. Do they sort properly in Explorer?

If you do more: Create a list of words using characters from your language in Word or Excel. Sort the list. Does it sort properly?

***

If you try any of this stuff and have feedback for us on what we're doing right and not right, please let me know!

Next stop: Antarctica?

KieranS — Tue, 25 Jul 2006 01:46:00 GMT

A couple of days ago I posted about the Microsoft Locale Builder. But what about the locales that we're adding new for Vista? A few interesting facts:

When Vista ships, it will include locale support for over 130 languages built in as part of Windows locales.
The built-in writing system support plus the Locale Builder and Keyboard Layout Creator tools will make Windows linguistically accessible for >90% of the world's literate population -- this is over 3.5 billion people!
Windows will ship locales for languages and regions in every continent except Antarctica.
Every locale that Windows ships has linguistically appropriate sorting behavior built in.
We are adding over 60 new locales on top of what we shipped with XP, including new support for customers in Africa, Latin America, and Asia.
The Local Language Program is producing fully localized versions of Windows Vista for 35 languages, with partial localization support for over 60 more.
Windows locales are now supplementing LCID identifiers with string-based identifiers that will allow developers to use IETF names in addition to LCIDs to retrieve locale data via our APIs. The name-based support makes it easier for users to create custom locales using standard names that everyone in the industry will recognize.
We are working with beta testers for all of the new locales that we have added for Vista in order to ensure that the data we ship are of the highest possible quality.

If you're a beta tester and you're finding problems with your locale support (or with some other aspect of your international support), please let me know.

Languages, regions, locales, oh my!

KieranS — Thu, 20 Jul 2006 23:12:00 GMT

Michael Kaplan has blogged quite a bit about our release of the Microsoft Locale Builder Beta, the final version of which is planned for use on Vista when it ships. We're pretty excited about the Locale Builder here for a whole bunch of reasons. Like The Microsoft Keyboard Layout Creator before it, the Locale Builder makes it possible for customers to create and share solutions faster and to ensure that the support available in Windows (support for formatting times, dates, currencies, and text, in the case of the Locale Builder) provides the best possible user experience across Windows applications. Straight up, you speak your language better than we do.

One of the pieces that I have been involved with since joining Microsoft is the investigation and maintainance of the NLS locale data that are used in Windows and .NET framework locales. Collecting data in an accurate, useful way poses many challenges. A whole array of things can impact language and technology standards in a region, including many factors whose rapid rate of change can make things interesting. Some things we need to keep in mind:

Changes in politics. Governments change at every scale. New leaders are elected; new representatives are appointed; new curricula are developed for school systems; official language legislation is passed and then revoked and then passed again.

Changes in standards. As technology evolves, new standards get created. As standards organizations change, new standards get created.

Changes in language or language use. Some languages are written in multiple writing systems. Vocabulary is created. A change in one part of the linguistic system can have ripple effects everywhere else in the system.

People move around. Just in my building, I work with native speakers of Spanish, French, Hebrew, Arabic, Chinese, Catalan, Hindi, Kannada, German, Portguese, Italian, Russian, Ukrainian, Japanese, Czech, Persian, and that's just off the top of my head. But they all work here, in Redmond. As people move around more and more, the usual pairings of language and region information inside a locale become less and less relevant. It's just not possible to predict the particular combinations that any individual user will want.

Even in a static universe, one size rarely fits all. I'm an en-US user, but I like a 24-hour clock. I'm weird like that.

The fact is that correct NLS data can be a moving target, and we're hoping that the Locale Builder can help address this for our customers. We're giving users the ability to change the locale data that we've shipped, not only for their own machines, but also for others, since they'll be able to share custom locales that they create. We're also giving users the ability to add new locales entirely, since we recognize that we provide just a small set of the possible languages and regions that our customers care about.

One thing I'm really interested in learning is which of these options will be more popular for our customers. Are people going to use the Locale Builder to modify existing Windows locales? In which case, which locales are people changing, and which pieces of data are they changing within those locales? Or are people going to use the tool to create new locales entirely? It's important to me that we understand what we're getting wrong today and what users feel that they need to fix.