Progressive Development : general programming

Motley says: "Developing international software is really, really hard. We need a brand new version."

James Waletzky — Tue, 16 Sep 2008 18:11:00 GMT

Summary

Motley: Developing international software is really, really hard. We need brand new binaries to ship in other languages.

Maven: Follow these tips when developing for international markets: design for one worldwide binary, ensure the software is globalized, do not build strings at run-time, expand UI labels by 40%, and test with pseudolocalized builds.

______________________________

[Context: Motley is a little shocked that the marketing team has asked the development team to produce a non-English version of the software they just shipped]

Motley: Arrrrrrggggghhhh. I've had it! We spend all this time shipping the first version of our software in English, and now the marketing team wants us to ship a Japanese version to expand our reach. I don't know the first thing about shipping software in languages other than English. It's too hard to do if you don't speak the language. We are going to have to hire a vendor to fully create a new version.

Maven: Calm down, Mot. It's not so bad. If you designed the software correctly from the beginning, shipping in another language really is not that difficult.

Motley: Okay, don't keep me in suspense. How does one "design the software correctly from the beginning?" I have to admit that we were not thinking about shipping an international version, and instead focused on getting the thing done as soon as possible in our default language - English.

Maven: There are various keys to shipping software in multiple languages. The first is to have one worldwide binary.

Motley: How can we ship one binary? If we ship English and Japanese, that will mean two different binaries that we have to build and distribute. Use your head, Mave!

Maven: Actually, no. Ultimately you want one executable for your application that is used regardless of what language is displayed in the UI. The user can potentially set the locale via the Windows control panel to switch languages, which does not change the .EXE that gets launched when the application icon is double-clicked. What does change, however, is what strings are loaded by the executable.

Motley: Ah, of course. That's what I meant. And, hmmmmm…. Fortunately we did use resource files for the UI strings for the most part to make it easier on the user experience people to make sure we developers who don't talk good English get our mistakes fixed.

Maven: Nice play on the grammar, Mot. The other benefit of being able to pass the resource files to user experience people is also being able to pass them to localizers for translation to other languages like Japanese. To do this, however, you need every visible user string in those resource files. Did we do that?

Motley: Well, probably not every string. We'll have to do a scrub of the strings. Shouldn't be a problem though. So that's it?

Maven: Not necessarily. You have to follow a few other rules. Firstly, do not build strings at run-time.

Motley: What exactly does that mean? Let's take a simple example: if I ask the user for their name and I want the software to say "Hello", I have to do something like: "Hello" + name. No real way around that.

Maven: Yes, that type of functionality is often required. However, there are better ways to put strings together. For example, in C# you want to do something like:

string.Format("Hello, {0}", name);

But remember, the string above is actually taken from a resource file. That gives the localizers the opportunity to change the positioning of the token in the string to match the language being translated. Some languages may force different locations of the name in the string. If you build the string in code with the '+' operator, the ordering is compiled in and you have to rebuild (and generate another binary) to change order if another language is "<name>, <hello>" instead of "<hello>, <name>".

Motley: At least the solution is easy. What else do we need to do?

Maven: Another big rule is to expand your UI labels by approximately 30% to 40% in length. In some languages, like German, a typical phrase has many more words/characters than the equivalent English phrase. As a result, just because your UI looks great for English does not mean that text will not be clipped for other languages. Have to be careful there.

Motley: Rules, rules, and more rules. Can't we just abbreviate for the other languages? I am kidding of course.

Maven: Additionally, don't forget that you should not make any assumptions about sort order, date/time formats, currencies, and other international differences. For your software to be truly globalized, it must work with any culture - even when Windows is set to display right-to-left instead of the standard left-to-right.

Motley: One thing at a time. Let's just focus on making v1.1 localizable (translatable to other languages) and globalized (functions correctly in differing cultures). We will likely have to make small modifications to our designs to accommodate this. But here is another question: I don't speak Japanese, so when we have a Japanese version available, how am I supposed to debug it? I want to avoid constantly switching between a Japanese and English build.

Maven: Great question. For initial testing that finds the vast majority of problems, as well as easier debugging, we can create what is called a pseudolocalized build.

Motley: Pseudo-what??

Maven: Pseudolocalized. We take a build that is localizable, and automatically insert some more, well, troublesome characters. We could insert accented 'e' characters to replace the 'e', as well as other characters that are similar to their English counterparts, thus still making phrases readable. In addition, the lengths of the strings are expanded, and each string starts with a delimiter character like '[' and ends with ']'. If you see a string in the UI that does not start and end with those characters, then you have a clipping bug. Pseudo-localized builds really help nail down international problems early in the development cycle.

Here is what a pseudolocalized application could look like:

Motley: Very cool. A lot of it looks like gibberish, but is still English-readable. The UI contains a lot of the characters that can be troublesome given the wrong font choice for the UI. I can definitely see this technique saving lots of bugs prior to check-in, provided the developers can generate a pseudo-localized build on their own.

Maven: Exactly. Since pseudolocalized builds really only require processing of resource files, they are generally easy to generate. The are tools out there that can do it for you, or you can write your own text processor to do simple string expansion and replacement. It's a valuable tool.

Motley: Good tips for a change, Mave. I guess we have some work to do to really make our software internationalized, but I don't feel it will be a lot of work given the choices we have already made. I better get to work!

______________________________

Maven's Pointer: There are lots of other tips to creating good international software, from requirements to development to testing. As a developer, it is little extra work to make your application localizable and globalized. Do this work up front - trust me. I was on a software project that lasted 6 months while we moved strings around. Not exactly fun work. If we had followed the tips above, avoided text in graphics (hard and expensive to translate), and avoided hard-coded user-visible strings in the code, we would have been in great shape to ship in multiple languages. Do it right the first time, and take the very minor hit to the schedule that comes with it.

Maven's Resources:

Developing International Software, 2nd Edition, by Dr. International, Microsoft Press, ISBN: 0735615837, November 2002.
NET Internationalization: The Developer's Guide to Building Global Windows and Web Applications, by Guy Smith-Ferrier, Addison-Wesley Professional, ISBN: 0321341384, August 2006.
Wikipedia entry on Pseudolocalization: http://en.wikipedia.org/wiki/Pseudolocalization

Motley says: "Native C++ code development is obsolete"

James Waletzky — Tue, 08 Apr 2008 19:22:00 GMT

Summary

Motley: Native C++ code development is obsolete. Everyone should be using a language like C# or Java.

Maven: Managed code is a great platform, but there are still reasons to use native C++, such as enhancing legacy code and developing an operating system (e.g. Windows Mobile). Understanding C++ makes you a better managed code developer.

______________________________

[Context: Maven and Motley are in the middle of discussing programming languages and, in particular, C#. Motley is pondering other languages and their usefulness]

Maven: I agree with you Mot - C# is a killer language. There are constant strides to move it forward and it is very intuitive for a new developer to learn.

Motley: I wouldn't want to develop in any other language, thank you very much. In fact, I am surprised other languages still exist. Well, Java has a place for the non-Windows developers, but other than that, why maintain them?

Maven: C# is great, but there are other languages out there that are still widely used. C++, for example, is still used in many different companies and serves a real need.

Motley: Hah! Name one. There are so many advantages to developing with managed code that you don't get in C++. C# is much more intuitive and easier to learn than C++. It is harder to make nasty mistakes considering you have garbage collection. Exception handling is a built-in construct. Your code is optimized by the Just-In-Time (JIT) compiler dynamically according to the platform you are executing on. I could go on and on.

Maven: Here's where you may want to have a chat with James. Before joining Cynthesis he worked for Microsoft in the Entertainment and Devices division group developing components for the Windows Mobile operating system that runs on tens of millions of so-called "smart phones". His work on that team was all native code.

Motley: Sucks to be him. I am willing to bet he is the victim of code complexity, a legacy code base, extremely long build times, nasty dangling pointer problems, and a host of other issues that are an order of magnitude more complicated than developing for .NET.

Maven: Yes, he told me that there were some nasty problems. But the point is that there are real reasons why the group chose to develop on a native C++ platform.

Motley: I'm waiting.

Maven: When you think about it, developing for mobile and embedded devices is almost like taking a step back in time a few years. Consider the type of hardware you are developing for:

Devices are constrained by CPU - a 200Mhz or 600Mhz processor (if you are lucky) is the norm. Writing software that exhibits high performance is a critical concern.
Devices are constrained by memory - 128MB of RAM (if you are lucky). Applications with high memory usage are not acceptable.
Devices are constrained by unconnected power - rechargeable batteries last a few days (if you are lucky). Software that sucks CPU and other resources has a big impact on battery life. Efficiency is king.
Devices are constrained by typically not having a floating point unit or graphics processor. As a result, the algorithms you choose must take efficiency into account.

When you consider all these constraints, you want to develop applications to be as lean and mean as possible. The Common Language Runtime (CLR) has significant overhead in the system and is a negative drain on efficiency.

Motley: Those concerns are less prominent in most other types of development, however.

Maven: You might think, but ignoring these constraints even on desktop workstations leads to much more bloated software and applications that do not play nice with one another. Although not necessarily high profile, the skills that a mobile developer possesses make her a better developer overall, regardless of application domain, platform or language. You started out on C++, didn't you?

Motley: Yeah, and I never want to go back. As far as I'm concerned, native C++ code development is obsolete.

Maven: C++ is far from obsolete. In fact, standardization of the new version of the language, C++ 0x, should be completed by end of 2009.

Motley: Great. I'll just take a job where developers use managed code and be happy.

Maven: That's fine, although there are many more reasons to join a team than just technology and programming language. We discussed that previously. Anyway, I would argue that you are a better developer now because you had a C++ background. You understand what a pointer is don't you?

Motley: Don't insult me.

Maven: Do you ignore the concept of a pointer in managed code?

Motley: Absolutely not. There are times to pass objects by reference. There are times when using COM Interop that I have to worry about AddRef and Release. There are times when I do server-side development that I have to explore heap fragmentation issues (and in rare circumstances heap corruptions). Understanding a pointer is a necessity.

Maven: I agree! In addition, and when writing an operating system like Windows Mobile, you may need finer control at many different levels of the architecture. You cannot take the hit in performance for garbage collection. You may need your own memory manager given hardware constraints. You need finer control over the creation and destruction of objects to ensure resources are freed as soon as possible.

Motley: Understood. It's just not for me. I did my time.

Maven: I prefer the .NET platform as well. James, however, just found such a fantastic team at Microsoft that he gave up his technology preference to build a cool application with super strong developers. Although COM and C++ have been around for years, they are here to stay for a while yet. There is lot of legacy code written in C++ that still must be maintained. Although we may wish that the entire development population wrote managed code, that day is still far off in the future.

______________________________

Maven's Pointer: Is Maven's "pointer" a bit of a misnomer if we talk mostly about managed code? Well, yes and no, as Motley indicated. Taking some time to understand the underlying technology behind managed code will improve you as a developer. This may include starting at the level of C++ but also delving down into intermediate language (what C# compiles into) and assembly language. By the way, if you understand how a Reverse Polish Notation (RPN) calculator works and its stack-based number manipulation, you already understand much of what you need to grasp intermediate language (IL).

Maven's Double Pointer Indirection: Microsoft made a big bet on .NET with Windows Vista a few years ago exposing the majority of the APIs as managed code. The best didn't really pan out and contributed to forcing a development reset partway through the cycle. Managed code has a great future for many different types of applications, but for developing operating systems, more improvements are in order.

Maven's Triple Pointer Indirection: This is a bit of a controversial topic. I'd love to hear the opinions of others out there. How long will native languages like C++ be around?

Maven's Resources:

Information on the next version of C++: http://en.wikipedia.org/wiki/C%2B%2B0x
C++ vs. C#: http://www.thinkingparallel.com/2007/03/06/c-vs-c-a-checklist-from-a-c-programmers-point-of-view/