I'm not a Klingon ( ) : Dates & Times

Japanese Calendars, How do I Test Support for Additional Eras?

shawnste — Thu, 24 Sep 2009 21:54:00 GMT

The Japanese Calendar is labeled by the reign of the current emporer. Windows has supported 4 Japanese calendar Eras, however in the future there may be more eras. Realizing this, we've added support in Windows 7, Server 2008R2 & .Net v4 for additional Japanese Eras.

There're a few things applications should know about extended Japanese Era support:

The first is that applications may see more than 4 eras!.
The current era could end
Future dates could change, moving from the current era to a new era. (So "Heisei 71" could be "something 12" or something like that)
The era numbers themselves could change, so the 1-4 era numbers shouldn't be relied upon.

Cool, so if we're being careful about all of those things, and are pretty sure we don't have any hard-coded dependencies on the Japanese Eras or when they are, how do we test it?

On Windows 7 / Server 2008 R2 (beta) and CLRv4 (beta) Japanese era information can be found in the registry:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\Calendars\Japanese\Eras]
"1868 01 01"="明治_明_Meiji_M"
"1912 07 30"="大正_大_Taisho_T"
"1926 12 25"="昭和_昭_Showa_S"
"1989 01 08"="平成_平_Heisei_H"

Additional eras can be added just by adding an entry to the table. The format for the value "YYYY MM DD"="JE_AJE_EE_AEE" where the first day of the era is represented in the value name by YYYY for the year (gregorian), MM for the month, and DD for the day. The value data contains the era strings, the "full" era name, followed by the abbreviated name (in Japanese), then the full name followed by the abbreviated name in English. Each value is seperated by _ (underscore). So in "YYYY MM DD"="JE_AJE_EE_AEE", JE represents the Japanese Era name, AJE represents the abbreviated Japanese Era. EE represents the English Era name, and AEE represents the abbreviated English Era name.

An example would be:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\Calendars\Japanese\Eras]
"1868 01 01"="明治_明_Meiji_M"
"1912 07 30"="大正_大_Taisho_T"
"1926 12 25"="昭和_昭_Showa_S"
"1989 01 08"="平成_平_Heisei_H"
"2020 09 01"="仮名_仮_Test Era_X"

(I'm not trying to make any predictions with the 2020, it's just a test number :) Also remember this only works in Windows 7 and other newer products.

[Edited: This is intended to support the possibility of a future era, not to enable older eras. There's the potential that lots of stuff won't work correctly if eras prior to the existing 4 are added]. Also note that this registry key is pretty flexible. If you want you could add a bunch of data for a bunch of eras < 1868. (I don't have the data.) That might break more stuff because then "Era 1" would turn into "Era 8" or something. (Which is why I said not to depend on the Era number, it's pretty meaningless and arbitrary for us). As noted, we only really intended to support adding new eras, not the historical ones. Prior to 1870 or so the calendar was lunar not gregorian based, so conversions prior to then are pretty meaningless anyway.

Hope this is interesting,

Shawn

DateTime.Parse() and DateTime.TryParse() fail in some cases

shawnste — Fri, 11 May 2007 01:36:00 GMT

DateTime.Parse and DateTime.TryParse fail in .Net 2.0 if the user locale date override has a space in it. Ie: if its something like "M d yyyy". .Net is getting tripped up over the space (we're working on a fix for SP1/Orcas).

A workaround is to pass the culture without the user overrides as the format specifier. This works because the non-overridden culture doesn't have the spaces that confuse the tokenizer, and the parser ignores spaces anyway.

using System;
using System.Globalization;

class Test
{
	static void Main()
	{
		String strDate = "01 Jan 2007";

		Console.WriteLine(strDate);
		Console.WriteLine(CultureInfo.CurrentCulture.DateTimeFormat.ShortDatePattern);
		Console.WriteLine(DateTime.Now.ToShortDateString());

		Console.WriteLine();
		
		DateTime dDate;

		// This works because the user defaults are ignored and the spaces are handled
		// by parse, which ignores spaces anyway.
		dDate = DateTime.Parse(strDate, new CultureInfo(CultureInfo.CurrentCulture.Name, false));
		Console.WriteLine(dDate);

		// This fails because the pattern has a space
		dDate = DateTime.Parse(strDate);
		Console.WriteLine(dDate);
	}
}

The output looks like this (if you set the user override for the date pattern in intl.cpl to "M d yyyy")

>csc test.cs
>test
01 Jan 2007
M d yyyy
5 10 2007

1 1 2007 12:00:00 AM

Unhandled Exception: System.IndexOutOfRangeException: Index was outside the boun
ds of the array.
   at System.Globalization.DateTimeFormatInfo.InsertHash(TokenHashValue[] hashTa
ble, String str, TokenType tokenType, Int32 tokenValue)
   at System.Globalization.DateTimeFormatInfo.CreateTokenHashTable()
   at System.Globalization.DateTimeFormatInfo.Tokenize(TokenType TokenMask, Toke
nType& tokenType, Int32& tokenValue, __DTString& str)
   at System.__DTString.GetSeparatorToken(DateTimeFormatInfo dtfi)
   at System.DateTimeParse.Lex(DS dps, __DTString& str, DateTimeToken& dtok, Dat
eTimeRawInfo& raw, DateTimeResult& result, DateTimeFormatInfo& dtfi)
   at System.DateTimeParse.TryParse(String s, DateTimeFormatInfo dtfi, DateTimeS
tyles styles, DateTimeResult& result)
   at System.DateTimeParse.Parse(String s, DateTimeFormatInfo dtfi, DateTimeStyl
es styles)
   at System.DateTime.Parse(String s)
   at Test.Main()

Hopefully this is helpful to people who've run into this problem. Of course another workaround is just to use a different date seperator. Of course parsing is generally tricky, particularly if you're asking for the user culture and data. See Culture data shouldn't be considered stable (except for Invariant) for a list of some of the issues. In general a calendar control is more reliable when getting data from a user.

'til next time,

Shawn

DateTime.Parse() fails when separator is " " (space) in .Net v2.0 RTM

shawnste — Mon, 19 Mar 2007 21:45:00 GMT

In Whidbey (Microsoft.Net v2.0), DateTime.Parse() throws an exception if the date separator is a space. This was fixed in future QFEs, including the Whidbey Vista release, so you'll only see this on XP, 2003, etc and not on Vista.

This is fixed in future versions, so if you apply most any QFE to .Net 2.0 RTM this bug'll be fixed.

How do I get localization in JScript (JavaScript) in IE7?

shawnste — Wed, 06 Sep 2006 02:08:00 GMT

I ran into an interesting problem with a Vista sidebar calendar gadget. It seems that it is not exactly trivial to find out interesting NLS data like localized month names, first day of week, etc. from JScript.

Eventually I found a workaround for some of the data. VBScript has MonthName, WeekDayName and WeekDay functions that can be used for some of the data I was looking for. These can be wrapped in VBScript functions which JScript can then call from the same page. I calculated FirstDayOfWeek information by using the WeekDay function for a known date.

These are the helpers that I ended up with:

<script language="VBscript">
' Helper functions so we can do this in JScript as well
Function MyMonthName(month)
   MyMonthName = MonthName(month)
End Function
Function MyAbbrevMonthName(month)
   MyAbbrevMonthName = MonthName(month, True)
End Function
Function MyDayName(day)
   MyDayName = WeekDayName(day)
End Function
Function MyAbbrevDayName(day)
   MyAbbrevDayName = WeekDayName(day, True)
End Function
' Get First Day of Week where 1 is Sunday, 2 is Monday, etc.
Function FirstDayOfWeek()
   Dim MyDate
   MyDate = DateSerial(2002, 9, 1) ' A Sunday
   FirstDayOfWeek = (9 - WeekDay(MyDate, 0))
   if FirstDayOfWeek = 8 then FirstDayOfWeek = 1
End Function
</script>

Unfortunately some items like Shortest Day Names and Month/Year patterns can't be retrieved this way, but I was able to solve most of my problem. The attached CalendarData.html uses these helper functions to dump the values I was interested in using both VBScript and JScript.

I also embedded the HTML here, so here's the output, hope this is interesting:

ISO 8601 Week of Year format in Microsoft .Net

shawnste — Wed, 25 Jan 2006 02:48:00 GMT

Several people have noticed that Calendar.GetWeekOfYear() is almost like the ISO 8601 week when passed CalendarWeekRule.FirstFourDayWeek and DayOfWeek.Monday, however it is a little bit different. Specifically ISO 8601 always has 7 day weeks. If the first partial week of a year doesn't contain Thursday, then it is counted as the last week of the previous year. Likewise, if the last week of the previous year doesn't contain Thursday then its treated like the first week of the next year. GetWeekOfYear() has the first behavior, but not the second. Ie:

Date	Day	GetWeekOfYear	ISO 8601 Week
12/31/2000	Sunday	52 of 2000	52 of 2000
1/1/2001	Monday	1 of 2001	1 of 2001
1/1/2005	Saturday	53 of 2004	53 of 2004
12/31/2007	Monday	53 of 2007	1 of 2008

Notice that the ISO 8601 week at the end of 2007 is different than the GetWeekOfYear() week. For GetWeekOfYear(), both the last week and the first week have fewer than 7 days. Also notice that even though its a 2007 date, its considered the first week of 2008. Similarly the first day of 2005 is considered to be the last week of 2004 by either method.

A simple workaround to consistently get the ISO 8601 week is to realize that consecutive days Monday through Sunday in ISO 8601 weeks all have the same week #. So Monday has the same week # as Thursday. Since Thursday is the critical day for determining when the week starts each year my solution is to add 3 days if the day is Monday, Tuesday or Wednesday. The adjusted days are still in the same week, and use values that GetWeekOfYear and ISO 8601 agree on.

Note that if the requirement is to compute a date in the 2004W536 form, the code will still have to detect that a week 53 in January means that we need to decrement the year by 1, and a week 1 in December requires incrementing the year by 1.

Here's my example. I made this a complete program, however GetIso8601WeekOfYear() is the worker function. I used a static calendar, which probably isn't necessary. Bala pointed out that one could derive a class from GregorianCalendar and override GetWeekOfYear(), but I'm not sure what the repercussions of using such a calendar elsewhere would be. You can try it if you want, but for now this sample is just a simple static method. Main is just here to run through a bunch of days and show when the week calculations differ.

using System;
using System.Globalization;

class Test
{
    // Need a calendar. Culture's irrelevent since we specify start day of week
    private static Calendar cal = CultureInfo.InvariantCulture.Calendar;

    // This presumes that weeks start with Monday.
    // Week 1 is the 1st week of the year with a Thursday in it.
    public static int GetIso8601WeekOfYear(DateTime time)
    {
        // Seriously cheat. If its Monday, Tuesday or Wednesday, then it'll
        // be the same week# as whatever Thursday, Friday or Saturday are,
        // and we always get those right
        DayOfWeek day = cal.GetDayOfWeek(time);
        if (day >= DayOfWeek.Monday && day <= DayOfWeek.Wednesday)
        {
            time = time.AddDays(3);
        }

        // Return the week of our adjusted day
        return cal.GetWeekOfYear(time, CalendarWeekRule.FirstFourDayWeek, DayOfWeek.Monday);
    }

    static void Main(string[] args)
    {
        // 1/1/1990 starts on a Monday
        DateTime dt = new DateTime(1990, 1, 1);
        Console.WriteLine("Starting at " + dt + " day: " + cal.GetDayOfWeek(dt) + " Week: " +GetIso8601WeekOfYear(dt));

        for (int i = 0; i < 100000; i++)
        {
            for (int days = 0; days < 7; dt=dt.AddDays(1), days++)
            {
                if (GetIso8601WeekOfYear(dt) != cal.GetWeekOfYear(dt, CalendarWeekRule.FirstFourDayWeek, DayOfWeek.Monday))
                {
                    Console.WriteLine("Iso Week " + GetIso8601WeekOfYear(dt) +
                        " GetWeekOfYear: " + cal.GetWeekOfYear(dt, CalendarWeekRule.FirstFourDayWeek, DayOfWeek.Monday) +
                        " Date: " + dt + " Day: " + cal.GetDayOfWeek(dt));
                }
            }
        }
    }
}

Custom Time Zones in Windows

shawnste — Sat, 21 Jan 2006 00:32:00 GMT

Someone asked about adding the "Olson" database of timezones to Windows, and I learned about tzedit.exe and the registry keys. Tzedit.exe is a simple tool that allows you to add or edit the time zone registry entries, which live in the registry at HKLM\Software\Microsoft\Windows NT\CurrentVersion\Time Zones\...

Windows KB 317211 discusses various ways of changing time zone data, but the gist of it is that keys are added or removed from the Time Zones key in the registry. If one wanted to put the whole Olson database in the registry it should be reasonably straightforward to make a tool to do so (but your time zone drop down would get pretty big so I'm not sure how it'd impact usability).

The cryptic part of the registry information is the "TZI" value, but KB 221542 provides a VB program to muck with time zone data, which defines the following type to describe this data, so that should be helpful.

Private Type REGTIMEZONEINFORMATION
   Bias As Long
   StandardBias As Long
   DaylightBias As Long
   StandardDate As SYSTEMTIME
   DaylightDate As SYSTEMTIME
End Type

So far I'm not aware of any tool that actually imports the Olson data, but by putting together these pieces it should be possible.

It should be noted that time zone data isn't simple to work with. Governments keep changing their rules for things like daylight savings time and what the time zone offset should be in which places. Also these KB articles are referring to shipping versions of Windows, I'm not sure how the behavior may differ in Windows Vista, so caveat programmer.