How to Work Around Problems Serializing DateTime in XML [Anthony Moore]

How to Work Around Problems Serializing DateTime in XML [Anthony Moore]

  • Comments 8

I wrote a while ago about problems with the way DateTime worked with features like DataSet, XML Serialization (WebServices) and XmlConvert. Basically, DateTime is always treated as a local time by these XML-based systems, which means that you get incorrect results if you use UTC or whole dates with them. What makes this worse is that it is actually a best practice to the UTC times rather than local times. People often don’t discover this problem until they are very invested in a technology and test their application with different tiers in different time zones for the first time.

 

At the time of the previous post, there was discussion about whether this would be fixed in a service pack. Right now, this is very unlikely to happen, because these problems are all corrected in Whidbey and we are getting close to it being released, so the value of a fix in servicing is much reduced.

 

The Beta 2 release will be available fairly soon. It will be a release of at least the quality level of V1.0 Beta 2. Many companies chose to deploy live applications on the V1.0 Beta 2 and many are choosing to do so with the Whidbey Beta 2, as the quality level will be very close to that of the final release. So, a work-around to consider for all of these problems is to adopt Beta 2 a little sooner than you were planning.

XmlConvert

There is a fairly easy work-around to this problem if you call XmlConvert directly. Instead of this:

 

String output = XmlConvert.ToString(dateTimeInput);

 

Call this:

 

String output = dateTimeInput.ToString(“yyyy-MM-ddTHH:mm:ss.fffffff”);

 

This is a valid XML date format, and is almost identical to what XmlConvert currently does, without the problematic local time zone. This is suitable for whole Dates and UTC dates, and you can use XmlConvert.ToDateTime on the other end.

 

It is slightly more correct to use this format for UTC times:

 

String output = dateTimeInput.ToString(“yyyy-MM-ddTHH:mm:ss.fffffffZ”);

 

However, this one won’t work with XmlConvert.ToDateTime on the other end as expected, because it will default to converting this to local time for you. If you want to use this format, use this call instead of XmlConvert.ToDateTime:

 

DateTime output = DateTime.ParseExact(stringInput,

                                      “yyyy-MM-ddTHH:mm:ss.fffffffZ”,               

                                      DateTimeStyles.AdjustToUniversal);

 

This is not so close to what XmlConvert.ToDateTime does. It calls ParseExact with a set of 24 different formats of which this is one. If you want to support multiple formats, pass in an array with the XML formats you want. Another option is to call XmlConvert.ToDateTime and then call ToUniversalTime on the result, but this has a reliability caveat (see below).

 

Web Services (XML Serialization)

 

It is harder to work around this problem in Web Services. Here are some options for using a UTC or a whole date with Web Services.

 

  1. Internally use the data as a DateTime, but serialize it either as a String using the format mentioned above or as an Int64 tick count.
  2. Call DateTime.ToLocalTime on the DateTime before putting it in the serializable class and call DateTime.ToUniversalTime after taking it out. This will effectively “cancel out” the adjustment, and can be used whether you are dealing with a whole date or a UTC time.
  3. For whole dates where you do not need to serialize the time portion, you can change the XML schema to indicate that it is a date rather than a date and time, which will change the format to “yyyy-MM-dd”, which will serialize correctly.
  4. For UTC times, you can have an extra property on the class that exists for serialization only. This work-around is outlined in this article.
  5. Make all machines use the same time zone.
  6. If you have a chance to pre-process the XML before it is sent out, you can manually strip out the time zone offset out of the XML text. For example, a typical XML date and time looks like this: “2005-01-28T03:14:42.0000000-07:00”. You can use a Regex to remove the “-07:00”. You do not need to re-inject anything on the other end, as no adjustment is made if there is no time zone information. Do not try to replace the time zone offset with “Z” or “+00:00”. While technically a more correct representation, the existence of time zone information will cause the serializer to do an extra conversion to local.

 

Obviously some of these work arounds are quite horrible. For Web Services I would recommend (1) for UTC times and (3) for whole dates. Options (2) and (4) have a reliability caveat explained below.

 

DataSet

 

DataSet is the hardest technology of the three to work around this problem. Some options:

 

  1. Change the column types to be Int64 or String.
  2. Call DateTime.ToLocalTime on the DateTime before putting it in the DataSet and call DateTime.ToUniversalTime after taking it out. This will effectively “cancel out” the adjustment, and can be used whether you are dealing with a whole date or a UTC time.
  3. Make all machines use the same time zone.
  4. Use Remoting to serialize the DataSet in binary form. This also has performance benefits. This KB article has an example.
  5. If you have a chance to pre-process the XML before it is sent out, you can manually strip out the time zone offset out of the XML text. For example, a typical XML date and time looks like this: “2005-01-28T03:14:42.0000000-07:00”. You can use a Regex to remove the “-07:00”. You do not need to re-inject anything on the other end, as no adjustment is made if there is no time zone information. Do not try to replace the time zone offset with “Z” or “+00:00”. While technically a more correct representation, the existence of time zone information will cause the serializer to do an extra conversion to local.

 

This is the most difficult situation, because all of these work-arounds have problems. Option (1) involves bypassing the database type system. Option (2) has a reliability caveat explained below. I would actually recommend (4) or (5) for this technology.

 

Reliability Caveat for Local Conversions

Some of these work-arounds involve conversion between the local time and UTC and others involve using UTC and ensuring that it is never converted. Some of the work-arounds involving local conversion are a little easier, but they should not be used if you have an application that needs to be 100% reliable 24/7. There is one hour per year where local clock times are inherently lossy, because there is an hour that gets repeated when you wind the clock back as you leave daylight savings time. In V1.0 and V1.1, the DateTime did not differentiate these two times, so even temporarily converting to local can cause data loss.

 

In the Whidbey release, the data format has been modified to ensure that local times are not lossy in this situation and can differentiate between these two times. Despite this, UTC is still the recommended way to persist times and do arithmetic on them, because it works across changes in the time zone and the arithmetic is not affected by daylight savings time transitions.

  • Wait a sec, so .NET 2.0 doesn't fix the problems with date time serialization during remoting?? Eek!

    For example, we often want to store just _date_ values in the database. We return a dataset from the server in California to, say, a client in Florida. Florida fills out a datetime value with a date (i.e. March 7, 2005 at 12:00a), and sends the dataset back to California... where it arrives as March 6, 2005 at 9:00p).

    Considering that we're using Windows GUI databinding doing manual massaging on fields is very anti-good-development-practices behavior...
  • Whidbey does solve the problem you describe with serializing just _date_ values in a database. The Unspecified mode will turn off these conversions and whole dates will be preserved. With DataSet, you can specify whether the whole column should be in the mode.

    Let me know if there are still concerns with this.
  • This is ridiculous. The fact that whomever built the XML Serialization in 1.0/1.1 in the first place broke the definition of DateTime (it clear states that is meant to be akin to the decimal value of a 'money' ... currency is something you add on ... as was the intent of time zone in the case of DateTime), but now you aren't fixing it? So many elegant things in .NET and then there are these absolutely ridiculous broken nasty warts that aren't being taken away.

    Most likely this is due to compatibility BUT people need to go through the excercise of 'porting' onto 2.0 to begin with ... they are making the effort to change, so just fix the damn thing and reprimand the people who wrote the XML Serialization code in the first place.

    I would have considered some (if feasible) some form of compiler warning system where people could turn on a 'I'm porting code from 1.1, warn me for compatibility' and it would warn on types containing date times that are heading into the serializer .... if not the compiler than at least some form of analysis tool for goodness sake.
  • Also if compatibility between 2 connected systems is required, 1 that is .NET 1.1 and the other 2.0, that support a configuration for the app (config file, or maybe flag on the xml serialization class) that would handle datetime in the same fashion as 1.1.

    OR

    Better is to have it so the XML format contains a version number (don't recall if it does or not) ... IF there is no version number (assuming 1.1 didn't do that ... which it should have), then remain compatible to 1.1 .... the goal is that you can change the serialization mechanics based on the version ... under the hood in libraries, use the strategy pattern as an implementation mechanic so you could swap out the mechanics based on the version.

    Unfortunately this really nasty item is being perpetuated into the future. Kill it while you can PLEASE (probably too late).
  • I definitely agree that it is extremely unfortunate what happened with XML DateTime formats in V1.0 and V1.1. It was a big mistake. Just leaving a few "z" characters off the format strings would have saved us a whole world of pain.

    It would be great if we did have a migration infrastructure like you suggest. We frequently run into agonizing correctness vs. compatability tradeoffs like this that make us ask questions about something like this. However, with Whidbey we have to stick to opt-in new featueres, the Obsolete attribute and FxCop as our versioning tools. Thanks very much for your feedback about this.

    I don't agree that we are perpetuating a serious problem into the future. This will be fixed "by default" in Web Services V2.0. In XmlConvert, there there are a couple of methods that we are effectively deprecating, so warnings will tell you to use a different method that does not have this bug. In WebData you also need to opt-in to the solution, but in that case it is much better to specify the nature of DateTime for the whole column anyway.

  • Este post es en respuesta a este caso de Edgar. Desafortunadamente no lo pude poner como comentario en

  • PingBack from http://paidsurveyshub.info/story.php?title=bcl-team-blog-how-to-work-around-problems-serializing-datetime-in-xml

Page 1 of 1 (8 items)