DateTime FAQ Entries

DateTime FAQ Entries

  • Comments 28

I have recently created some new DateTime FAQ entries to address some questions people have about using DateTime on blogs. Our web site is in transition, so I'm posting some of these into the blog. However, they will eventually be rolled into the DateTime FAQ on the BCL web site.

Why is DateTime not always UTC internally even for cases when it is used a local time?

 

People often ask why DateTime is not always UTC internally, or why we don’t change it to be so now. This would address a number of problems with the DateTime, but there are various reasons why we can’t do it. The short answer is that we probably could have designed DateTime this way from the beginning, but it would not possible to now transmute the DateTime into working this way without creating unacceptable compatibility problems.

 

In V1.0 the decision was made to have a purely numeric DateTime, and the context of the time zone was to be external. The driving reasons behind this were compatibility, simplicity and performance. Compatibility is relevant because the key legacy systems that .NET needed to interoperate with, COM, Win32 and databases, also had numeric-only instances. Marshaling from one to the other would have required a guess about the time zone context that would be wrong in many cases. There was no easy guess either because the convention for Win32 times was universal and the convention for COM times was local.

 

Simplicity and performance were relevant because it is much simpler and faster to be dealing with a purely numeric value, rather than having many methods converting back and forth between a local view and a universal representation.

 

After shipping V1.0 it quickly became obvious that the combination of this simple DateTime representation, combined with the fact that most public APIs were using local instances created a big problem. Local instances have reliability problem during daylight savings boundaries and whenever time zones change.

 

Serious investigation was done as to whether DateTime could change to be UTC internally and represent an absolute point in time. However this invariably resulted in behavior changes from V1.0 that would cause reasonably functioning code to break. It also created a “performance compatibility” problem in that it would not be possible to retain the performance characteristics of the class to any reasonable degree with all that extra logic going on.

 

Instead we chose a solution in Whidbey that solved the most difficult problems in the class and presented minimal compatibility risk. Local instances can now be converted back to universal time without data loss, and you can annotate whether the time is local universal or unspecificed. Unfortunately compatibility has also meant that using this updated information has to be opt-in in most cases, so you need to pass in new flags and values to consume this information in technologies like XML Serialization.

 

Admitted, this is a very unfortunate situation. You still need to know about a lot of the details and pitfalls of DateTime in order to use it correctly. If we could have seen the full implications of the earliest design decisions, we may well have begun with a UTC based DateTime instead. As it is, it will be possible to get reliable DateTime usage in Whidbey, but you have to know what you are doing. We are planning some FxCop rules to help out with this.

 

All this being said, it may be possible to create an entirely new class that does represent an absolute point in time and does have time zone context. This may be considered for a post-Whidbey release.

What is the Recommended Way to Store a DateTime in Binary?

The way most people serialize a DateTime in Binary form is to use ticks. This the simplest and fastest way to store a DateTime, although this is not the recommended practice for local times, which are discussed below.

 

public Int64 StoreDateTime(DateTime value) {

      return value.Ticks;

}

 

public DateTime ReadDateTime(Int64 value) {

      return new DateTime(value);

}

 

In Whidbey, the DateTime has additional information about whether it is Local, Universal or Unspecified. To preserve this information there are new ToBinary and FromBinary APIs:

 

public Int64 StoreDateTime(DateTime value) {

      return value.ToBinary;

}

 

public DateTime ReadDateTime(Int64 value) {

      return DateTime.FromBinary(value);

}

 

These techniques will not work well for local times, and will be inconsistent with recommended techniques for storing a DateTime in Text (see below). It is not recommended because a Local DateTime instance exists in the context of the current machine’s time zone, so if this changes while the DateTime is persisted it will render the data invalid. The most typical scenario here is if the reader and writer are different machines in different time zones.

 

If you deal with Local DateTime instances, the recommended way to handle this is to store them as UTC:

 

public Int64 StoreDateTimeLocal(DateTime value) {

      return value.ToUniversalTime().Ticks;

}

 

public DateTime ReadDateTimeLocal(Int64 value) {

      return new DateTime(value).ToLocalTime();

}

 

This obviously makes things difficult if you don’t know whether the data is local at compile time. A binary serialization API that is more consistent with the new text serialization APIs is still being considered for the Whidbey release.

What is the Recommended Way to Store a DateTime in Text?

Most of the standard formats for the DateTIme do not preserve all the information in the DateTime. The also vary from one culture to another. Thus, the recommended format for storing a typical DateTime is like so;

           

public string StoreDateTime(DateTime value) {

// e.g 2003-10-26T14:33:41.1234567

return DateTime.ToString(“yyyy-MM-ddTHH:mm:ss.fffffff”,

CultureInfo.InvariantCulture);

}

public DateTime ReadDateTime(String value) {

return DateTime.Parse(value, CultureInfo.InvariantCulture);

}

 

This format is useful, both because it is a standardized format, used in XML among other standards, and because with the 7 decimal places, it retains all the precision of the DateTime.

 

There is a critical piece of information that is lost here, which is the context of the time zone of the DateTime, if it is available. If you are communicating with a 3rd party system, or if your application has writers and readers on different time zones, it is recommended to use a slightly different format.

 

If you handle your Dates in Universal time, use this format:

 

public string StoreDateTimeUtc(DateTime value) {

      // e.g 2003-10-26T14:33:41.1234567Z

      return DateTime.ToString(“yyyy-MM-ddTHH:mm:ss.fffffffZ”,

CultureInfo.InvariantCulture);

}

public DateTime ReadDateTimeUtc(String value) {

return DateTime.Parse(value, CultureInfo.InvariantCulture,

DateTimeStyles.AjdustToUniversal);

}

 

Note that it is important to pass AdjustToUniversal to this routine, because the output format identifies the time as UTC, and converted to local time by default by Parse if they have any sort of time zone marker.

 

If you handle your Dates in local time, use this format:

 

public string StoreDateTimeLocal(DateTime value) {

      // e.g 2003-10-26T14:33:41.1234567-07:00

      return DateTime.ToString(“yyyy-MM-ddTHH:mm:ss.fffffffzzz”,

CultureInfo.InvariantCulture);

}

public DateTime ReadDateTimeLocal(String value) {

return DateTime.Parse(value, CultureInfo.InvariantCulture);

}

Note that in this case, if the reader and writer are in different time zones, because Parse converts to local time by default, it will be effectively adjusted to the local time as it is parsed in.

 

Beware: do NOT use a UTC format to store a local time or a Local format to store a UTC time. It will actually seem to work at first, but it will not correctly adjust when time zones change, and 3rd party systems reading the information will get the wrong date.

 

In Whidbey from Beta 1 onwards, there is a simpler way to do all this:

 

public string StoreDateTime(DateTime value) {

      return DateTime.ToString(“o”, CultureInfo.InvariantCulture);

}

public DateTime ReadDateTime(String value) {

return DateTime.Parse(value, CultureInfo.InvariantCulture,

DateTimeStyles.RoundTripKind);

}

 

The “o” format is a new shortcut. In Whidbey you can store whether the DateTime is Local, Utc or Unspecified in the DateTime instance. The “o” format will pick a different format depending on what the Kind property of the DateTime is. The RoundTripKind style will effectively preserve what the kind was when persisted based on the format read in. The above formats are not invalid, but since they can only handle one type of DateTime, this the preferred means of persistence if you get DateTime instances from unknown sources, such as if you are writing a serialization engine.

 

  • [Anthony Moore]

    I forgot to put my name in the blog. I would be interested in any feedback on this content.
  • Quote: "All this being said, it may be possible to create an entirely new class that does represent an absolute point in time and does have time zone context. This may be considered for a post-Whidbey release."

    Given the seriousness of the problems surrounding this whole issue, wouldn't it be better to add such a second DateTime-like struct already in Whidbey? Post-whidbey is VERY far away... What are the arguments to postpone it?
  • ::If we could have seen the full implications
    ::of the earliest design decisions, we may
    ::well have begun with a UTC based DateTime
    ::instead.

    Interesting. Someone was sleeping here.

    You COULD have seen and SHOULD have seen this. This is a TYPICAL problem that ihas been plaguing the programmer community for a long time - and too vendors have been tremendously ignorant to this.

    Example?

    SQL - DateTime and timezone information. Added to the standard years ago, soon YUKON will be probably the first (!) database to support them.

    I myself got stuck with this about 10 years ago, and have been using things UTC ever since then.

    I can only support Luc here - put the additional classes into Whidbey. You have the time. Post-Whidbey is YEARS away.
  • I agree, we should have a new DateTime struct that uses UTC sooner rather than later.
  • We've been kicking around the same set of problems here. Most of the time it's not a big issue but having a "Date" type would be useful sometimes. Birthday is a classic example, how do you represent this in a database if the system has timezones on different days during use? You can do this by storing midnight UTC and using context to say if timezone is significant (e.g. it's not on a birthday) but it does stink slightly.
  • Brad Abrams leads us to a new BCL Blog posting on DateTime by Anthony Moore.
  • Considering the complexity of getting the correct behavior from DateTime (yikes), it does seem like you should seriously consider adding a DateTime2 (or DateUniversalTime or UniversalDateTime or DateTimeUtc) to the Whidbey release.
  • Yup, it is something we will have to think through carefully, but we are interested in ensuring that all of the applicable scenarios for DateTimes are supported Keith: We'll keep you all posted when we have more details on where we plan to head.
  • I added a new post as a reply to this feedback:
    http://weblogs.asp.net/bclteam/archive/2004/06/01/145487.aspx
  • Microsoft has got the problem wrong for some 20 years and now they try to give recommendations...some of which are wrong again! What can I say?

    We have tried (with no success) to pitch a solution to Microsoft, which is part of our very generic library based on tensor theory. The library is aimed for graphics applications, but time can be also represented as field (like gravity, or any other vector) and can undergo tensor transformations (at quite high abstraction level). By doing so, you do not care how you represent the time, or even that it is time that you are dealing with - it may be just a coordinate on a screen or a gravity. All the algorithms are generic (and lighting fast).

    The result - yes, we can handle time and conversions to local including daylight saving adjustment any way you want. This is not only with much higher precision than the proposed methods, but correctly and consistently. To demonstrate this, please have a close look at two sample time scales of the graphs illustrated here: www.24x7.com.au/2d.htm
    You will see that the local time is not linear; it is not even continuous. Because it is not continuous, it does not have inverse function (no conversion from Local time to UTC) - it is simply not possible and that's basic maths (non-continuous function does not have an inverse).

    Dear Microsoft, if you have no idea about the problem, please consult someone who knows something about it. I would be happy to help, but please, do not make recommendations if you got it wrong every time for the last two decades, okay?

    Kind Regards,

    Pawel Achtel
    3D Software Development
    www.24x7.com.au
  • I've got to agree that the DateTime as-is simply leads to a Pit of Failure, not of Success, and for that reason is a bad API. It's just too easy to get wrong.

    I have no idea about this academic 'tensor' thing above. Regardless, UniversalDateTime makes more sense. Having one type with so many interpretations sucks, the UTC-ness is part of the type IMHO.

    Your new ToBinary is just as breaking if someone tries to use it as the Ticks (which most people seeing an Int64 time field would do). Just fix it by migration, leave the old class there for those dealing with local time only or backwards compat. Let the world move forward.

Page 1 of 2 (28 items) 12