Why Not Add a New DateTime Date Type in Whidbey?

Why Not Add a New DateTime Date Type in Whidbey?

  • Comments 20

[Anthony Moore]

Thanks very much to those who have posted feedback on this issue:

http://weblogs.asp.net/bclteam/archive/2004/05/21/136918.aspx

This reply to the feedback got quite lengthy, so I started a new entry on it.

>> "You COULD have seen and SHOULD have seen this"

We would definitely agree that this is a bad situation and we should have seen this in the first version of the product. The toughest bit is that for the V1.0 and V1.1 products, it is hard for us to even provide good guidelines for completely reliable DateTime scenarios. Although there is dissatisfaction that the currently Whidbey plan is sufficient, it does at least allow for guidelines to be created that can actually be followed.

Three replies emphatically ask that we consider adding a complete new DateTime type in Whidbey that is UTC internally. It is probably worth sharing some of the existing discussion on this point.

Once the full extent of the DateTime problems were understood, a great many options for correcting it were considered. These included changing the implementation to be UTC internally and deprecating the existing data type and replacing it with a completely new one.

It sounds like people are bought into the notion that retroactively changing the internal implementation of the current DateTime to always be UTC and also staying sufficiently compatible accross versions is not possible. This option was thoroughly explored and was a plan of record for a brief time until all the compatibility issues with it were fully explored.

Another option explored was switching over every API that returned a Local instance to returning a UTC instance. Of the 84 APIs that took or returned DateTime in the V1.0 NDP, this would have involved deprecating 81 of them, and replacing them with versions that had an identical signature, but a DateTime instance that was UTC instead of local. This would have created a lot of disruption to owners and consumers of the APIs alike. We would need a very good reason to do this. It was seen as preferable if there was a way allow people to get reliable behavior without having to switch every call involving a DateTime over to using a different API.

This lead to the current plan of record, which is compatible enough to allow existing APIs to get the extra reliability without needing to change either the signature or meaning of their results.

The option of deprecating DateTime and replacing it has many of the same undesirable consequences. More than just deprecating the type istelf, you are then making every API that takes or returns the API obsolete and making it a requirement to switch over to somthing new. We could do this, but we would need a very good reason and a lot of lead time. Given that it was possible to solve the key reliability problem by just making the Local->UTC conversion non-lossy, this was seen a better alternative.

It is suggested that there is still time in Whidbey to add a new DateTime. Actually, there probably is enough time to add such a new type. However, I could get it to you even more quickly by sending you a sample, and it would be no less useful. The reason is that while there may be time to add this new data type, there is not the time to integrate it anywhere near as deeply as you would want. You would need to integrate it with Remoting, Serialization, XML Serialization, the Debugger, XML Convert, Type Converters, etc. etc. You would also want to add several hundred new APIs in technologies like ASP.NET and System.Net to provide versions of their APIs that would accept this new type. There is not enough time to get this level of integration with the rest of the product, so if we did add this type, it would not be of much more use than some sample code.

Given the cost of actually trying to replace something as widely used as DateTime, it is not even clear if it is a good option long term. Part of the value of the .NET Framework is the standardization and simplicity in having, for example, just one way to represent a string. A 2nd DateTime type would erode that and would need a very compelling reason to exist.

A more likely option is that there might be a sort of supplemental time type that is not destined to replace DateTime, but to interoperate well with it, and to be recommended for scenerios where you are doing cross-time-zone stuff. Part of the problem here also is that this is tied to another large work item, which is time zone enumeration. This is quite an expensive work item for us, as outlined in the DateTime FAQ: http://www.gotdotnet.com/team/clr/bcl/TechArticles/techarticles/DateTimeFAQ/FAQ.aspx

This will probably sound defensive and adverserial. Believe me, I don't see this as a good situation and I have actually asked all these same questions you are asking now many times, and we have tried to thoroughly explore them all. But we do feel that the current solution for Whidbey draws the best line between fixing unreliable usage, maintaining compatability across versions and keeping the framework simple and consistent.

I look forward to further discussion of this issue.

  • "simplicity in having, for example, just one way to represent a string"

    Bad example. See http://weblogs.asp.net/shawnfa/archive/2004/05/27/143254.aspx
  • We just went through these problems, and decided to roll our own DateTime class with the appropriate behavior.

    We also recognized another serious shortcoming to the DateTime class; that is, that a time is always required. Sometimes you just want a Date without a time, and then without timezone considerations. Having a specific type for a timeless and timezoneless Date was a pretty significant win for us.
  • In the long run, I think that adding a separate type, and deprecating the non-timezone-aware DateTime, would be by far the best solution, even if it would be labor-intensive.

    In the short run (Whidbey timeframe), why not take steps in that direction? Include a UtcDateTime class in the Whidbey BCL, integrate it superficially with a handful of APIs, and suggest (without actually deprecating the old one) that customers -- *and third-party library vendors* -- start using the new UtcDateTime instead of DateTime. Nobody will be able to use it fully, since few APIs will support it yet, but developers will have plenty of lead time to start migrating parts of their code toward a UtcDateTime. Then the next (post-Whidbey) Framework could officially provide deep support for UtcDateTime and deprecate the old DateTime.

    If there was a standard UtcDateTime class that everybody could use, third parties and open-source developers would quickly move to fill the gaps in the APIs. Just having that standard class (like the example you gave of a standard String class) would go a long way toward solving the problem in the short term, as well as being a good start toward a long-term solution.

    (BTW, I like Brad's idea of a date-only type with no timezone awareness... put my vote in for adding one of those someday.)
  • Halfway through my comments, I noticed that Joe White had already written what I was trying to convey, and had done a much better job at it.

    I strongly agree that in the end we will need another type ("UtcDateTime" or whatever it will be called). It could live side-by-side with the current DateTime. There is no need to depreciate DateTime at this time; let it be to support the existing APIs. Over time (hopefully partly during the Whidbey timeframe and more during Orcas), the types that work with the existing DateTime type can be extended to work with the new "UtcDateTime".
  • If you can't make the current DateTime work, then add a new one that does work. If you know the problem and choose to ignore it (or even worse, acknowledge it and not fix it) then you have done the Wrong Thing. "It is a lot of work" is not a good excuse for something as important as this.
  • ++ to the other comments.

    A UTC based/aware DateTime type is a necessity for the projects I work on. We can (and will) code our own implementation when the time come for us to fully use .NET, but this issue may be a sticking point on how quickly we go to full use.
  • Would it not be possible to add a property to the existing DateTime class of a type which is TimeZone (e.g. ActiveTimeZone), default this to TimeZone.CurrentTimeZone (for backwards compatability) and introduce a TimeZone.UTC static property (on the TimeZone object) which you could set the ActiveTimeZone property on the DateTime class to, resulting in the DateTime class using UTC?
  • I added a new post as a reply to this feedback:
    http://weblogs.asp.net/bclteam/archive/2004/05/21/136918.aspx

    In short:
    Q: Why Not Add a New DateTime Date Type in Whidbey?
    A: Because after two decades Microsoft still does not understand the problem (yet tries to provide a "solution")

  • Don't deprecate DateTime. DateTime is still very useful for a very common case (local time).

    ADD UniversalDateTime (DateTimeUtc) for the cases where the time zone is known and desired to be UTC. That's it. Add it to the APIs that know they need UTC time. Don't deprecate methods that already take DateTime and assume it's UTC, add the overload and document carefully on the DateTime version.

    I partly agree with String/SecureString scenario. The 'simplicity' of having a single time is provably wrong here. The hacks (yes, hacks) introduced in the original posting as "guidelines" are a worse solution. Instead of getting the problem right it is forcing more ugliness that is extremely easy to miss or get wrong onthe developer using the API.
  • Thanks very much for the feedback so far. This is definitely influencing our thinking on the long term direction this area should take. In particular I had been vacillating about whether a 2nd DateTime type would work, and this feedback has convinced me more that the new data type is probably the way to go forward, either in this version or the next.

    I expressed the concern I had about a 2nd data type poorly, so I will try to re-express this: while you generally want to avoid having two classes for the same logical concept, the more critical thing is to avoid too much scenario overlap so that people don’t know what to use where. The example of String and SecureString is a good one, because the set of scenarios in which you would ideally use one rather than the other has almost no overlap. So, String has a good reason to exist. By comparison, in the Win32 world it was much harder to justify things like BSTR and LPSTR, which had almost complete scenario overlap.

    In the case of DateTime, there is desire for a DateTime that is always UTC. The Whidbey DateTime can be used in this mode, so in a sense it already offers a superset of the raw functionality. The problem is that it is so hard to use. A separate data type should have a strong reason for being. A UTC DateTime has a lot of overlap, as does a pure Date data type, which the Whidbey DateTime can also do if used in the right way. It’s not a show-stopper, but it makes it a harder sell.

    SecureString is another good counter example of providing something before there is time to integrate it broadly. My team also provided this type, so I'm curious to see how it is received. It is currently hooked up to almost nothing else, so it will be post-Whidbey before it is really useful. It would be interesting to hear if this causes more aggravation than the class provides benefit.

    However, a difference here is that we provided SecureString when we were about 90% done with defining new types. Right now we are more like 99% done, so it significantly harder to justify. Another risk with introducing such a fundamental type this late is that there is not enough time for it to bake and gather feedback to detect design problems. It would be terrible to make design mistakes in the new class and have to deprecate that too.

    In reading some of the comments, people don't seem to be picking up that DateTime can be used for UTC times as well as local. In fact that will become the recommended usage pattern. Its problem is just that it is not constrained to only that usage, so it must be used correctly. It is surprising that people are thinking of creating their own types, as you would normally do this because scenarios are just not supported rather than being hard to use. We have significant internal use of DateTime and the only obvious duplication of it (SqlDateTime) was to solve a different functionality gap (Nullability, now solved with Nullable<DateTime>).

    Overall, I think that we do need a new DateTime type. It should have two key characteristics: (A) it should always be UTC internally so that out of the box the serialization and arithmetic is absolute, and (B) it should be able to represent a time zone other than local or UTC if required, which is a very common feature request as well. It is fair to say that the current DateTime could not reasonably be extended to this 2nd scenario. Support for a time zone offset or ID would need to be baked in from the start. The right design probably involves being dependent on this, and unfortunately it is a large work item.

    There is a suggestion to switch to a UTC mode via a static property. This would not work because you can't control everyone's code in the same app-domain. You could write code to depend on it being set one way, but then break by calling a library that depends on it being set another. It is generally better to not have global or app-domain state controlling such fundamental behavior.

    We will continue to gather feedback about this. If there is significant enough feedback on this issue from enough different sources it will obviously still be on the table. It is, however, very late in the product cycle. I’m having to fight hard for each individual method we add at this stage.

    Thanks.
  • Anthony, good summarization of the feedback and the issues. Your proposal for a replacement DateTime (DateTime2?) seems reasonable. I think most following this issue would want to see MS take the necessary time to get the solution right. OTOH as someone mentioned (BradA?) it will be a long time after Whidbey ships before you have the chance to put in this new type. It sure would be nice to nip this problem before it spreads into too much code. BTW, is part B something you could implement in steps ie. support local and UTC time zones at first then back-in support for other time-zones later without it being a breaking change?
  • Thanks, Keith.

    I have been thinking about that very problem. It depends on how the design comes out, and primarily whether you store a time zone ID in addition to the offset. If you don't have the ID, then you end up with times that are precise and presentable, but you couldn't distinguish, for example, between Perth times and Singapore times, which often have the same offset but have different daylight savings rules. Without knowing this you could not adjust appropriately through arithmetic, or preserve the informaiton for its own sake. This might not be a show-stopper.

    Even then you would have to be very careful to enable forward compatability on the serialization so that you could potentially add time zone ids later, and then you might be forced to make any behavior that keys off this new information opt-in for compatability reasons, as we must do for the current Whidbey DateTime changes.

    This feedback, combined with some other discussions with internal teams in the last few weeks, is leading us to consider the question of whether there will be a new DateTime and what it will look like sooner rather than later. We are trying to put a plan together in the next couple of months, because we definitely want to make this better in Longhorn if we can. What might fall out of this is a decision to have a new type, and the rough design of that type.

    It is not out of the question that such a type could be checked into Whidbey if the forward-compatability issues could be sorted out. However, it is admittedly a very long shot.
  • "Overall, I think that we do need a new DateTime type. It should have two key characteristics: (A) it should always be UTC internally so that out of the box the serialization and arithmetic is absolute, and (B) it should be able to represent a time zone other than local or UTC if required, which is a very common feature request as well."

    Having (B) will just resort in the same problems that we have now. Instead, have a method on the new DateTimeUTC like this:

    public DateTime ToLocalTime();

    (which does (B) without mucking things up).

    There also needs to be a
    FromUnixTime(long seconds) and
    long ToUnixTime()
    methods on the DateTime* classes (structs, really). It seems rather pathetic to not have it on those classes.

    RE: overlapping classes.
    I don't see it as a problem. DateTimeUTC would probably be used mainly for database/storage work. DateTime can be used for how it should have been used: representing local time where timezones do not come into play (displaying current time, benchmarks, etc.)

    I'd also like to see a separate Date (and Time, for completeness-sake) classes. It just makes sense to. (and gives better 1:1 with db types)
Page 1 of 2 (20 items) 12