Browse by Tags

Tagged Content List
  • Blog Post: Precision and accuracy of DateTime

    The DateTime struct represents dates as a 64 bit number that measures the number of “ticks” since a particular start date. Ten million ticks equals one second. That’s a quite high degree of precision . You can represent dates and times to sub-microsecond accuracy with a DateTime, which is typically more...
  • Blog Post: Fun with Floating Point Arithmetic, Part Five

    I went to Joel Spolsky's geek dinner at Crossroads the other night, which was a lot of fun. I didn't get much of a chance to chat with Joel, as he was surrounded by a cadre of adoring fans three deep the whole time. I mostly hung out with KC and Larry and some other attendees and had an interesting talk...
  • Blog Post: Fun with Floating Point Arithmetic, Part Four

    A reader also asked the other day why it is that in VBScript, CSng(0.1) = CDbl(0.1) is False . Forget about binary floating point for a moment. Suppose that we had two fixed-point decimal systems, say one with five digits after the decimal place and one with ten. You want to represent one-third. In our...
  • Blog Post: Floating Point And Benford's Law, Part Two

    A number of readers asked for an explanation of my offhand comment that Benford's Law can be used to show that binary is the best base for doing floating point math. One of the desired properties of a floating point system is that the "representation error" is as small as possible. For example, suppose...
  • Blog Post: Eric's Complete Guide To VT_DATE

    I find software horology fascinating. The other day, Raymond said "The OLE automation date format is a floating point value, counting days since midnight 30 December 1899. Hours and minutes are represented as fractional days." That's correct, but actually it is a little bit weirder than that. I...
  • Blog Post: Why does JScript have rounding errors?

    Try this in JScript: window.alert(9.2 * 100.0); You might expect to get 920, but in fact you get 919.9999999999999. What the heck is going on here? Boy, have I ever heard this question a lot. Well, let me answer that question with another question. Suppose...
Page 1 of 1 (6 items)