The first time I ran my histogram visualizer I asked for a Cauchy distribution with a minimum of -10 and a maximum of 10, and of course I got a graph that looks much like the one from my article of last week: Looks perfectly reasonable; I guess my program is correct right out of the gate, because I am...

Occasionally when I'm debugging the compiler or responding to a user question I'll need to quickly take apart the bits of a double-precision floating point number. Doing so is a bit of a pain, so I've whipped up some quick code that takes a double and tells you all the salient facts about it. I present...

The DateTime struct represents dates as a 64 bit number that measures the number of “ticks” since a particular start date. Ten million ticks equals one second. That’s a quite high degree of precision . You can represent dates and times to sub-microsecond accuracy with a DateTime, which is typically more...

Well, enough chit-chat, back to programming language design. Suppose you’re building electronic piano software. As we’ve discussed before, the “equal temperament” tuning for a piano goes like this: the 49th note from the left on a standard 88 key piano is A, and its frequency is 440 Hz. Each octave...

User: Recently I found out about a peculiar behaviour concerning division by zero in floating point numbers in C#. It does not throw an exception, as with integer division, but rather returns an "infinity". Why is that? Eric: As I've often said, "why" questions are difficult for me to answer. My first...

I got a question recently about where in the .NET framework the "special numbers" were defined. The questioner was actually asking about the Double.NaN , Double.PositiveInfinity , etc, special values for floating point numbers. Of course there are other "special numbers" defined by the framework, such...

One more thing -- I said earlier that the VBScript float-to-string algorithm was a little bit different than the JScript algorithm. We can demonstrate quite easily by comparing the outputs of two nigh-identical programs: ' VBScript print 9.2 * 100.0 < 920.0 print 919.9999999999999 < 920.0 print...

I went to Joel Spolsky's geek dinner at Crossroads the other night, which was a lot of fun. I didn't get much of a chance to chat with Joel, as he was surrounded by a cadre of adoring fans three deep the whole time. I mostly hung out with KC and Larry and some other attendees and had an interesting talk...

A reader also asked the other day why it is that in VBScript, CSng(0.1) = CDbl(0.1) is False . Forget about binary floating point for a moment. Suppose that we had two fixed-point decimal systems, say one with five digits after the decimal place and one with ten. You want to represent one-third. In our...

I've been getting lots of mail, questions and pointers to interesting articles on some of the trials and tribulations of using floating point arithmetic correctly. Please do keep it coming! Though I am certainly no expert in this area, I'm happy to take a crack at any questions you might have. To sum...

A number of readers asked for an explanation of my offhand comment that Benford's Law can be used to show that binary is the best base for doing floating point math. One of the desired properties of a floating point system is that the "representation error" is as small as possible. For example, suppose...

While I was poking through my old numeric analysis textbooks to refresh my memory for this series on floating point arithmetic, I came across one of my favourite weird facts about math. A nonzero base-ten integer starts with some digit other than zero. You might naively expect that given a bunch of ...

A month ago I was discussing some of the issues in integer arithmetic , and I said that issues in floating point arithmetic were a good subject for another day. Over the weekend I got some questions from a reader about floating point arithmetic, so this seems like as good a time as any to address them...

A number of people have pointed out to me over the years that VBScript's Round function is a bit weird. It seems like it should be pretty straightforward -- you pick the integer closest to the number you've got, end of story. But what about, say, 1.5? There are two closest integers. Do you go up or down...

Try this in JScript: window.alert(9.2 * 100.0); You might expect to get 920, but in fact you get 919.9999999999999. What the heck is going on here? Boy, have I ever heard this question a lot. Well, let me answer that question with another question. Suppose...