Ed Wilson: When 6 is 5.99999999999999


Ed Wilson: When 6 is 5.99999999999999

  • Comments 3

One of the great things about being the Microsoft Scripting Guy, is answering the hundreds of e-mails a week that I receive. Yes, believe it or not, I try to answer all the e-mail that comes to the Scripter@Microsoft.Com email alias. Many times, the answer is, “dude, I am totally swamped, and would love to write your script for you, but I cannot. Why don’t you try the Scripting Guys Forum instead.” or words to that effect. If I am not very busy, I may very well go ahead and write the script for the person—after all they are my fans, and customers.

Occasionally, however, I receive an e-mail that causes me to stop what I am doing and investigate. This was the case yesterday, when I received an e-mail with this code in it.

a = 4

While a < 6

    Wscript.Echo a

    a = a + .1

Wend

It seemed innocent enough. Until I ran it! That is when I saw that the code would return the following information:

4
4.1
4.2
4.3
4.4
4.5
4.6
4.7
4.8
4.9
5
5.1
5.2
5.3
5.4
5.49999999999999
5.59999999999999
5.69999999999999
5.79999999999999
5.89999999999999
5.99999999999999

Cool, I thought! It turns out that the problem is with floating point numbers. Computers are binary, and they do not do fractions very well. In fact, they really do not do fractions at all. They convert everything to binary. So, some fractions work out perfectly, and others do not. There is no exact binary representation for 1/10 or 1/3 – the question then is how far out do you want to go before you have to do the rounding. If you are old enough, you may remember the so called “Pentium bug” from way back in 1994. Because of the floating point number issue, every time you add 1/10 you are actually adding the number nearest to 1/10 that can be represented in a 52 bit number. Eventually these errors add up as shown in our sample.

Interestingly enough, if had let the script run a little bit longer, we would have seen it “fix itself,” at least for a little while. For more information on this topic, take a look at Eric Lipperts Blog.

  • "...after all they are my fans, and customers."

    VANITY, dude!

  • Thanks Ed.  A few other things you might want to mention:

    (1) VBScript supports a Currency data type which does not have this issue. The currency type accurately represents decimal numbers, but only to four digits of precision after the decimal point. (In .NET programming, use the System.Decimal data type; there is no Currency in the framework.)

    (2) Here's an article from my second day of blogging, back in 2003, that explains how this works in JScript:  http://blogs.msdn.com/ericlippert/archive/2003/09/15/why-does-jscript-have-rounding-errors.aspx

    (3) Finally, note that VBScript has some heuristics in its string formatting code which JScript does not. VBScript will detect SOME situations in which rounding error has occurred and will convert the 5.9999... to "6" when printing the result. However, it cannot detect all such situations, for the obvious reason that the CORRECT answer to your calculation might be "5.9999...". You shouldn't rely on those heuristics; if you need decimal arithmetic, you should use the currency type in VBScript.

  • My book is about to be published and mentions MSFT ... in other words it is a real book, about real users.  I think it might be of interest to MSFT, but I am unsure who to contact.  Either at MSFT or possibly Crispin Porter.  Any suggestions would be greatly appreciated.

Page 1 of 1 (3 items)
Leave a Comment
  • Please add 2 and 5 and type the answer here:
  • Post