Eric Fleegal's WebLog

. . . .

Stanley110 asks...

Sorry I haven't had time to post in quite a while.  I’ve been developing the new Xbox Live Arcade for the upcoming Xbox 360, a project that has taken considerable time and effort.

 

Stanley110 asks:

A) Excel uses double precision. What benefit with respect to accuracy or precision of the calculated result is there with respect to single precision?

B) Electronic calculators use single precision? Is this true?

C) Aside from computer speed and things like that, is there any difference in the accuracy and precision of an aritmetic calculation when it is by double precision than by single precision.

 

Short Answer to Question A: The precision benefits of double-precision over single precision are exactly as the name suggests: at least double the precision of single precision.  Accuracy is a different question altogether.  With carefully coded algorithms, single precision can yield very accurate results; however, most users (even most computer scientists) are not trained to devise such algorithms for all but the simplest cases.  Most of the time (though not always), if you perform the same computation in double and single precision, the double precision result will usually be more accurate.  I hate to make a blanket statement like that, so PLEASE note the qualifiers before flaming me with email :-)

 

More info on floating point
http://en.wikipedia.org/wiki/IEEE_floating-point_standard

http://en.wikipedia.org/wiki/Talk:Computer_numbering_formats

 

Accuracy and precision are not the same thing

http://en.wikipedia.org/wiki/Accuracy

http://en.wikipedia.org/wiki/Talk:Accuracy_and_precision

 

(yes, I am a fan of Wikipedia; its freaking brilliant!)

 

Excel uses double precision mainly because it’s what’s available on most architectures.  Moreover, if care is taken to properly account for error, doubles are precise enough for many financial computations, especially the kind used by most Excel users.  However, even simple tax or interest computations can be perturbed by the use of double precision (so be careful and check your results!)

 

Short Answer to Question B: No; well, maybe some cheap calculators given away in a box of Cap’n Crunch, but useful calculators will have at least 12 decimal digits of precision ( single prec has only about 7; log10(2^24) ).  Most inexpensive calculators use doubles or extended precision since the chips for it are fairly inexpensive.  Really nice calculators use extended double, quad-precision or even provide decimal-floating point precision with 28 decimal digits or more of precision.  Some even use rational number systems under certain circumstances to represent numbers like 1/10, 1/3 etc.  The built in Windows calculator, for instance, provides 32 decimal digits of precision and uses rationals for certain computations.

 

Gossip: I heard a rumor that Excel may (soon?) provide computations using .NET’s decimal type.  But I haven’t been able to confirm this.  So naturally I must spread the rumor.

 

Short Answer to C:  I recommend the articles above.  Keep in mind that on many systems, double precision computations are just as fast or even faster than single precision computations. 

Published Thursday, September 01, 2005 1:29 AM by ericflee

Comments

No Comments
Anonymous comments are disabled

© 2008 Microsoft Corporation. All rights reserved. Terms of Use  |  Trademarks  |  Privacy Statement
Microsoft
Page view tracker