Wednesday, September 26, 2007 11:59 AM
Lori Pearce [MSFT]
Why even computers get the math wrong from time to time
There was a nice little blog post on Good Morning Silicon Valley that led to another, really interesting blog post titled "Arithmetic is Hard--To Get Right." Turns out there are "...exactly 12 instances out of the 9 quintillion possibilities it [Excel] goes completely bonkers" and 77.1 x 850 is one of them.
I tried it and did indeed, get 10000 instead of 65535.
The original post is here.