Fun with Floating Point Arithmetic, Part Four

re: Fun with Floating Point Arithmetic, Part Four

Dave Bacher — Tue, 18 Jan 2005 21:03:00 GMT

In general, if you're dealing with money, the correct way is to use an integer multilpied by a scaling factor, instead of an IEEE float of any variety (single or double).

The most common case where you see this problem is when someone's dealing with currency -- it hits double precision inaccuracies a lot, and the runtime tries to compensate for it.

By using fixed point math (multiply the values by 1000, for example, then divide and convert into a float/double once your done), you eliminate any rounding errors, etc. at the expense of a small increase in the amount of code that you have to write.

SQL Server, Oracle and ISO C++ provide mechanisms for handling this. I believe the .NET runtime has support for it as well.

re: Fun with Floating Point Arithmetic, Part Four

Dave Bacher — Tue, 18 Jan 2005 21:04:00 GMT

Oh and that goes for any time you care about significance in any computer math -- never use a single or a double if you care about how many digits come after the decimal.

re: Fun with Floating Point Arithmetic, Part Four

Eric Lippert — Tue, 18 Jan 2005 21:30:00 GMT

Yes, the "decimal" type in .NET is a base-ten floating point number. It's a 96 bit integer scaled by 10^exp, where exp is any value from 0 to -28.

re: Fun with Floating Point Arithmetic, Part Four

James Mastros (theorbtwo) — Wed, 19 Jan 2005 00:29:00 GMT

As a matter of fact, when dealing with money, it's often an error to carry any more information that the smallest unit in the currency you're dealing with (for the USD, EUR, and GBP, that's 1/100, but it is 1/1000, or 1/1 in some currencies). For example, even if the formulae for a morgtage say that the lendee should pay $1.001 per month, that isn't possible. Instead, they would pay exactly $1.00. That sort of error, when compounded over 40 years, can become noticible.

I once, a long time ago, wrote a customer tracking system that did 6% tax calculations, and automatically assessed late fees. It was not until it was in production and started assessing late fees to people who showed up as owing "$0.00" and a significant amount of head-scratching did we figure out that some people ended up owing fractional cents. (Of course, if I were writing the system today, I would have done more tests before putting it into production.)

re: Fun with Floating Point Arithmetic, Part Four

Norman Diamond — Wed, 19 Jan 2005 02:12:00 GMT

> The Visual Basic designers decided that loss
> of information is worse than manufacturing
> new information and applied that rule
> consistently to the variant arithmetic logic.

Except when printing. See your previous blog entry ^_^

1/18/2005 4:29 PM James Mastros (theorbtwo)

> when dealing with money, it's often an error
> to carry any more information that the
> smallest unit in the currency

Surely not. If interest is compounded daily and every 6 months you're supposed to pay the amount of interest that has accrued during those 6 months, you shouldn't compute each day's interest as zero and add up all those zeroes. You need to compute with enough extra digits to make the final result accurate within one of those smallest units, and then round once. (And of course you probably shouldn't be using floating point for any part of this.)

re: Fun with Floating Point Arithmetic, Part Four

Joe — Tue, 14 Mar 2006 20:51:58 GMT

public static double roundnum(double num, int place)
{
double n;
n = num * Math.Pow(10, place);
n = Math.Sign(n) * Math.Abs(Math.Floor(n + .5));
return n / Math.Pow(10, place);
}

didn't write this....but looking for comments on it

re: Fun with Floating Point Arithmetic, Part Four

Sandeep — Thu, 26 Apr 2007 17:24:33 GMT

This is for Dave...

A BIG THANKS!!

i just multiplied by 100 ( i had only 2 decimal places) and it solved my issue.

my code was something like this:

If pctTotal > 100 then

...

Response.Write("Percentage total cannot be greater than 100. Your percentage total is " & pctTotal )

...

End If

I was stumped when I got the message like this:

Percentage total cannot be greater than 100. Your percentage total is 100

i just changed into If CInt(pctTotal*100))>10000

Now its all fixed. Thanks guys!