Fabulous Adventures In Coding : Floating Point Arithmetic

As Timeless As Infinity

Eric Lippert — Thu, 15 Oct 2009 13:25:00 GMT

User: Recently I found out about a peculiar behaviour concerning division by zero in floating point numbers in C#. It does not throw an exception, as with integer division, but rather returns an "infinity". Why is that?

Eric: As I've often said, "why" questions are difficult for me to answer. My first attempt at an answer to a "why" question is usually "because that's what the specification says to do"; this time is no different. The C# specification says to do that in section 4.1.6. But we're only doing that because that's what the IEEE standard for floating point arithmetic says to do. We wish to be compliant with the established industry standard. See IEEE standard 754-1985 for details. Most floating point arithmetic is done in hardware these days, and most hardware is compliant with this specification.

User: It seems to me that division by zero is a bug no matter how you look at it!

Eric: Well, since clearly that is not how the members of the IEEE standardization committee looked at it in 1985, your statement that it must be a bug "no matter how you look at it" must be incorrect. Some industry experts do not look at it that way.

User: Good point. What motivated this design decision?

Eric: I wasn't there; I was busy playing Jumpman on my Commodore 64 at the time. But my educated guess is that it is desirable for all possible operations on all floats to produce a well-defined float result. Mathematicians would call this a "closure" property; that is, the set of floating point numbers is "closed" over all operations.

Positive infinity seems like a reasonable choice for dividing a positive number by zero. It seems plausible because of course the limit of 1 / x as x goes to zero (from above) is "positive infinity", so why shouldn't 1/0 be the number "positive infinity"?

Now, speaking as a mathematician, I find that argument specious. A thing and its limit need not have any particular property in common; it is fallacious to reason that just because, say, a sequence has a particular limit that a fact about the limit is also a fact about the sequence. Mathematically, "positive infinity" (in the sense of a limit of a real-valued function; let's leave transfinite ordinals, hyperbolic geometry, and all of that other stuff out of this discussion) is not a number at all and should not be treated as one; rather, it's a terse way of saying "the limit does not exist because the sequence diverges upwards".

When we divide by zero, essentially what we are saying is "solve the equation x * 0 = 1"; the solution to that equation is not "positive infinity", it is "I cannot because there is no solution to that equation". It's just the same as asking to solve the equation "x + 1 = x" -- saying "x is positive infinity" is not a solution; there is no solution.

But speaking as a practical engineer who uses floating point numbers to do an imprecise approximation of ideal arithmetic, this seems like a perfectly reasonable choice.

User: But surely it is impossible for the hardware to represent "infinity".

Eric: It certainly is possible. You've got 32 bits in a single-precision float; that's over four billion possible floats. All bit patterns of the form

?11111111???????????????????????

are reserved for "not-a-number" values. That's over sixteen million possible NaN combinations. Two of those sixteen million NaN bit patterns are reserved to mean positive and negative infinity. Positive infinity is the bit pattern 01111111100000000000000000000000 and negative infinity is 11111111100000000000000000000000.

User: Do all languages and applications use this convention of division-by-zero-becomes-infinity?

Eric: No. For example, C# and JScript do but VBScript does not. VBScript gives an error if you do that.

User: Then how do language implementors get the desired behaviour for each language if these semantics are implemented by the hardware?

Eric: There are two basic techniques. First, many chips which implement this standard allow the programmer to make float division by zero an exception rather than an infinity. On the 80x87 chip, for example, you can use bit two of the precision control register to determine whether division by zero returns an infinity or throws a hardware exception.

Second, if you don't want it to be a hardware exception but do want it to be a software exception, then you can check bit two of the status register after each division; it records whether there was a recent divide-by-zero event.

The latter strategy is used by VBScript; after we perform a division operation we check to see whether the status register recorded a divide-by-zero operation; if it did, then the VBScript runtime creates a divide-by-zero error and the usual VBScript error management process takes over, same as any other error.

Similar bits exist for other operations that seem like they might be better treated as exceptions, like numeric overflow.

The existence of the "hardware exception" bits creates problems for the modern language implementor, because we are now often in a world where code written in multiple languages from multiple vendors is running in the same process. Control bits on hardware are the ultimate "global state", and we all know how irksome it is to have global, public state that random code can stomp on.

For example: I might be misremembering some details, but I seem to recall that Delphi-authored controls set the "overflows cause exceptions" bit. That is, the Delphi implementors did not use the VBScript strategy of "try it, allow it to succeed, and check to see whether the overflow bit was set in the status register". Rather, they used the "make the hardware throw an exception and then catch the exception" strategy. This is deeply unfortunate. When a VBScript script calls a Delphi-authored control, the control flips the bit to force exceptions but it never "unflips" it. If, later on in the script, the VBScript program does an overflow, then we get an unhandled hardware exception because the bit is still set, even though the Delphi control might be long gone! I fixed that by saving away the state of the control register before calling into a component and restoring it when control returns. That's not ideal, but there's not much else we can do.

User: Very enlightening! I will be sure to pass this information along to my coworkers. I would be delighted to see a blog post on this.

Eric: And here you go!

Every Number Is Special In Its Own Special Way

Eric Lippert — Tue, 28 Nov 2006 21:21:00 GMT

I got a question recently about where in the .NET framework the "special numbers" were defined. The questioner was actually asking about the Double.NaN, Double.PositiveInfinity, etc, special values for floating point numbers. Of course there are other "special numbers" defined by the framework, such as Math.PI.

The question was easily answered but it got me thinking, which, as we know, is usually trouble.

Clearly zero is a very special number, being the first natural number.

One is pretty special too, being the multiplicative identity.

Two is the only even prime.

Three is the lowest odd prime...

Clearly lots of numbers are special. This led me to propose the following theorem:

Theorem: Every natural number (0, 1, 2, ...) is a special number.

Proof:

Let’s posit that the set of nonspecial natural numbers is nonempty, and deduce a contradiction.

If there exists a nonspecial natural number then there must be a lowest nonspecial natural number.

What an unusual property! The lowest nonspecial natural number!

Whatever number has that unusual property must be kinda... special.

Therefore the lowest nonspecial natural number is special.

Therefore, if the set of nonspecial natural numbers is nonempty then it contains a special number.

That is clearly nonsensical, therefore the set of nonspecial natural numbers is empty.

Therefore all natural numbers are special, QED.

And yet I can't find anything special about 7920687935872092847630945767548023. But it must be special somehow!

Extending the proof show that all real numbers are special is left as an exercise for the reader.

Fun With Floating Point Arithmetic, Part Six

Eric Lippert — Wed, 26 Jan 2005 21:02:00 GMT

One more thing -- I said earlier that the VBScript float-to-string algorithm was a little bit different than the JScript algorithm. We can demonstrate quite easily by comparing the outputs of two nigh-identical programs:

' VBScript
print 9.2 * 100.0 < 920.0
print 919.9999999999999 < 920.0
print 920.0000000000001 > 920.0

' JScript
print(9.2*100.00 < 920.0);
print(919.9999999999999 < 920.0);
print (920.0000000000001 > 920.0);

As you'd expect, the last two comparisons of each result in true. But why does the first also result in true? Because of the very issues we've been talking about in the last five parts -- 9.2 cannot be exactly represented as a float. There is some representation error. When the float is multiplied by 100, the representation error also gets 100 times larger, and that's big enough to make it slightly smaller than 920.

If that's the case then why do these programs produce different output?

print 9.2 * 100.0
print 919.9999999999999
print 920.0000000000001

print(9.2*100.00);
print(919.9999999999999);
print(920.0000000000001);

The VBScript program produces 920, 920, 920. The JScript program produces 919.9999999999999, 919.9999999999999, 920.0000000000001. What is up with that?

The JScript algorithm for converting floats to strings is designed to have as much precision as possible. Since 919.9999999999999 and 920.0 have different binary representations as floats, they have different string representation.

The VBScript algorithm on the other hand assumes that if you have 919.9999999999999 or 920.0000000000001, that probably what has happened is you've run into a floating point error accrual issue, and it rounds it back to the correct value for you when it displays the string.

This heuristic means that VBScript (paradoxically) loses a small amount of precision and yet displays more accurate results for typical cases. The down side is that VBScript is unable to display full precision when you really DO want to represent 919.9999999999999. Such cases are quite rare though, and the error created in such cases is tiny.

Fun with Floating Point Arithmetic, Part Five

Eric Lippert — Thu, 20 Jan 2005 18:56:00 GMT

I went to Joel Spolsky's geek dinner at Crossroads the other night, which was a lot of fun. I didn't get much of a chance to chat with Joel, as he was surrounded by a cadre of adoring fans three deep the whole time. I mostly hung out with KC and Larry and some other attendees and had an interesting talk about digital rights management, corporate blogging, and the difficulties of finding good quality Jackie Chan movies in the original Chinese.

I ran into Wesner Moise, who I first met long ago when he was working on Access or Excel or one of those Office kind of products but haven't really run into since. He was rather surprised that I remembered his name, but hey, how many Wesner Moises do you think I meet?

Anyway, coincidentally, Wesner has also been running a series on the perils of floating point and integer mathematics on his blog recently. Check it out!

*********************************

I said a while back that floating point math is nothing like the math we're used to. Consider some of the properties that define real number addition. For instance:

closure: x + y is a number
commutative: x + y = y + x
unique zero: a + b = a if and only if b = 0
associative: (x + y ) + z = x + (y + z)

and so on, and similarly for multiplication. Commutivity still holds, but many of these rules do not work in floating point arithmetic. I'll dig into a few of them here -- for more math rules that don't work in floating point, see Wesner's blog articles on the subject. He lists dozens of them!

Consider the closure property, for example. It's not true in VBScript:

print 10^308 + 10^308 ' Overflow error

It is true in JScript, if you consider Infinity to be a number.

The commutative property is true in both VBScript and JScript, but the unique zero, and hence the associative property, are true in neither: print 10^20 = 10^20 + 5000 prints True -- So clearly, zero is not a unique number which, when added, results in the same value. (Zero is unique in that it is the only number when added to EVERY number, results in no change. But almost every number has multiple values which result in no change when added.)

That means that the associative property goes out the window: print 10^20 + (5000 + 5000) = (10^20 + 5000) + 5000 prints False

The fact that the order in which you make the additions can affect the result makes a difference if you are designing algorithms that must add up lots of little things to one big thing. In those cases, you should try to add together all the little things first, and then add the total to the big thing. That way, the small additions are done with the most precision possible.

There's a related error due to rounding off. 1/100 cannot be represented perfectly accurately in binary any more than 1/3 can be represented perfectly in decimal:

var steps = 100;
var start = 10;
var stop = 11;
var current = start;
do
{
print(current);
current = current + (stop-start)/steps;
} while (current < stop)

Which ends up with

...
10.96999999999998
10.979999999999979
10.989999999999979
10.999999999999978

This actually takes 101 steps, because the last one through the loop gets to 10.999999999999978, which is less than 11.0. This tiny accumulated error results in the algorithm running for 1% too many steps. A better algorithm is to do the looping in integers and compute the current value anew every time:

var steps = 100;
var start = 10;
var current;
for (var step = 0; step < steps ; ++step)
{
current = start + step/steps;
print(current);
}

That actually runs the right number of steps.

A corollary of this is that you should almost never compare two floating point numbers for equality, because you never know when some rounding error might have crept in. Rather, subtract them and look at the absolute difference. For instance, if you know that x and y are positive floats that are likely to be close to each other, don't say if (x==y), say if (Math.abs(x-y) < 0.0001) or if (Math.abs(x-y) < 0.0000001 * x) or whatever makes sense in your application. (If you need to deal with NaNs and infinities, simple subtraction is anything but!)

We saw above that addition of a small number to a large number can lead to an erroneous result, but the error was extremely small. An error of 5000 in a number as big as 10²⁰ is 0.00000000000002%, which ain't bad. We'd expect that subtraction of two numbers very close to each other would also produce an erroneous result, but with a similar error percentage.

We'd be wrong. Here's an example:

Suppose you have to solve an arbitrary quadratic equation:

A x² + B x + C = 0

The solutions are well known, so we can write a little subroutine:

Sub SolveQuadratic(A, B, C)
Discriminant = B*B-4*A*C
If Discriminant < 0 Then
    Print "No real solutions"
Else
    Print (-B + Sqr(Discriminant)) / (2 * A)
    Print (-B - Sqr(Discriminant)) / (2 * A)
End If
End Sub
SolveQuadratic 2, 5, -12

And sure enough, it prints out -4 and 1.5, done. What about

SolveQuadratic 1, -10000000.0000001, 1

The correct solutions are 10000000 and 0.0000001, but this prints out 10000000 and 9.96515154838562E-08, yielding an error of nearly 3.5%! We've got about a hundred trillion times as much error here as we did in the addition. Why?

Because B and the root of the discriminant are very, very close to each other. You only get 15 decimal places of accuracy, and we've used them all up. Therefore, the difference is going to be very inaccurate. Their sum, however, is going to be quite accurate, since they're of similar size and precision.

Fortunately, in this example there's a trick. The product of the two solutions is always C / A, so we can use this fact to write a better algorithm:

Sub SolveQuadratic(A, B, C)
Discriminant = B*B-4*A*C
If Discriminant < 0 Then
Print "No real solutions"
Exit Sub
End If
Soln1 = (-B - Sqr(Discriminant)) / (2 * A)
Soln2 = (-B + Sqr(Discriminant)) / (2 * A)
If Abs(Soln1) < Abs(Soln2) Then Soln1 = Soln2
Soln2 = C / (A * Soln1)
Print Soln1
Print Soln2
End Sub

Which produces a much more accurate result.

But wait a minute -- addition is order-dependent, as we've seen. so are multiplication and division. Should that be ( C / A ) / Soln1 or C / (A * Soln1) ? Or does it even matter? Figuring that out is left as an exercise for the reader!

Fun with Floating Point Arithmetic, Part Four

Eric Lippert — Tue, 18 Jan 2005 19:07:00 GMT

A reader also asked the other day why it is that in VBScript, CSng(0.1) = CDbl(0.1) is False.

Forget about binary floating point for a moment. Suppose that we had two fixed-point decimal systems, say one with five digits after the decimal place and one with ten. You want to represent one-third. In our first system, the closest we can get is 0.33333. In our second system, the closest we can get is 0.3333333333.

Now we compare these two things. But this is comparing apples to oranges -- two things need to be the same type to sensibly compare them. We have a choice -- we can either convert the type with more precision to the less-precise format and then compare, or we can convert the less-precise type to the more precise format and compare.

If we did the former, then we'd truncate the long one and it would compare equal to the short one. If we did the latter, then clearly they would not be equal because we'd be comparing 0.3333300000 with 0.3333333333.

The analogy holds for doubles and singles. In a single, 1/10 in binary is 0.00011001100110011001100. In a double, it's 0.000110011001100110011001100110011001100110011001100110. If we compare by converting the double to a single, then clearly they are equal -- and, in fact a billion or so doubles which are close enough to 1/10 also compare equal. If we compare by converting the single to a double before comparing, clearly they are not equal.

VBScript always converts to the more precise format before doing the comparison.

You might think that this is kind of bogus. Surely if we're comparing a more precise value to a less precise value, it makes sense to say that the less significant bits are, well, less significant and throw them away. By converting the less precise format to a more precise format, we are essentially manufacturing new precision that didn't previously exist. We're just making it up out of whole cloth.

In the world of science, there's a word for that. It's called "fraud".

Yep, we are totally cheating. This is one of those unfortunate "gotchas" which you've got to be very careful of if you're mixing doubles with singles. There is some justification for it though.

Consider addition, for example. If you have a single, say 0.10000000000000000000000, and a double 0.10000000000000000000000111000…, and you add them together, do you expect that the result will be a single or a double? For this situation, many people would say that the sensible thing to do is to treat the single as a double and add them together, rather than losing the information in the less significant bits of the double. Yet this is once more manufacturing new precision for the single.

It comes down to a simple decision. Which is more important: not losing existing information, or not creating new information arbitrarily?

Once you pick which factor is more important, you've got to apply the rules that entails consistently. You can't say that for addition and subtraction, you convert singles to doubles, but do it the other way for comparisons. If you do that then you get into the rather ridiculous situation that two numbers can have a nonzero difference and yet compare as equal!

The Visual Basic designers decided that loss of information is worse than manufacturing new information and applied that rule consistently to the variant arithmetic logic. Hence, the same goes for operations between integer and floating point types; the integer types are converted to floating point types and the operations are done in floats. You'd certainly never say that 100 + 0.25 should avoid manufacturing new precision, convert the double to an integer, and result in 100, I hope. Similarly, comparisons between the integer 100 and the double 100.25 are done by converting the integer to a double, not converting the double to an integer.

In one case, a comparison can be done by converting to neither type. If you're comparing a 32 bit integer to a single-precision float, you can't convert the single to an integer or the integer to a single without one of them being potentially lossy. In that case, both are converted to doubles. In the VBScript implementation we consult this handy table for what conversion is used when comparing currency, 8-byte float, 4-byte float, 4-byte integer and 2-byte integer to each other:

(As an aside, in JScript .NET, where we have 64 bit integers and 64 bit floats which could be compared, we're in this cleft stick again, but this time with no clear way out! There is no larger type to which both can be losslessly converted. Comparing a 64 bit integer to a 64 bit float is a bad idea.)

Unless you have really compelling backwards-compatibility reasons, avoid using single precision floats altogether. In VBScript both a single and a double are stored as a 16 byte VARIANT, so there is no space savings. And on the chip, both single and double precision floats are converted to an internal extended format (which I believe is 80 bits), processed in that format, and then converted back to singles or doubles when the operation is done. There are no significant savings in either time or space obtained by using singles, and you get potentially a lot of pain because things don't compare the way you might think they do. Avoid, avoid, avoid.

Fun With Floating Point Arithmetic, Part Three

Eric Lippert — Mon, 17 Jan 2005 21:54:00 GMT

I've been getting lots of mail, questions and pointers to interesting articles on some of the trials and tribulations of using floating point arithmetic correctly. Please do keep it coming! Though I am certainly no expert in this area, I'm happy to take a crack at any questions you might have.

To sum up the story so far, "normal" floats are all numbers representable by this pattern:

+/- 1.(52 binary digits after the decimal point) x 2^exp, where exp is any integer from -1022 through +1023.

We also have special floats to represent zero, other tiny floats with less than 52 digits of precision, infinities and NaNs, but we'll ignore those for now.

Clearly a float can represent every 53 bit integer with full fidelity. After you get to 54 bit integers though, only every other integer is going to be representable with full fidelity. With 55 bit integers, only every fourth, and so on.

A reader wrote in to ask some questions about how floating point numbers are displayed in decimal. He noticed a weirdness in VBScript, but it's actually easier to show the scenario in JScript. (There are additional factors at play in VBScript which I may get to in another article later.) Consider the following:

var x = 0x8000000000000800;

That would be a 64 bit unsigned integer. It's obviously too large to fit into a 32 bit signed integer, so JScript generates a float and assigns it to the variable slot. However, since this number requires exactly 53 bits, it can be represented with full fidelity as a float. It is not rounded.

In decimal notation, this value should be 9223372036854777856. But if we print out the value of x, we get 9223372036854777000! Why is it rounded off when this particular float has full precision?

Maybe it doesn't have full precision. Maybe I've been lying to you this whole time. Maybe in fact the float is stored in decimal internally, with a 16 slot decimal digit buffer! Fortunately, we can test this hypothesis out.

print(x % 0x800); // 0
print(x % 10); // 6

Whew! The mod operator shows that JScript believes that this number is evenly divisible by 2048, and that the last digit when represented in base 10 is in fact 6.

But that just makes it even more confusing! If JScript knows that the last decimal digit is a six, why does converting the number to a string end in a zero?

Because we do not want to ever make it look like a float has more precision than it actually does. By lopping off the last few decimal digits and replacing them with zeros, we emphasize that floats are accurate only to about fifteen or sixteen significant decimal digits. Imagine the confusion that would result if the situation were reversed:

var x = 9223372036854777000;
print(x); // prints 9223372036854777856

Where did the extra precision come from? To the naïve user who does not realize that numbers are stored in binary internally, this looks really bizarre. They put in something with 16 significant digits and something with 19 comes out! We do this rounding because in the real world, people expect floating point numbers to act like decimal numbers, not binary numbers.

A correct and efficient float-to-string algorithm which shows numbers with a prescribed level of decimal precision is surprisingly difficult to write, particularly if you add the requirement that the string-to-float algorithm have nice "round trip" properties. There are lots of places where things can go slightly wrong.

You've probably noticed already, for instance, that the algorithm which JScript uses does NOT have the property that the decimal integer which comes out is the closest decimal integer to the actual value. Given that we're going to round to a fixed number of decimal significant digits, we would expect that 9223372036854777856 would be rounded to 9223372036854778000, not 9223372036854777000.

In fact, the specification categorically states that the last digit need not be correctly rounded, because doing so is a pain. Section 9.8.1 of ECMA specification 262, Revision 3 reads as follows: (emphasis added)

The operator ToString converts a number m to string format as follows:

If m is NaN, return the string "NaN".

If m is +0 or -0, return the string "0".
If m is less than zero, return the string concatenation of the string "-" and ToString(-m).
If m is infinity, return the string "Infinity".
Otherwise, let n, k, and s be integers such that k ³ 1, 10^k^-¹ £ s < 10^k, the number value for s ´ 10ⁿ^-^k is m, and k is as small as possible. Note that k is the number of digits in the decimal representation of s, that s is not divisible by 10, and that the least significant digit of s is not necessarily uniquely determined by these criteria.
If k £ n £ 21, return the string consisting of the k digits of the decimal representation of s (in order, with no leading zeroes), followed by n-k occurrences of the character ‘0’.
If 0 < n £ 21, return the string consisting of the most significant n digits of the decimal representation of s, followed by a decimal point, followed by the remaining k-n digits of the decimal representation of s.
If -6 < n £ 0, return the string consisting of the character ‘0’, followed by a decimal point, followed by -n occurrences of the character ‘0’, followed by the k digits of the decimal representation of s.
Otherwise, if k = 1, return the string consisting of the single digit of s, followed by lowercase character ‘e’, followed by a plus sign ‘+’ or minus sign ‘-’ according to whether n-1 is positive or negative, followed by the decimal representation of the integer abs(n-1) (with no leading zeros).
Return the string consisting of the most significant digit of the decimal representation of s, followed by a decimal point, followed by the remaining k-1 digits of the decimal representation of s, followed by the lowercase character ‘e’, followed by a plus sign ‘+’ or minus sign ‘-’ according to whether n-1 is positive or negative, followed by the decimal representation of the integer abs(n-1) (with no leading zeros).

NOTE The following observations may be useful as guidelines for implementations, but are not part of the normative requirements of this standard.

If x is any number value other than -0, then ToNumber(ToString(x)) is exactly the same number value as x.
The least significant digit of s is not always uniquely determined by the requirements listed in step 5.

For implementations that provide more accurate conversions than required by the rules above, it is recommended that the following alternative version of step 5 be used as a guideline:

5. Otherwise, let n, k, and s be integers such that k ³ 1, 10^k^-¹ £ s < 10^k, the number value for s ´ 10ⁿ^-^k is m, and k is as small as possible. If there are multiple possibilities for s, choose the value of s for which s ´ 10ⁿ^-^k is closest in value to m. If there are two such possible values of s, choose the one that is even. Note that k is the number of digits in the decimal representation of s and that s is not divisible by 10.

If you are really interested in this subject, read this excellent paper: http://www.ampl.com/REFS/rounding.pdf. This paper was highly influential in the design and implementation of the JScript float-to-string conversion algorithm.

VBScript has some even weirder rules for conversion of floats to strings -- VBScript's FormatNumber method will convert that float to the string 9,223,372,036,854,780,000.00, which has one fewer digits of precision. The reasoning behind that will have to wait for another post!

Floating Point And Benford's Law, Part Two

Eric Lippert — Thu, 13 Jan 2005 17:54:00 GMT

A number of readers asked for an explanation of my offhand comment that Benford's Law can be used to show that binary is the best base for doing floating point math.

One of the desired properties of a floating point system is that the "representation error" is as small as possible. For example, suppose we want to express "one third", in ten-place fixed point decimal notation. The closest we can get is "0.3333333333", which has a representation error of 0.0000000000333333…

A useful way to get a handle on representation error is to look at the granularity of the system. By granularity I mean the smallest difference we can make between two values. For ten-place fixed point decimal notation, the smallest nonzero difference between any two numbers is 10^-10. In a floating point system, the granularity changes as the exponent changes. Really large numbers have large granularity; the difference between two successive floats might be millions or billions. And really small numbers have tiny granularity. The maximum possible representation error is half the granularity. Since there is a clear relationship between granularity and representation error, I’m just going to talk about granularity from now on. Small granularity = small representation error = goodness.

Let's consider two similar systems, one which uses a binary mantissa and one which uses a hexadecimal mantissa. I’m going to continue my convention of showing binary numbers in blue, and we’ll do hex in green.

Suppose we've got on the one hand, 32 bit IEEE floats, aka "singles". That is, we've got one sign bit, nine bits of exponent biased by -255 (so exponents can be from -255 to 256), a 22 bit mantissa, and an implicit leading "1." So a number like (0, 100000011, 1100000000000000000000) would be +1.1100 x 2⁴ = 28

I’m getting sick of spelling out the exponents in non-biased binary. From now on, I’m just going to give them in decimal. And I’m going to give the sign bit by plus and minus, not zero and one.

We now want a hex system with roughly similar range that doesn't take up more storage. Suppose we've got one sign bit, seven bits of exponent biased by -63, and six hex digits for the mantissa. We don't get a leading "1." because not every hex number can be so expressed, so we'll have to use a leading 0. That system has roughly the same range, and if we were to "behind the scenes" express this thing as bits, we'd still be using 32 bits. One for the sign, seven for the exponent, and 24 for the mantissa.

In this system, (+, 2, A00000) would be +0.A00000 x 16² = 160

An immediate disadvantage of this system is that numbers have multiple representations. (+, 2, A00000) and (+, 3, 0A0000) are the same number. Let’s ignore that for now. We’ll ignore all hex mantissas with leading zeros. (And besides, the binary IEEE system wastes lots of cases for denormals and NaNs too, so it's not clear that this is any worse.)

What's the granularity of the hex system when, say, the exponent is 2? Well, consider a number like (+, 2, A00000) – the smallest possible number higher than this is (+, 2, A00001) which is 2^-16 larger. Clearly, 2^-16 is the granularity for all values with an exponent of 2. More generally, if the exponent is N then the granularity is 2^4N-24.

How would we represent (+, 2, A00000) in our binary system? That would be (+, 7, 0100000000000000000000) = +1.01 x 2⁷ = 160. The next largest number that can be represented in our system is (+, 7, 0100000000000000000001) which is 2^-15 larger. So the hex system has smaller granularity and hence smaller representation error, and is therefore the better system, right?

Not so fast. Let’s make a chart.

Decimal       Hex             Binary                  Binary Granularity Exponent

  16    (+, 2, 100000) (+, 4, 0000000000000000000000) -18
  32    (+, 2, 200000) (+, 5, 0000000000000000000000) -17
  48    (+, 2, 300000) (+, 5, 1000000000000000000000) -17
  64    (+, 2, 400000) (+, 6, 0000000000000000000000) -16
  80    (+, 2, 500000) (+, 6, 0100000000000000000000) -16
  96    (+, 2, 600000) (+, 6, 1000000000000000000000) -16
112    (+, 2, 700000) (+, 6, 1100000000000000000000) -16
128    (+, 2, 800000) (+, 7, 0000000000000000000000) -15
144    (+, 2, 900000) (+, 7, 0010000000000000000000) -15
160    (+, 2, A00000) (+, 7, 0100000000000000000000) -15
176    (+, 2, B00000) (+, 7, 0110000000000000000000) -15
192    (+, 2, C00000) (+, 7, 1000000000000000000000) -15
208    (+, 2, D00000) (+, 7, 1010000000000000000000) -15
224    (+, 2, E00000) (+, 7, 1100000000000000000000) -15
240    (+, 2, F00000) (+, 7, 1110000000000000000000) -15

The hex system always has a granularity exponent of -16. In 8/15ths of the cases, the binary system has worse granularity than the hex system. In 4/15ths of the cases, they have the same granularity, and in only 3/15ths of the cases, the binary system has better granularity. Therefore the hex system is clearly the same or better most of the time. We should use this system instead of binary floats.

Not so fast! We’ve forgotten Benford’s Law! That's true if any number is as likely as any other, but that's not realistic.

Suppose the numbers that we are manipulating obey Benford’s Law. In that case, we would expect that fully a quarter of them would begin with 1 when encoded in hex. We’d expect another quarter of them to begin with 2 or 3, another quarter to begin with 4, 5, 6 or 7, and the remaining quarter to begin with 8, 9, A, B, C, D, E and F. If we make that assumption then we must conclude that the binary system is better half the time, equal a quarter of the time, and worse a quarter of the time.

Clearly this isn't the case just for the exponent 2. For any hex exponent N, the hex system will have a granularity exponent of 4N-24. For numbers in the range expressible by the hex system with that exponent, a quarter of the time, the binary system will have a granularity exponent of 4N-26, a quarter of the time it'll be 4N-25, a quarter of the time it'll be 4N-24, and the remaining quarter it'll be 4N-23, so three-quarters of the time it'll be as good or better. On average, the binary system is considerably better if data are distributed according to Benford's Law.

This is just one example, not a proof. But we could generalize this example and show that for any system with a given number of bits, binary mantissas yield smaller representation errors on average than mantissas in any other base.

Benford's Law

Eric Lippert — Wed, 12 Jan 2005 20:18:00 GMT

While I was poking through my old numeric analysis textbooks to refresh my memory for this series on floating point arithmetic, I came across one of my favourite weird facts about math.

A nonzero base-ten integer starts with some digit other than zero. You might naively expect that given a bunch of "random" numbers, you'd see every digit from 1 to 9 about equally often. You'd see as many 2's as 9's. You'd see each digit as the leading digit about 11% of the time. For example, consider a random integer between 100000 and 999999. One ninth begin with 1, one ninth begin with 2, etc.

But in real-life datasets, that's not the case at all. If you just start grabbing thousands or millions of "random" numbers from newspapers and magazines and books, you soon see that about 30% of the numbers begin with 1, and it falls off rapidly from there. About 18% begin with 2, all the way down to less than 5% for 9.

This oddity was discovered by Newcomb in 1881, and then rediscovered by Frank Benford, a physicist, in 1937. As often is the case, the fact became associated with the second discoverer and is now known as Benford's Law.

Benford's Law has lots of practical applications. For instance, people who just make up numbers wholesale on their tax returns tend to pick "average seeming" numbers, and to humans, "average seeming" means "starts with a five". People think, I want something between $1000 and $10000, let's say, $5624. The IRS routinely scans tax returns to find unusually high percentages of leading 5's and examines those more carefully.

Benford's result was carefully studied by many statisticians and other mathematicians, and we now have a multi-base form of the law. Given a bunch of numbers in base B, we'd expect to see leading digit n approximately ln (1 + 1/n) / ln B of the time.

But what could possibly explain Benford's Law?

Multiplication. Most numbers we see every day are not random quantities in of themselves. They're usually computed qualities with some aspect of multiplication to them.

Consider, for example, any property which grows on a percentage basis. Like, say, the Dow Jones Industrial Average. It typically grows a few percent a year. Suppose, just to pick a rate, that on average the DJIA grows at 7% a year. At that rate, it doubles about every ten years. Suppose that the DJIA is 10000. After ten years of having 1 as the leading digit, it finally gets to 20000. Ten years go by again, but in that ten years, it doubles to 40000, not 30000. Therefore, those ten years were spent about half starting with 2, and about half starting with 3. Ten more years go by, and it doubles again to 80000. Now ten years have 4, 5, 6 and 7 as the leading digits in only ten years. Eventually we get up to 100000, and spend another ten years starting with 1. Pick a random date and you'd expect that the DJIA on that day would be twice as likely to start with 1 as 2, and four times as likely to start with 1 as 5.

We can easily write a program that demonstrates Benford's Law. As we multiply more and more numbers together, they tend to clump based on Benford's Law:

var counters = [
[0,0,0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0,0,0] ];

for (var multiplications = 0 ; multiplications <= 3; ++multiplications)
{
for (var trial = 0 ; trial < 10000 ; ++trial)
{
    var num = 1;
    for (var mult = 0 ; mult <= multiplications; ++mult)
      num = num * (Math.floor(Math.random() * 1000) + 1);
    var lead = num.toString().substr(0,1);
    counters[multiplications][lead] ++;
}
}
print(counters.join("\n"));

A typical run produces data from which we can draw this table:

                         Leading Digit
Mults       1    2    3    4   5   6   7   8   9
0       1102 1069 1085 1125 1167 1107 1083 1124 1138
1       2416 1752 1443 1162 1019 756 643 453 356
2       3046 1854 1265 979 778 632 551 468 427
3       3090 1814 1197 924 779 661 582 491 462
predicted 3010 1761 1249 969 792 669 580 511 458

As you can see, with no multiplications, the distribution is a flat 11% for each. But by the time we get up to two or three multiplications, we're almost exactly at the distribution predicted by Benford's Law.

What does this have to do with floating point math? Well, we could conceivably design chips that did decimal or hexadecimal floating-point arithmetic. Would such chips yield more accurate results? Well, recall that last time, we used the fact that you can stuff a leading 1 onto a bit field to define a number. Binary is the only system in which every number except 0 begins with a leading 1! You can make a statistical argument which shows that for bases other than binary, in which you cannot always assume a leading digit, have on average a larger representation error. The argument is somewhat subtle, so I'm not going to actually go through the details of it, but suffice to say that we can show that for typical uses, binary is the least error-producing system we can come up with given that we'll almost always be working with data which follow Benford's Law.

Floating Point Arithmetic, Part One

Eric Lippert — Mon, 10 Jan 2005 19:34:00 GMT

A month ago I was discussing some of the issues in integer arithmetic, and I said that issues in floating point arithmetic were a good subject for another day. Over the weekend I got some questions from a reader about floating point arithmetic, so this seems like as good a time as any to address them.

Before I talk about some of the things that can go terribly wrong with floating point arithmetic, it's helpful (and character building) to understand how exactly a floating point number is represented internally.

To distinguish between decimal and binary numbers, I'm going to do all binary numbers in blue fixed-width.

Here's how floating point numbers work. A float is 64 bits. Of that, one bit represents the sign: 0 is positive, 1 is negative.

Eleven bits represent the exponent. To determine the exponent value, treat the exponent field as an eleven-bit unsigned integer, then subtract 1023. However, note that the exponent fields 00000000000 and 11111111111 have special meaning, which we'll come to later.

The remaining 52 bits represent the mantissa.

To compute the value of a float, here's what you do. You take the mantissa, and you stick a "1." onto its left hand side. Then you compute that value as a 53 bit fraction with 52 fractional places. Then you multiply that by two to the power of the given exponent value, and sign it appropriately.

So for example, the number -5.5 is represented like this: (sign, exponent, mantissa)

(1, 10000000001, 0110000000000000000000000000000000000000000000000000)

The sign is 1, so its a negative number. The exponent is 1025 - 1023 = 2. Put a 1. on the top of the mantissa and you get 1.0110000000000000000000000000000000000000000000000000 = 1.375 and sure enough, -1.375 x 2² = -5.5

This system is nice because it means that every number in the range of a float has a unique representation, and therefore doesn't waste bits on duplicates.

However, you might be wondering how zero is represented, since every bit pattern has 1. plunked onto the beginning. That's where the special values for the exponent come in. If the exponent is 00000000000, then the float is considered a "denormal". It gets 0. plunked onto the beginning, not 1., and the exponent is assumed to be -1022. This has the nice property that if all bits in the float are zero, it's representing zero. Note that this lets you represent smaller numbers than you would be able to otherwise, as we'll see, though you pay the price of lower precision. Essentially, denormals exist so that the chip can do "graceful underflow" -- represent tiny values without having to go straight to zero.

If the exponent 11111111111 and the fraction is all zeros, that's Infinity. If the exponent is 11111111111 and the fraction is not all zeros, that's considered to be Not A Number -- this is a bit pattern reserved for errors.

So the biggest and smallest positive normalized floats are

(0, 11111111110, 1111111111111111111111111111111111111111111111111111)

which is 1.1111111111111111111111111111111111111111111111111111 x 2¹⁰²³, and

(0, 00000000001, 0000000000000000000000000000000000000000000000000000)

which is 1.000 x 2^-1022

The biggest and smallest positive denormalized floats are

(0, 00000000000, 0000000000000000000000000000000000000000000000000001)

which is 0.0000000000000000000000000000000000000000000000000001 x 2^-1022 = 2^-1074, and

(0, 00000000000, 1111111111111111111111111111111111111111111111111111)

which is 0.1111111111111111111111111111111111111111111111111111 x 2^-1022

Next time: floating point math is nothing like real number math.

Bankers' Rounding

Eric Lippert — Fri, 26 Sep 2003 21:03:00 GMT

A number of people have pointed out to me over the years that VBScript's Round function is a bit weird. It seems like it should be pretty straightforward -- you pick the integer closest to the number you've got, end of story. But what about, say, 1.5? There are two closest integers. Do you go up or down?

The Round function goes to the nearest integer, and if there are two nearest integers then it goes to the even one. 1.5 rounds to 2, 0.5 rounds to 0.

Why's that? Why not just arbitrarily say that we always round down in this situation? Why round down sometimes and up some other times? There actually is a good reason!

This algorithm is called the Bankers' Rounding algorithm because, unsurprisingly, it's used by bankers. Suppose a data source provides data which is often in exactly split quantities -- half dollars, half cents, half shares, whatever -- but they wish to provide rounded-off quantities. Suppose further that a data consumer is going to derive summary statistics from the rounded data -- an average, say.

Ideally when you are taking an average you want to take an average of the raw data with as much precision as you can get. But in the real world we often have to take averages of data which has lost some precision. In such a situation the Banker's Rounding algorithm produces better results because it does not bias half-quantities consistently down or consistently up. It assumes that on average, an equal number of half-quantities will be rounded up as down, and the errors will cancel out.

If you don't believe me, try it. Generate a random list of numbers that end in 0.5, round them off, and average them. You'll find that Bankers' Rounding gives you closer results to the real average than "always round down" averaging.

The Round, CInt and CLng functions in VBScript all use the Banker's Rounding algorithm.

There are two other VBScript functions which turn floats into integers. The Int function gives you the first integer less than or equal to its input, and the Fix function gives you the first integer closer to zero or equal to its input. These functions do not round to the nearest integer at all, they simply truncate the fractional part.

UPDATE: What about FormatNumber? See this post.

Why does JScript have rounding errors?

Eric Lippert — Mon, 15 Sep 2003 20:41:00 GMT

Try this in JScript:

window.alert(9.2 * 100.0);

You might expect to get 920, but in fact you get 919.9999999999999. What the heck is going on here?

Boy, have I ever heard this question a lot.

Well, let me answer that question with another question. Suppose you did a simple division, say

window.alert(1.0 / 3.0);

Would you expect an infinitely large window that said "0.33333333333..." with an infinite number of threes, or would you expect ten or so digits? Clearly you'd expect it to not fill your computer's entire memory with threes. But that means that I must ask why are you willing to accept an error of 0.00000000000333333... in the case of dividing one by three but not willing to accept a smaller error of 0.000000000001 in the case of multiplying 9.2 by 100?

The simple fact is that computer arithmetic frequently accumulates tiny errors. Any mathematics which results in numbers that cannot be represented by a small number of powers of two will result in errors.

Let's look at this cases a little closer. We're trying to multiply 9.2 by 100.0. 100.0 can be EXACTLY represented as a floating point number because it's an integer. But 9.2 can't be -- you can't represent 46 / 5 exactly in base two any more than you can represent 1 / 3 exactly in base ten. So when converting from the string "9.2" to the internal binary representation, a tiny error is accrued. However, it's not all bad -- the 64 bit binary number which represents 9.2 internally is (a) the 64 bit float closest to 9.2, and (b) the algorithm which converts back and forth between strings and binary representation will round-trip -- that binary representation will be converted back to 9.2 if you try to convert it to a string.

But now we go and throw a wrench in the works by multiplying by one hundred. That's going to lose the last few bits of precision because we just multiplied the accrued error by a factor of one hundred. The accrued error is now large enough that the string which most exactly represents the computed value is NOT "920" but rather "919.9999999999999".

The ECMAScript specification mandates that JScript display as much precision as possible when displaying floating point numbers as strings.

You may note that VBScript does not do this -- VBScript has heuristics which look for this situation and deliberately break the round-trip property in order to make this look better. In JScript you are guaranteed that when you convert back and forth between string and binary representations you lose no data. In VBScript you sometimes lose data; there are some legal floating point values in VBScript which are impossible to represent in strings accurately. In VBScript it is impossible to represent values like 919.9999999999999 precisely because they are automatically rounded!

Ultimately, the reason that this is an issue is because we as human beings see numbers like "920" as SPECIAL. If you multiply 3.63874692874 by 4.2984769284 and get a result which is one-billionth of one percent off, no one cares, but when you multiply 9.2 by 100.0 and get a result which is one-billionth off, everyone yells (at me!) The computer doesn't know that 9.2 is more special than 3.63874692874 -- it uses the same lossy algorithms for both.

All languages which use floating point arithmetic have this feature -- C++, VBScript, JScript, whatever. If you don't like it, either get in the habit of calling rounding functions, or don't use floating point arithmetic, use only integer arithmetic. (Note that VBScript supports a "currency" type which is fixed-point number not subject to rounding errors. It can only represent four places after the decimal point though.)