As Timeless As Infinity

As Timeless As Infinity

  • Comments 33

User: Recently I found out about a peculiar behaviour concerning division by zero in floating point numbers in C#. It does not throw an exception, as with integer division, but rather returns an "infinity". Why is that?

Eric: As I've often said, "why" questions are difficult for me to answer. My first attempt at an answer to a "why" question is usually "because that's what the specification says to do"; this time is no different. The C# specification says to do that in section 4.1.6. But we're only doing that because that's what the IEEE standard for floating point arithmetic says to do. We wish to be compliant with the established industry standard. See IEEE standard 754-1985 for details. Most floating point arithmetic is done in hardware these days, and most hardware is compliant with this specification.

User: It seems to me that division by zero is a bug no matter how you look at it!

Eric: Well, since clearly that is not how the members of the IEEE standardization committee looked at it in 1985, your statement that it must be a bug "no matter how you look at it" must be incorrect. Some industry experts do not look at it that way.

User: Good point. What motivated this design decision?

Eric: I wasn't there; I was busy playing Jumpman on my Commodore 64 at the time. But my educated guess is that it is desirable for all possible operations on all floats to produce a well-defined float result. Mathematicians would call this a "closure" property; that is, the set of floating point numbers is "closed" over all operations.

Positive infinity seems like a reasonable choice for dividing a positive number by zero. It seems plausible because of course the limit of 1 / x as x goes to zero (from above) is "positive infinity", so why shouldn't 1/0 be the number "positive infinity"?

Now, speaking as a mathematician, I find that argument specious. A thing and its limit need not have any particular property in common; it is fallacious to reason that just because, say, a sequence has a particular limit that a fact about the limit is also a fact about the sequence. Mathematically, "positive infinity" (in the sense of a limit of a real-valued function; let's leave transfinite ordinals, hyperbolic geometry, and all of that other stuff out of this discussion) is not a number at all and should not be treated as one; rather, it's a terse way of saying "the limit does not exist because the sequence diverges upwards".

When we divide by zero, essentially what we are saying is "solve the equation x * 0 = 1"; the solution to that equation is not "positive infinity", it is "I cannot because there is no solution to that equation". It's just the same as asking to solve the equation "x + 1 = x" -- saying "x is positive infinity" is not a solution; there is no solution.

But speaking as a practical engineer who uses floating point numbers to do an imprecise approximation of ideal arithmetic, this seems like a perfectly reasonable choice.

User: But surely it is impossible for the hardware to represent "infinity".

Eric: It certainly is possible. You've got 32 bits in a single-precision float; that's over four billion possible floats. All bit patterns of the form

?11111111???????????????????????

are reserved for "not-a-number" values. That's over sixteen million possible NaN combinations. Two of those sixteen million NaN bit patterns are reserved to mean positive and negative infinity. Positive infinity is the bit pattern 01111111100000000000000000000000 and negative infinity is 11111111100000000000000000000000.

User: Do all languages and applications use this convention of division-by-zero-becomes-infinity?

Eric: No. For example, C# and JScript do but VBScript does not. VBScript gives an error if you do that.

User: Then how do language implementors get the desired behaviour for each language if these semantics are implemented by the hardware?

Eric: There are two basic techniques. First, many chips which implement this standard allow the programmer to make float division by zero an exception rather than an infinity. On the 80x87 chip, for example, you can use bit two of the precision control register to determine whether division by zero returns an infinity or throws a hardware exception.

Second, if you don't want it to be a hardware exception but do want it to be a software exception, then you can check bit two of the status register after each division; it records whether there was a recent divide-by-zero event.

The latter strategy is used by VBScript; after we perform a division operation we check to see whether the status register recorded a divide-by-zero operation; if it did, then the VBScript runtime creates a divide-by-zero error and the usual VBScript error management process takes over, same as any other error.

Similar bits exist for other operations that seem like they might be better treated as exceptions, like numeric overflow.

The existence of the "hardware exception" bits creates problems for the modern language implementor, because we are now often in a world where code written in multiple languages from multiple vendors is running in the same process. Control bits on hardware are the ultimate "global state", and we all know how irksome it is to have global, public state that random code can stomp on.

For example: I might be misremembering some details, but I seem to recall that Delphi-authored controls set the "overflows cause exceptions" bit. That is, the Delphi implementors did not use the VBScript strategy of "try it, allow it to succeed, and check to see whether the overflow bit was set in the status register". Rather, they used the "make the hardware throw an exception and then catch the exception" strategy. This is deeply unfortunate. When a VBScript script calls a Delphi-authored control, the control flips the bit to force exceptions but it never "unflips" it. If, later on in the script, the VBScript program does an overflow, then we get an unhandled hardware exception because the bit is still set, even though the Delphi control might be long gone! I fixed that by saving away the state of the control register before calling into a component and restoring it when control returns. That's not ideal, but there's not much else we can do.

User: Very enlightening! I will be sure to pass this information along to my coworkers. I would be delighted to see a blog post on this.

Eric: And here you go!

 

  • That's one more reason why I strongly prefer Decimal over Double "by default" (i.e. when there are no other clear reasons to prefer one over another), and recommend the same to those new to C# - because Decimal has no INF or NAN values, and all arithmetic (including division by zero) is always checked.

    (The other reason is that there's no such Decimal value x for which (x+1)==x, while there are plenty such Double values. Regardless of the rationale for such values, people often forget about this little peculiarity of float/double, and it can be extremely confusing for them when they actually run into it.)

  • Say hello to Mr. User from me! As always, he's got some interesting questions...

  • "Positive infinity seems like a reasonable choice for dividing a positive number by zero. It seems plausible because of course the limit of 1 / x as x goes to zero is "positive infinity", so why shouldn't 1/0 be the number "positive infinity"?"

    -----

    While your further statements about limits are all reasonable, this paragraph is simply untrue: the limit of 1/x as x decreases to 0 (a so-called right-hand limit) diverges toward 'positive infinity'; however the limit of 1/x as x increases to 0 (left-hand limit) is *negative* infinity.

    Good point. I'll clarify the text. -- Eric

    Based on this I'd reject 1/0 := positive infinity simply for the inconsistency, all arguments about what it means to have a limit, divergent or not, aside.

    My personal opinion is that it would have been better to have a specific NaN reserved to mean "undefined"; this NaN would logically be none of positive, negative, zero or infinite. But then again, I understand that the infinity value almost always crops up in scientific calculations exactly at the point where something really is diverging to positive infinity, so I see why this was a reasonable, if not entirely formally justifiable, choice. -- Eric

  • Actually INFs are not NaNs if you go by the _isnan function in C++ ( MS VC8 that is ).

  • @Deskin Miller : a good way to illustrate this is the euqation x*y = 1, which is the formula for the standard hyperbola.

    http://demo.activemath.org/ActiveMath2/search/show.cmd?id=mbase://LeAM_calculus/functions/ex1_rat_function

  • I hated that overflow behaviour back in my Delphi days.  It always seemed to take over at the most senseless times, like in the middle of a checksum calculation, even when I had that option turned off in the project settings (obviously because some package in the dependency chain was turning it back on).

    I'm pretty sure I can even remember an instance where I'd explicitly put a block of code after a {$Q-} and it STILL threw a runtime exception because some opaque internal component had turned it back on.  Ridiculous.

    On the subject of division by zero, though, I've been bitten a few times by the infinity result.  The most recent happened when populating a chart (using doubles as point values), and instead of "gracefully" throwing an exception, it happily added the infinite values, causing the chart component to hang and the entire program to become non-responsive.  Took me nearly a day to track that one down.  I know it's all in the IEEE spec, but I do kind of wish that C# had some compiler option or special operator to throw an exception instead.

  • I believe signalling on division by zero is the best approach. Perhaps I am somewhat biased as one of the current maintainers of the Delphi compiler, but if you don't deal with the division by zero when and where it happens, it can bleed through the rest of your calculations, and depending on what calculations you did, it may not be clear any longer where exactly you divided by zero.

    Delphi programmers must similarly go to great lengths to reset the FPU control word, in particular to re-set back Extended (80 bit) precision, after various meddling MSVC RTLs modify it after a LoadLibrary call, or ActiveX component, etc. etc.

    Usually we fix it by saving away the state of the control register before calling into a component and restoring it when control returns. That's not ideal, but there's not much else we can do.

    LOL. Dude, I feel your pain. -- Eric

  • It doesn't even have to be different languages to create problems. If you use Direct3D you need to specify a specific flag to the initialize function in order not to mess with the FPU state (although it's not the exception flags but another FPU flag).

  • I've the same thought as mathematician. Especially in engineering type application, "division by zero" always mean something can be further digested instead of an exception.

    Although we can catch the exception and do something follow-up (better than none), it does not tell what the actual result is (+Inf , -Inf or NaN ?). Fortunately C# applies the IEEE floating point standard or such things can be difficult to implement.

    And this is the engineering thinking: http://www-h.eng.cam.ac.uk/help/tpl/programs/Matlab/NaNInF.html

  • @Pavel Minaev,

    I agree that using Decimal would be a better choice or, in any case, on a more familiar ground, for most people who started at the time of C\C++, Pascal (not yet Delphi), and Assembler, because the "zero-divide error" being a hardware, or at least a very low-level, "gut response", of a computer, is almost as traditional as  the notion of the Earth revolving around the Sun.

    But for the same very people, the types (or, rather, the *words*) "float" and "double" are the first to come to mind when thinking the floating-point arithmetic (isn't the Decimal type new to .NET/C#? I'm not too sure) So my question is: why not use Decimal for all that "new age", 16-million-NaN-values sutff, and let the "good old" float and double behave "as before?"

    But then again: the answer is, as Eric points out, "because the specifications, and the standards, say so," I guess...

  • Speaking from a mathematical perspective, it actually is possible and sometimes useful to extend the real field R to include the limits of sequences like 1,2,3... and -1,-2,-3... This can be done in two different ways — by introducing two "infinities" (+∞ and -∞), giving R the topology of a closed interval, or a single infinity ∞ to which both sequences converge, giving R the topology of a circle (this construction is analogous to the Riemann sphere). In both these constructions, operations like 1/∞ can be defined to have a definite result, although this breaks the algebraic structure of R.

    http://en.wikipedia.org/wiki/Extended_real_number_line

    Yes, I know. That's why I deliberately called out that I was not considering mathematical systems in which infinities are numbers, like Cantor's system of transfinite ordinals, or geometries that have "a point at infinity", and so on. -- Eric

  • "float" and "double" are historically associated specifically with IEEE binary floating-point standard, which is also the one including INF and NaNs. In other words, people who started with C/C++ (and Java) would actually be more likely to expect the behavior of division by zero as described, and not get an exception. So it really isn't quite "new age".

    On the other hand, System.Decimal is new (not the idea, but this particular implementation - so far as I know, it doesn't match VT_DECIMAL, nor any other pre-existing decimal spec), so its semantics aren't guided by any standard.

    This is news to me. In what way does it not exactly match the VT_DECIMAL spec? It had better match that spec exactly, because the compiler does compile-time decimal arithmetic on decimal constants by making VT_DECIMALs and calling the OLE Automation decimal math routines. If you know something that I don't about this, please let me know. -- Eric

    So far as I can tell, it it specifically designed following the "principle of least astonishment" - consider:

    - No magic NaN or INF values.

    - All operations are checked for overflow and throw if any such happens. Division by zero also throws. Underflow, however, is permitted, and the result is 0.

    - "Implicit zeros" are not permitted - that is, you cannot specify the position of floating point such that it goes "beyond the edge" of the sequence of decimal digits defined by significand, even though the exponent field size allows for such values. The consequence of this is "guaranteed sane" behavior for decimal arithmetic, such that it's never the case that (a+1)==a, as I mentioned earlier can happen for float/double.

    - All results are rounded (to allowed number of decimal places) using banker's rounding, minimizing accumulation of rounding errors.

    - Explicit zeros after decimal point are permitted, and are preserved in arithmetic operations - so 1+2=3, but 1.0+2.0=3.0. This can be used to encode precision information, and preserve it when passing numbers around and operating on them.

    Decimal is one little bit of .NET and C# that I really appreciate being there since 1.0, and presented in an accessible way (compare to Java's BigDecimal...), and that I think is very much unappreciated and undervalued by many, and definitely not getting the praise that it deserves.

    My only pity is that floating-point literals are double by default when no suffix is specified, but that is a really minor nit, and understandable for historical reasons (C++ guys probably wouldn't appreciate it if literals quietly changed meaning, or if they had to suffix their doubles with "d" everywhere).

  • @Pavel,

    Ok, I'm convinced now; besides, I personally use Decimal quite often (compared to others whose code I have read so far). One little question that remains is: does the System.Decimal's implementation utilize the 80x87 floating-point co-processor (or whatever part of the modern mlti-core CPUs plays that role?) If it does not, then here you have the reason why the System.Decimal is so badly undervalued and, as you say, does not get the praise it deserves... apart from the legacy code that has been "translated" from C++ or Java and has "float" and "double" all over the place, and no Decimal at all.

    Decimal arithmetic is actually done in integers behind the scenes. -- Eric

  • @Pavel:

    "My only pity is that floating-point literals are double by default when no suffix is specified, but that is a really minor nit, and understandable for historical reasons."

    There is no absolutely no reason to use float in new software (of course compatibility with existing software is always a good reason). double and float are the same in every respect except precision. And you should always want better precision. So it makes perfect sense that by default 3.14 is a double rather than a float.

    There are some very narrow corner cases where the smaller memory footprint of a float, or its very slightly better performance makes it a good choice, but you should leave that choice to experts (which most programmers are not, judging by the huge amount of questions/posts on the web regarding floating point arithmetic; sadly I stumble upon code like "if (denominator == 0.0)" far too often).

    Consider that even WPF, which surely should strive for small memory footprint and good performance, uses double all over the place, rather than float.

  • Just realized you may have wanted decimal literals rather than double by default! While I understand your love for the decimal type, it is a lot slower than double. And by a lot, I mean as much as 100x slower for addition (and "only" 10x for division).

    So since most software works well with double, it makes sense to keep it the default (again consider WPF as an example and imagine how it would perform if it used decimal all over the place).

Page 1 of 3 (33 items) 123