Why does JScript have rounding errors?

re: Why does JScript have rounding errors?

Kaspar Karlsson — Wed, 02 May 2007 01:37:28 GMT

Maestro,

I must agree with Jon Kale.

sometimes it's important to have an integer when it should be, and a floating point number when needed, not just because an integer is prettier.

for example, if you want to calculate prime numbers, you'll have to divide the number (x) you want to check by all integers ranging from 2 - ceil(sqrt(x)) (we call that integer y), and see what's left (z) (thus executing z = x % y). if z == (int) 0, then you'll know that x isn't prime, as it can be divided by y w times (w being an integer and positive). however, if z is not (int) 0, you'll have to check further (by increasing y). but z can be (int) 0, plus/minus this very very very small JScript rounding error margin, making it NOT (int) 0. in that case you can't be sure wheter you have a prime number or not...

of course calculating prime numbers in JScript is plain stupid when you can do that with Python several thousands times faster, but this is just an example to show that that very very very small JScript rounding error margin can make a real difference. I can think of other cases.

Adios, K.

Chronological Index

Fabulous Adventures In Coding — Fri, 10 Jun 2005 23:55:20 GMT

2003/09/12 What's Up With Hungarian Notation? 2003/09/12 Eric's Complete Guide To BSTR Semantics 2003/09/12...

re: VB 6

Rob Chartier's Excogitation — Thu, 19 Feb 2004 22:34:00 GMT

RE: Why does JScript have rounding errors?

Eric Lippert — Tue, 23 Sep 2003 18:50:00 GMT

> after all, 920 differs from 920.0: Well, if they differ, what's their difference? Last I checked, 920 - 920.0 = 0, add 920.0 to both sides and you get 920 = 920.0, QED. As a mathematically inclined pedant with a degree in mathematics from the University of Waterloo, lemme tell ya, you're going to have to start from the set-theoretic underpinnings of the real number system if you want to make this argument, and it is not an argument that I myself would care to advance. That 920 in Z and the 920 in R are the same number dude.

RE: Why does JScript have rounding errors?

Jon Kale — Tue, 23 Sep 2003 13:17:00 GMT

'Ultimately, the reason that this is an issue is because we as human beings see numbers like "920" as SPECIAL.' Erm... surely you mean 'numbers like "920.0" as SPECIAL'. After all, 920 differs from 920.0: one's in Z, while the other's in R, and as any fule kno, the numbers in Z have some very interesting properties which don't generalise to R, and vice versa. Now, this is of course a mathematically inclined pedant writing this... but even so it raises an interesting point, which is that people in the main conflate integers and rationals when they can, even when the conversion isn't necessarily valid - so VBS does the "least-suprise" thing whereas JS does the "right" thing.

RE: Why does JScript have rounding errors?

Eric Lippert — Mon, 22 Sep 2003 00:12:00 GMT

Hmm, that works fine for me and I don't recall ever fixing a bug in the mod operator. Can you send me a SMALL program that reproduces the fault along with the version numbers on JScript.DLL and CScript.exe? Thanks, Eric

RE: Why does JScript have rounding errors?

Abhi — Sun, 21 Sep 2003 20:02:00 GMT

Hi, Thanks for the headup :) I used JScript in WSH for doing 108 % 100 and i got the answer as 0. i get correct answers for all the other numbers. I think this is a bug too.