Integer Arithmetic In VBScript, Part One

re: Integer Arithmetic In VBScript, Part One

Norman Diamond — Thu, 02 Dec 2004 01:08:00 GMT

> If you require integer arithmetic beyond 53
> bits [...] then you'll need to use special
> techniques.

Such as, for example, programmatically fire up an instance of the Calculator accessory if your version of Windows is sufficiently recent, push input to it, and copy its output.

> Developing libraries that manipulate
> arbitrarily large numbers efficiently is
> definitely character-building

OK, instead of making a pun about which datatype is which (character-building vs. integer-building) (oops), I'll byte. Microsoft did develop and deliver bignum arithmetic in the Calculator accessory, but not in a library for developers. Did Microsoft really prioritize building character in non-Microsoft developers rather than in itself, or was this just a coincidence?

re: Integer Arithmetic In VBScript, Part One

Eric Lippert — Thu, 02 Dec 2004 01:28:00 GMT

Funny, how fifty thousand people work here, and yet everyone externally seems to assume that everyone here knows what everyone else here was thinking at all times.

Believe it or not, I haven't the faintest idea what the design decisions that went into calc.exe were. I barely remember the design decisions that I made for my own product!

re: Integer Arithmetic In VBScript, Part One

BonzoESC — Thu, 02 Dec 2004 05:36:00 GMT

"Funny, how fifty thousand people work here, and yet everyone externally seems to assume that everyone here knows what everyone else here was thinking at all times. "

B-B-But the nerds on slashdot kept calling you guys the Borg! What happened to solidarity? And unity? United we stand, divided we fall!

The #1 reason to comment your code is not for your future replacement, it's for your future self.

re: Integer Arithmetic In VBScript, Part One

Tonetheman — Thu, 02 Dec 2004 16:50:00 GMT

Even though this is going on to a tangent... the problem that you speak of about lumping everyone at MS in solid brain clump is really just now coming to an end. Until all of the blogging started... effectively MS was one big brain clump, unless of course you knew someone or happened to know a newsgroup where one of you guys were posting.

I still find myself lumping everyone at MS into a single evil clump. Though more and more I find that it is not true. And the blogs are changing that fact everyday. Whoever started the blog push at MS did a great job towards making MS more approachable. And hopefully has been rewarded.

re: Integer Arithmetic In VBScript, Part One

Eric Lippert — Thu, 02 Dec 2004 18:44:00 GMT

It's an irony -- Microsoft is widely perceived as being a monolithic entity externally, and yet internally the #1 complaint on the company poll, year after year is "it is too hard to communicate effectively with other teams".

Microsoft is anything but a monolithic entity. Rather, it's a loose-knit collection of disparate fiefdoms. Some cooperate, some compete, it's a big old mess that takes someone as smart as Bill Gates to mold into some kind of vaguely coherent whole.

MS Blogs

BonzoESC — Thu, 02 Dec 2004 20:58:00 GMT

"I still find myself lumping everyone at MS into a single evil clump. Though more and more I find that it is not true. And the blogs are changing that fact everyday. Whoever started the blog push at MS did a great job towards making MS more approachable. And hopefully has been rewarded."

About half of my daily blog reads are MS - This one, Raymond Chen, and Major Nelson. It's certainly teaching /me/ better habits.

re: Integer Arithmetic In VBScript, Part One

Mat Hall — Fri, 03 Dec 2004 08:48:00 GMT

"Developing libraries that manipulate arbitrarily large numbers efficiently is definitely character-building..."

And then some! Back in the dim and distant past I wrote a bunch of code in FORTRAN for just this purpose, and it was an interesting experience. (For reasons unknown, the version of FORTRAN we used had the world's least precise floating point math routines, and anything longer than about 5 significant digits was useless. For a supposedly math/engineering-centric language this was unhelpful...)

re: Integer Arithmetic In VBScript, Part One

Eric Brown — Fri, 03 Dec 2004 18:50:00 GMT

Mat: If you need more than 5 significant digits in your engineering usage, you're not going to be able to build whatever it is you're simulating anyway.

Consider that resistors are typically +- 20%, unless you spend big bux and get the +- 1% (but then you need to worry about the inductance - precision resistors are often wirewound). Capacitors are even worse.

Civil engineering is way worse than that; typically, you'll be doing good to get one decimal point of accuracy.

Of course, that's why civil engineers stick in about a 5x safety margin.

re: Integer Arithmetic In VBScript, Part One

JP — Fri, 03 Dec 2004 18:55:00 GMT

Why the check for 32 bits and then for 16? Isn't (&h10000 - 1) = &hffff? I guess not since 65535 is not -1. Inherited problems from 16 bit world?

re: Integer Arithmetic In VBScript, Part One

Eric Lippert — Fri, 03 Dec 2004 18:56:00 GMT

That's a somewhat specious analysis. Engineering is about more than just multiplying mass times 9.8 to determine force.

There are all kinds of engineering applications of mathematics which require high precision during the operation in order to get an answer that is within a couple significant digits of reality.

Numeric solutions to differential equations, for example, are notorious for accumulating error terms. The higher precision you can get the model, the less error is going to be introduced from the calculation.

re: Integer Arithmetic In VBScript, Part One

Eric Lippert — Fri, 03 Dec 2004 18:57:00 GMT

The interactions between 16 and 32 bit numbers can themselves be a little tricky; I should talk about those in another post.

re: Integer Arithmetic In VBScript, Part One

Norman Diamond — Mon, 06 Dec 2004 01:10:00 GMT

Also even though you can't build a machine to such tight tolerances, you can measure and make corrections until you can compute the necessary compensations (adjustments) to make in subsequent operations. When your spacecraft takes off its speed is going to be more than 1% different from the ideal, but when you instruct it to use precious fuel for a midcourse correction you want your computations to be a lot closer than 1%.

re: Integer Arithmetic In VBScript, Part One

Jim Howell — Thu, 17 Jan 2008 23:00:21 GMT

Eric--

Thanks for posting the workaround. We need to incorporate the ability to do these hex to dec and dec to hex conversions on very large numbers in order to convert MEID numbers (the new serial numbers issued by the FCC for cellular phones) in our database applications. Just one real world example of people playing around with great big numbers!

Thanks --Jim

re: Integer Arithmetic In VBScript, Part One

Jay Ohman — Thu, 16 Jul 2009 15:58:12 GMT

Old blog, but still quite helpful. Creating some IP range comparison math, it made sense (to me) to convert the dotted-octet address to decimal, then compare. I ran into the "VBScript runtime error: Overflow" error for addresses beyond 127.255.255.255 (2^31-1) using Mod. Since I needed Mod for addresses up to 255.255.255.255 (2^32-1), I was thwarted by 1 bit! Thankfully, the workaround was posted.