Integer Arithmetic In VBScript, Part One

Integer Arithmetic In VBScript, Part One

  • Comments 14

I've received some questions recently on how integer arithmetic works in VBScript, so I thought I might spend a few entries talking about some low-level bit twiddling topics. Here's one of the mails I got this morning:

There seems to be a limitation on the largest number that the VBScript "mod" operator works against. I was wondering if you could explain this limitation and how to overcome it. I have tried type casting using CDbl or CCur, but that does not seem to overcome the problem.

The mod operator, just to refresh your memory, returns the remainder of an integer division. So, 25 mod 7 is 4, because 7 goes into 25 three times leaving a remainder of 4.

And indeed, the writer is correct. 2147483647 mod 2 returns 1, but 2147483648 mod 2 produces a type mismatch error. Since that's 2^31, you should be able to guess at what's going on here -- the mod operator only operates on values that can be fit into a 32 bit signed integer.

Let me describe precisely what we do. The mod operator takes two arguments which of course can both be of any variant type. The type conversions work like this:

  • If either argument is a byref variant then we dereference to get the underlying variant and replace the argument with the result.
  • If either argument is an object with a default property then we fetch the object's default property (if one exists) and replace the argument with the result.
  • If either argument is anything other than Empty, Null, a 16/32 bit signed integer, a 32/64 bit float, a currency, a date, a string, a Boolean, or an unsigned byte then we raise an error.
  • If either argument is Null then we return Null.
  • Otherwise, if either argument is a 32 bit signed integer, a 32/64 bit float, a currency, a date or a string then both arguments are converted to 32 bit signed integers. This may throw a type mismatch error if the conversion cannot be performed. We compute the 32 bit signed integer which is the modulus, and return.
  • Otherwise, if either argument is Empty, 16 bit signed integer, or Boolean then we convert both arguments to 16 bit signed integers, compute the 16 bit signed integer which is the modulus, and return.
  • Otherwise, the only scenario left is the rare but possible case that both arguments are unsigned bytes. We compute the modulus and return an unsigned byte.

(Incidentally, the "integer division" operator in VBScript behaves pretty much exactly the same, as you'd expect given the obvious relationship between the two operators.)

VBScript provides modular arithmetic on signed 32/16 bit integers and unsigned bytes, and that's it. If you want to implement a modulus function that operates on currencies or floats or some other data type, you'll have to do the work yourself. How hard it is to write such a beast depends upon the desired range. We give you an operator that works over the range of a 32 bit signed integer, which seems like plenty to me. If you want something that works over the range of, say, a 53 bit signed integer, then this works:

Function MyMod(ByVal a, ByVal b)
    a = Fix(CDbl(a))
    b = Fix(CDbl(b))
    MyMod = a - Fix(a/b) * b
End Function

This works just fine on 2147483648. In fact, it works well right up to 9007199254740990. But try x = MyMod(9007199254740991, 2) and you'll get zero, even though this is obviously an odd number. 64 bit floats stop having integer-level accuracy when they exceed 2^53, so you'll get crazy results if you try. (Currencies are only integer-accurate to about 50 bits, so they just make the situation worse.)

If you require integer arithmetic beyond 53 bits -- like, say, you're writing your own RSA implementation for some crazy reason, and need to do modular arithmetic on 1000+ bit integers -- then you'll need to use special techniques. Developing libraries that manipulate arbitrarily large numbers efficiently is definitely character-building, but I'd recommend that you do it in C or C# or some other hard-typed language designed for bit twiddling, not in VBScript.

  • > If you require integer arithmetic beyond 53
    > bits [...] then you'll need to use special
    > techniques.

    Such as, for example, programmatically fire up an instance of the Calculator accessory if your version of Windows is sufficiently recent, push input to it, and copy its output.

    > Developing libraries that manipulate
    > arbitrarily large numbers efficiently is
    > definitely character-building

    OK, instead of making a pun about which datatype is which (character-building vs. integer-building) (oops), I'll byte. Microsoft did develop and deliver bignum arithmetic in the Calculator accessory, but not in a library for developers. Did Microsoft really prioritize building character in non-Microsoft developers rather than in itself, or was this just a coincidence?
  • Funny, how fifty thousand people work here, and yet everyone externally seems to assume that everyone here knows what everyone else here was thinking at all times.

    Believe it or not, I haven't the faintest idea what the design decisions that went into calc.exe were. I barely remember the design decisions that I made for my own product!
  • "Funny, how fifty thousand people work here, and yet everyone externally seems to assume that everyone here knows what everyone else here was thinking at all times. "

    B-B-But the nerds on slashdot kept calling you guys the Borg! What happened to solidarity? And unity? United we stand, divided we fall!

    The #1 reason to comment your code is not for your future replacement, it's for your future self.
  • Even though this is going on to a tangent... the problem that you speak of about lumping everyone at MS in solid brain clump is really just now coming to an end. Until all of the blogging started... effectively MS was one big brain clump, unless of course you knew someone or happened to know a newsgroup where one of you guys were posting.

    I still find myself lumping everyone at MS into a single evil clump. Though more and more I find that it is not true. And the blogs are changing that fact everyday. Whoever started the blog push at MS did a great job towards making MS more approachable. And hopefully has been rewarded.
  • It's an irony -- Microsoft is widely perceived as being a monolithic entity externally, and yet internally the #1 complaint on the company poll, year after year is "it is too hard to communicate effectively with other teams".

    Microsoft is anything but a monolithic entity. Rather, it's a loose-knit collection of disparate fiefdoms. Some cooperate, some compete, it's a big old mess that takes someone as smart as Bill Gates to mold into some kind of vaguely coherent whole.
  • "I still find myself lumping everyone at MS into a single evil clump. Though more and more I find that it is not true. And the blogs are changing that fact everyday. Whoever started the blog push at MS did a great job towards making MS more approachable. And hopefully has been rewarded."

    About half of my daily blog reads are MS - This one, Raymond Chen, and Major Nelson. It's certainly teaching /me/ better habits.
  • "Developing libraries that manipulate arbitrarily large numbers efficiently is definitely character-building..."

    And then some! Back in the dim and distant past I wrote a bunch of code in FORTRAN for just this purpose, and it was an interesting experience. (For reasons unknown, the version of FORTRAN we used had the world's least precise floating point math routines, and anything longer than about 5 significant digits was useless. For a supposedly math/engineering-centric language this was unhelpful...)
  • Mat: If you need more than 5 significant digits in your engineering usage, you're not going to be able to build whatever it is you're simulating anyway.

    Consider that resistors are typically +- 20%, unless you spend big bux and get the +- 1% (but then you need to worry about the inductance - precision resistors are often wirewound). Capacitors are even worse.

    Civil engineering is way worse than that; typically, you'll be doing good to get one decimal point of accuracy.

    Of course, that's why civil engineers stick in about a 5x safety margin.
  • Why the check for 32 bits and then for 16? Isn't (&h10000 - 1) = &hffff? I guess not since 65535 is not -1. Inherited problems from 16 bit world?
  • That's a somewhat specious analysis. Engineering is about more than just multiplying mass times 9.8 to determine force.

    There are all kinds of engineering applications of mathematics which require high precision during the operation in order to get an answer that is within a couple significant digits of reality.

    Numeric solutions to differential equations, for example, are notorious for accumulating error terms. The higher precision you can get the model, the less error is going to be introduced from the calculation.
  • The interactions between 16 and 32 bit numbers can themselves be a little tricky; I should talk about those in another post.
  • Also even though you can't build a machine to such tight tolerances, you can measure and make corrections until you can compute the necessary compensations (adjustments) to make in subsequent operations. When your spacecraft takes off its speed is going to be more than 1% different from the ideal, but when you instruct it to use precious fuel for a midcourse correction you want your computations to be a lot closer than 1%.
  • Eric--

    Thanks for posting the workaround.  We need to incorporate the ability to do these hex to dec and dec to hex conversions on very large numbers in order to convert MEID numbers (the new serial numbers issued by the FCC for cellular phones) in our database applications.  Just one real world example of people playing around with great big numbers!  

    Thanks --Jim

  • Old blog, but still quite helpful.  Creating some IP range comparison math, it made sense (to me) to convert the dotted-octet address to decimal, then compare.  I ran into the "VBScript runtime error: Overflow" error for addresses beyond 127.255.255.255 (2^31-1) using Mod.  Since I needed Mod for addresses up to 255.255.255.255 (2^32-1), I was thwarted by 1 bit!  Thankfully, the workaround was posted.

Page 1 of 1 (14 items)