Fabulous Adventures In Coding : VBScript

As Timeless As Infinity

Eric Lippert — Thu, 15 Oct 2009 13:25:00 GMT

User: Recently I found out about a peculiar behaviour concerning division by zero in floating point numbers in C#. It does not throw an exception, as with integer division, but rather returns an "infinity". Why is that?

Eric: As I've often said, "why" questions are difficult for me to answer. My first attempt at an answer to a "why" question is usually "because that's what the specification says to do"; this time is no different. The C# specification says to do that in section 4.1.6. But we're only doing that because that's what the IEEE standard for floating point arithmetic says to do. We wish to be compliant with the established industry standard. See IEEE standard 754-1985 for details. Most floating point arithmetic is done in hardware these days, and most hardware is compliant with this specification.

User: It seems to me that division by zero is a bug no matter how you look at it!

Eric: Well, since clearly that is not how the members of the IEEE standardization committee looked at it in 1985, your statement that it must be a bug "no matter how you look at it" must be incorrect. Some industry experts do not look at it that way.

User: Good point. What motivated this design decision?

Eric: I wasn't there; I was busy playing Jumpman on my Commodore 64 at the time. But my educated guess is that it is desirable for all possible operations on all floats to produce a well-defined float result. Mathematicians would call this a "closure" property; that is, the set of floating point numbers is "closed" over all operations.

Positive infinity seems like a reasonable choice for dividing a positive number by zero. It seems plausible because of course the limit of 1 / x as x goes to zero (from above) is "positive infinity", so why shouldn't 1/0 be the number "positive infinity"?

Now, speaking as a mathematician, I find that argument specious. A thing and its limit need not have any particular property in common; it is fallacious to reason that just because, say, a sequence has a particular limit that a fact about the limit is also a fact about the sequence. Mathematically, "positive infinity" (in the sense of a limit of a real-valued function; let's leave transfinite ordinals, hyperbolic geometry, and all of that other stuff out of this discussion) is not a number at all and should not be treated as one; rather, it's a terse way of saying "the limit does not exist because the sequence diverges upwards".

When we divide by zero, essentially what we are saying is "solve the equation x * 0 = 1"; the solution to that equation is not "positive infinity", it is "I cannot because there is no solution to that equation". It's just the same as asking to solve the equation "x + 1 = x" -- saying "x is positive infinity" is not a solution; there is no solution.

But speaking as a practical engineer who uses floating point numbers to do an imprecise approximation of ideal arithmetic, this seems like a perfectly reasonable choice.

User: But surely it is impossible for the hardware to represent "infinity".

Eric: It certainly is possible. You've got 32 bits in a single-precision float; that's over four billion possible floats. All bit patterns of the form

?11111111???????????????????????

are reserved for "not-a-number" values. That's over sixteen million possible NaN combinations. Two of those sixteen million NaN bit patterns are reserved to mean positive and negative infinity. Positive infinity is the bit pattern 01111111100000000000000000000000 and negative infinity is 11111111100000000000000000000000.

User: Do all languages and applications use this convention of division-by-zero-becomes-infinity?

Eric: No. For example, C# and JScript do but VBScript does not. VBScript gives an error if you do that.

User: Then how do language implementors get the desired behaviour for each language if these semantics are implemented by the hardware?

Eric: There are two basic techniques. First, many chips which implement this standard allow the programmer to make float division by zero an exception rather than an infinity. On the 80x87 chip, for example, you can use bit two of the precision control register to determine whether division by zero returns an infinity or throws a hardware exception.

Second, if you don't want it to be a hardware exception but do want it to be a software exception, then you can check bit two of the status register after each division; it records whether there was a recent divide-by-zero event.

The latter strategy is used by VBScript; after we perform a division operation we check to see whether the status register recorded a divide-by-zero operation; if it did, then the VBScript runtime creates a divide-by-zero error and the usual VBScript error management process takes over, same as any other error.

Similar bits exist for other operations that seem like they might be better treated as exceptions, like numeric overflow.

The existence of the "hardware exception" bits creates problems for the modern language implementor, because we are now often in a world where code written in multiple languages from multiple vendors is running in the same process. Control bits on hardware are the ultimate "global state", and we all know how irksome it is to have global, public state that random code can stomp on.

For example: I might be misremembering some details, but I seem to recall that Delphi-authored controls set the "overflows cause exceptions" bit. That is, the Delphi implementors did not use the VBScript strategy of "try it, allow it to succeed, and check to see whether the overflow bit was set in the status register". Rather, they used the "make the hardware throw an exception and then catch the exception" strategy. This is deeply unfortunate. When a VBScript script calls a Delphi-authored control, the control flips the bit to force exceptions but it never "unflips" it. If, later on in the script, the VBScript program does an overflow, then we get an unhandled hardware exception because the bit is still set, even though the Delphi control might be long gone! I fixed that by saving away the state of the control register before calling into a component and restoring it when control returns. That's not ideal, but there's not much else we can do.

User: Very enlightening! I will be sure to pass this information along to my coworkers. I would be delighted to see a blog post on this.

Eric: And here you go!

Bug Psychology

Eric Lippert — Mon, 01 Jun 2009 16:48:00 GMT

Fixing bugs is hard.

For the purposes of this posting, I’m talking about those really “crisp” bugs -- those flaws which are entirely due to a failure on the developer’s part to correctly implement some mechanistic calculation or ensure some postcondition is met. I’m not talking about oops, we just found out that the product name sounds like a rude word in Urdu, or the specification wasn’t quite right so we changed it or the code wasn’t adequately robust in the face of a buggy caller. I mean those bugs where you were asked to compute some value and you just plain get the result wrong for some valid inputs.

Let me give you an example.

The first bug I ever fixed at Microsoft as a full-time employee was one of those. To understand the context of the bug, start by reading this post from the early days of FAIC, and then come back.

Welcome back, I hope you enjoyed that little trip down memory lane as much as I did.

Now that you understand how a VT_DATE is stored, that explains this bizarre behaviour in VBScript:

print DateDiff("h", #12/31/1899 18:00#, #12/30/1899 6:00#) / 24
print DateDiff("h", #12/31/1899 18:00#, #12/29/1899 6:00#) / 24

This prints –1.5 and –2.5, as you’d expect. There’s a day and a half between 6 AM December 30th and 6 PM December 31st, and two and a half days between the other two dates. This is perfectly understandable. But if you just subtract the dates:

print #12/31/1899 18:00# - #12/30/1899 6:00#
print #12/31/1899 18:00# - #12/29/1899 6:00#

You get 1.5 and 3, not 1.5 and 2.5. Because of the bizarre date format that VT_DATE chooses, when you convert dates to numbers, you cannot safely subtract them if they straddle the magic zero date. That’s why you need the helpful “DateDiff”, “DateAdd” and so on, methods.

The bug I was assigned was that testing had discovered a particular pair of dates which DateDiff was not subtracting correctly. I took a look at the source code for one of the helper methods that DateDiff used to do one of the computations it needed along the way. To my fresh-out-of-college eyes it looked something like this:

if (frob(x) > 0 && blarg(y)) return x – y;
else if (frob(x) < blarg(y) && blah_blah(x) > 0 || blah_de_blah_blah_blah(x,y)) return frob(x) – x + y + 1;
else if…

There were seven such cases.

My urge was to dive right in and add an eighth special case that fixed the bug. But my ability to get it right in the face of all this complexity concerned me. It seemed like this was an awfully complicated function already for what it was trying to do.

I researched the history of the code a bit and discovered that in fact variations on this bug had been entered… seven times. Each special case in the code corresponded to a particular bug that had been “fixed”, a term I use guardedly in this case. A great many of those “fixes” had actually introduced new bugs, regressing existing correct behaviour, which then in turn were “fixed” by adding special cases on top of the broken special cases that had been added to “fix” previous bugs.

I decided that this coding horror would end here. I deleted all the code (all seven lines of it! I was bold!) and started over.

Deep breath.

Spec the code requirements first. Then design the code to meet the spec. Then write the code to the design.

Spec:

* Input: two doubles representing dates in VT_DATE format.
* VT_DATE format: signed integer portion of double is number of days since 12/30/1899, unsigned fractional part is portion of day gone by.
* For example: –1.75 = 12/29/1899, 6 PM.
* Output: double containing number of days, possibly fractional, between two dates. Differences due to daylight savings time, and so on, to be ignored.

Design strategy:

* Problem: Some doubles cannot simply be subtracted because negative dates are not absolute offsets from epoch time
* Therefore, convert all dates to a more sensible date format which can be simply subtracted.

Code:

double DateDiffHelper(double vtdate1, double vtdate2)
{
return SensibleDate(vtdate2) – SensibleDate(vtdate1);
}
double SensibleDate(double vtdate)
{
// negative dates like –2.75 mean “go back two days, then forward .75 days”:
// Transform that into –1.25, meaning “go back 1.25 days”.
return DatePart(vtdate) + TimePart(vtdate);
}

I already had helper methods DatePart and TimePart, so I was done. The new code was shorter, far more readable, generated smaller, faster machine code and most important, was clearly correct. No special cases; no bugs.

It’s not that my coworkers were dummies. Far from it. These were smart people. But computer geek psychology is such that it is very easy to narrow-focus on the immediately wrong thing, and try to tweak it until it does the right thing.

When faced with these sorts of “crisp” bugs, I try to restrain myself from diving right in. Rather, I try to psychoanalyze the person – who is, of course, usually my past self – who caused the bug. I ask myself “how was the person who wrote the buggy code fooled into thinking it was correct?” Did they not have a clear specification of what the method was supposed to do? Was it misleading? Did they have a clear plan for how to proceed? If so, where did it go wrong?

If there never was either a spec or a plan, then for all you know the whole thing might only be working by sheer accident. There could be any number of design flaws in the thing that just haven’t come to light yet. Editing such a beast means adding unknown to unknown. which seldom leads to good results. Sometimes coming up with a new spec, a new plan and scrapping an existing bug farm is the best way to proceed.

For many years after that, I would ask how to implement DateDiffHelper as my technical question for fresh-out-of-college candidates that I was interviewing for the scripting dev team. I reasoned that if that was the sort of problem I was given on my first day in the office, then maybe that would be a reasonable question to ask a candidate.

When you ask the same question over and over again, you really get to see the massive difference in aptitude between candidates. I had some candidates who just picked up a marker, wrote a solution straight out on the board, wrote down the test cases they’d use to verify it, mentally ran a few of the tests in their head, and then we’d have another half hour to chat about the weather. And I had some candidates who tried earnestly to write the version using special cases, despite my specifically telling them “you might consider transforming this bad format into something more pleasant to work with”, after they got stuck on the third special case. I’d point out a bug and immediately they’d write down code for another special case, rather than stopping to think about the fact that they’d just written buggy code three times already and told me it was correct three times.

Reading Code Over the Telephone

Eric Lippert — Fri, 16 May 2008 17:00:00 GMT

In my youth I once attended a lecture given by Brian Kernighan on the subject of code quality, which was very influential on my attitudes towards writing legible code. One of the things that Kernighan recommended was to endeavour write code that was so clear that it could be easily read over the phone and understood. Most people find code much easier to comprehend when read than when heard; if you can make it clear enough to be understood when heard, it's probably pretty clear code.

I was reminded of this when I got the following question from a colleague by email:

Subject: Stupid C# 3.0 lambda expression question

How does one read the => operator?

First off, I told my colleague that there are no stupid questions, only stupid people. No stupid people work here, so don't stress about it. This is a perfectly sensible question.

As far as I know, we do not have an "official" line on how to read this operator over the phone. In the absense of any other context, I personally would say c=>c+1 as "see goes to see plus one". Some variations that I've heard:

For a projection, (Customer c)=>c.Name: "customer see becomes see dot name"

For a predicate, (Customer c)=>c.Age > 21: "customer see such that see dot age is greater than twenty-one"

An unfortunate conflation is that the => operator looks a lot like ⇒, the "implies" operator in mathematics. Since => does not have the same semantics as ⇒, it is probably a bad idea to read => as "implies". (x⇒y would have the semantics of !x|y in C#.)

Incidentally, it is a little known fact that VB6 and VBScript implemented the ⇒ operator with the Imp keyword and the ⇔ operator with the Eqv keyword. They disappeared in VB.NET. Where did they go? It is a mystery!

Do Not Call IsBadFooPtr, Indeed

Eric Lippert — Wed, 27 Sep 2006 21:16:00 GMT

Here’s a story that I said a long time ago that I was going to tell you all, and then promptly forgot about it. Raymond Chen’s blog entry today reminded me of it, because this is the story of how I found out the hard way that IsBadFooPtr is bad, bad, bad. Those of you who have been following releases of the script engines incredibly carefully may remember this story, but hopefully we caused and then fixed the problem sufficiently quickly that the vast majority of customers never noticed.

But I’m getting ahead of myself.

One of the most basic and important rules of COM programming is that you never, ever, EVER change an interface without changing its GUID. Or, to put it another way, you simply never change an interface; if you need to, you create an entirely new interface.

You may have wondered why is COM so strict about this rule. It’s because when you write a C++ program that uses an interface, the compiler will generate code that depends upon the particular pattern of stack frame layouts for passing arguments remaining exactly the same even when the user installs a new version of the COM object on their machine. Because COM objects and their “clients” can be versioned independently, the GUID has to be associated with a particular “binary contract” describing how the two pieces of code communicate.

Now, the script engines always talk to scriptable objects via IDispatch or IDispatchEx, so if a scriptable object accidentally breaks its contract, the script engines don’t notice a bit. They let the dispatch layer take care of all those irksome details about stack frame layout, and the dispatch layer figures all this stuff out from the type library data in the object itself.

Perhaps you see where this is going.

The Replace method of the VBScript regular expression object originally took two “in” arguments, both strings, and returned a string. This was the IRegExp interface. Later we realized that we wanted to be able to pass things other than strings for the second argument, so we created a new IRegExp2 interface where the Replace method took a string and a variant, and returned a string. So far, so good.

Then we ported the script engines to the IA64 processor. During the port for some unknown reason one of the developers temporarily changed the second argument from a VARIANT to a VARIANT*, put a comment in the code saying that this needed to be changed back before it was checked in, checked it in, and, uh, we kinda shipped it like that.

We thoroughly violated the rules of COM, but none of our automated tests showed this, because of course, all of the scripting automated tests were written – you guessed it – in script. We had no tests which used the scriptable objects from early-bound code. (We do now!)

Unfortunately, some of our developer customers were using the VBScript regular expression object from C++ and other early bound languages, so as soon as their customers upgraded to the latest version of scripting, all hell broke loose. The caller was fulfilling their end of the contract by putting a sixteen byte variant on the stack, and the callee was suddenly expecting a four byte pointer to a variant, so it was both crashing and misaligning the stack.

We immediately started getting bug reports, of course. About half the bug reports said “please change this back ASAP, our customers are broken”, and the other half said “we have already shipped a patch of our code to our customers using the new stack layout, please do not change it back, ever, we do not want to ship another patch”.

Obviously we realized that we had badly screwed up and were now in a hideous bind. We came up with three options:

First, we could revert to the previous layout, and inconvenience the small number of customers who had already shipped patches to their customers. (And of course, inconvenience all of those people as well, who would have to apply a second patch in as many days.)

Second, we could keep the current layout and tell the rest of our affected customers that they needed to patch our mistake.

Third, we could revert to the previous layout and yet NOT inconvenience anyone.

Obviously the third option is best, but how to do it?

I ended up writing heuristics into the Replace method that examined the stack frame, and then rerouted the call to a special helper method if we detected a VARIANT* on the stack when we were expecting a VARIANT. The helper method did all the necessary magic to adjust the stack pointer, and so on, so that from the caller’s perspective, everything looked perfectly normal.

Now, remember, we expected that in the vast majority of cases, a VARIANT would be passed, not a VARIANT*. My heuristic detected which case we were in by extracting four bytes from the middle of the stack frame and calling IsBadReadPtr on it. If that was a valid pointer then I’d cast it to a VARIANT*, check to see whether the variant type field was a string (by far the most likely argument to Replace), and then call IsBadWritePtr on the bstrVal (because BSTR pointers are almost always writeable). There were a few other details to the heuristic, but the point is that in the vast majority of cases, IsBadReadPtr was expected to fail. Only in the rare cases where we were in an early-bound and third-party patched program should it succeed.

We shipped the fixed version with the heuristics out immediately.

At the time I did not realize how IsBadReadPtr was implemented. It actually puts a try-except around an attempt to read the pointer! If it gets an exception then it eats the exception and says yeah, it's bad. Essentially what I had done was inserted an almost-always-thrown exception into every call to Replace.

My goal of not inconveniencing anyone turned out to not quite work out as planned. The Active Server Pages team went nuts that afternoon. Their tests assume that all exceptions, whether handled or not, are flaws somewhere in ASP or scripting. Their test machines are all configured to track and report all first-chance exceptions, and suddenly every compatibility and stress test machine they had that happened to do a regular expression replace – that is, approximately all of them – suddenly started reporting hundreds of first-chance exceptions per minute when they upgraded to the latest released version.

So just when I thought that I could breathe easy again for a few minutes I started getting calls from the now highly vexed ASP team. I ended up writing my own versions of IsBadFooPtr which do the right thing – they call VirtualQuery to ask the operating system what the protection state of a given address is, rather than simply trying it out and throwing a first-chance exception on failure.

Long story short: don’t change an interface, don’t call IsBadFooPtr, and run your changes by all your important consumers before you ship a tricky fix!

Checking For Script Syntax Errors, This Time With Code

Eric Lippert — Wed, 12 Oct 2005 19:37:00 GMT

A number of people asked me to clarify yesterday's entry. Rather than try to talk you through it, I think the code is straightforward enough to speak for itself. Here's a little skeleton that I just whipped up.

#include <stdio.h>
#include <activscp.h>
#include <new>

const GUID CLSID_VBScript = {0xb54f3741, 0x5b07, 0x11cf, {0xa4, 0xb0, 0x00, 0xaa, 0x00, 0x4a, 0x55, 0xe8}};
const GUID CLSID_JScript = {0xf414c260, 0x6ac0, 0x11cf, {0xb6, 0xd1, 0x00, 0xaa, 0x00, 0xbb, 0xbb, 0x58}};

class MySite : public IActiveScriptSite {
private:

ULONG m_cref;
virtual ~MySite() {}

public:

  MySite() {
   this->m_cref = 1;
  }

STDMETHOD_(ULONG, AddRef)() {
return ++this->m_cref;
}

STDMETHOD_(ULONG,Release)() {
    --this->m_cref;
  if (this->m_cref == 0) {
     delete this;
      return 0;
  }
    return this->m_cref;
  }

  STDMETHOD(QueryInterface)(REFIID iid, void ** ppv) {
   if (ppv == NULL)
      return E_POINTER;
    if (IsEqualIID(iid, IID_IUnknown))
      *ppv = (IUnknown*)this;
    else if (IsEqualIID(iid, IID_IActiveScriptSite))
   *ppv = (IActiveScriptSite*)this;
    else {
      *ppv = NULL;
      return E_NOINTERFACE;
    }
    this->AddRef();
    return S_OK;
}

STDMETHOD(GetLCID)(LCID * plcid) {
return E_NOTIMPL;
}

  STDMETHOD(GetItemInfo)(
  LPCOLESTR pstrName,
   DWORD dwReturnMask,
    IUnknown ** ppunkItem,
    ITypeInfo ** ppti) {
    return E_NOTIMPL;
  }

STDMETHOD(GetDocVersionString)(BSTR * pbstrVersion) {
return E_NOTIMPL;
}

STDMETHOD(OnScriptTerminate)(
   const VARIANT * pvarResult,
   const EXCEPINFO * pexcepinfo) {
    return S_OK;
  }

STDMETHOD(OnStateChange)(SCRIPTSTATE state) {
return S_OK;
}

  STDMETHOD(OnEnterScript)() {
   return S_OK;
}

STDMETHOD(OnLeaveScript)() {
    return S_OK;
  }

STDMETHOD(OnScriptError)(IActiveScriptError * perror) {
    EXCEPINFO excepinfo;
    LONG column = 0;
    ULONG line = 0;
    DWORD context = 0;
    BSTR bstrLine = NULL;
    memset(&excepinfo, 0x00, sizeof excepinfo);
   perror->GetExceptionInfo(&excepinfo);
    if (excepinfo.pfnDeferredFillIn != NULL)
     excepinfo.pfnDeferredFillIn(&excepinfo);
    perror->GetSourceLineText(&bstrLine);
   perror->GetSourcePosition(&context, &line, &column);

   wprintf(L"Error on line %ld column %ld\n", line, column);
   if (bstrLine != NULL)
      wprintf(L"Line: %s\n", bstrLine);
    if (excepinfo.bstrDescription != NULL)
     wprintf(L"Description: %s\n", excepinfo.bstrDescription);
    if (excepinfo.bstrSource != NULL)
     wprintf(L"Source: %s\n", excepinfo.bstrSource);
    if (excepinfo.bstrHelpFile != NULL)
     wprintf(L"Help: %s\n", excepinfo.bstrHelpFile);

    SysFreeString(bstrLine);
    SysFreeString(excepinfo.bstrDescription);
    SysFreeString(excepinfo.bstrSource);
    SysFreeString(excepinfo.bstrHelpFile);

return S_OK;
}
};

void main() {
HRESULT hr = S_OK;
HRESULT hrInit;
IClassFactory * pfactory = NULL;
IActiveScript * pscript = NULL;
IActiveScriptParse * pparse = NULL;
IActiveScriptSite * psite = NULL;

hr = hrInit = OleInitialize(NULL);
if (FAILED(hr))
goto LError;

hr = CoGetClassObject(CLSID_VBScript, CLSCTX_SERVER, NULL,
IID_IClassFactory, (void**)&pfactory);
if (FAILED(hr))
goto LError;

hr = pfactory->CreateInstance(NULL, IID_IActiveScript, (void**)&pscript);
if (FAILED(hr))
goto LError;

psite = new(std::nothrow) MySite();
if (psite == NULL) {
hr = E_OUTOFMEMORY;
goto LError;
}

hr = pscript->SetScriptSite(psite);
if (FAILED(hr))
goto LError;

hr = pscript->QueryInterface(IID_IActiveScriptParse, (void**)&pparse);
if (FAILED(hr))
goto LError;

hr = pparse->ParseScriptText(L"Function Foo \n Foo = 123 \n End Funtcion \n",
NULL, NULL, NULL, 0, 1, 0, NULL, NULL);

if (FAILED(hr))
goto LError;

LError:

if (FAILED(hr))
printf("%0x\n", hr);

if (pparse != NULL)
pparse->Release();

if (psite != NULL)
psite->Release();

if (pscript != NULL) {
    pscript->Close();
   pscript->Release();
  }

if (pfactory != NULL)
pfactory->Release();

if (SUCCEEDED(hrInit))
OleUninitialize();
}

As you would expect, this program prints out the information about the error, and ParseScriptText returns SCRIPT_E_REPORTED to indicate that there was an error but it has already been reported. Had there been no error, the script would not have actually run; the engine is not started, just initialized.

Error on line 3 column 5
Line: End Funtcion
Description: Expected 'Function'
Source: Microsoft VBScript compilation error
80020101

Checking For Script Syntax Errors

Eric Lippert — Tue, 11 Oct 2005 20:35:00 GMT

A reader asked me recently whether there was a way to check a chunk of JScript or VBScript for syntax errors without actually running the code. I'm sure that there are many third-party tools which you could find that do this. If you have your own script host, you can do it yourself quite easily.

The trick is to initialize the engine by setting the script site, but do not do anything that would move the engine from initialized into started state. (Note that attempting to evaluate an expression moves the engine to started state.)

If the engine is initialzed but not started then calling ParseScriptText will check the text passed in for syntax errors but will not run the script. Rather, if the script compiles successfully, it is simply marked as "needs to run when the engine is started".

Unfortunately the script engines do not have an error-recovering parser, so they will detect only the first syntax error and then bail out. Still, better than nothing.

Implementing Event Handling, Part Two

Eric Lippert — Wed, 21 Sep 2005 21:51:00 GMT

It's been an insanely busy month for me, between having multiple out-of-town guests, throwing a party for the people who couldn't make it to the wedding, and oh yeah, getting up to speed on the C# compiler and trying to understand the implications that LINQ features are going to have on the current implementation. Not much time for blogging. Hopefully October will be a little bit more under control.

Anyway, I was talking about early-bound event binding. Basically the idea is that the source and the sink agree upon an interface that the source can call on the sink whenever the source wishes to fire an event. Before we get into that more, I want to talk briefly about early- and late-bound code.

At an implementation level, what is the difference between early- and late- bound code? In the early-bound world, a particular function on an interface gets called when the caller dereferences the callee's virtual function table and transfers control directly to the function by changing the instruction pointer on the chip. In the late-bound world, the caller calls IDispatch::Invoke and passes in a magic number that tells the callee what function it should be calling, and the callee is then responsible for dispatching the function appropriately. (Hence the name "IDispatch".)

The script engines use IDispatch all the time to call out to functions on objects. When calling an object late-bound, you don't know until you actually try the call whether or not it is going to succeed, because you do not know the interface the object implements. So you call GetIdsOfNames to get the magic number associated with a particular function, Invoke on that magic number, and let the callee sort it out. In this situation, the caller (the script engine) knows the name of the function it wants to call, and the callee (the object) can map the name to the appropriate identifier.

Now consider how this works if IDispatch is the interface over which the source (the object) is calling back the sink (the script host). This appears to be the same situation, but in fact it is completely different. It appears the same because from the perspective of how COM actually manages all the calls, its exactly the same. IDispatch is an interface like any other, and the source can call the sink's IDispatch to its heart's content if that's the interface that they agree to talk over.

But look at it from the point of view of the sink: the caller knows what the magic number means and the callee does not, but it's the callee who is being asked to do the dispatching! You're sitting there sinking events of who knows what object, and every now and then you get a call on IDispatch::Invoke with some unknown dispid. What the heck are you supposed to do with that? In the late-bound world, instead of getting a nice direct call on "Tick()... Tick()... Tick()..." you're getting "12... 12... 12..." and you have no idea what "12" means.

Let's tie this in to scripting. Suppose you're in Windows Script Host and you do something like:

Sub Timer_Tick()
'whatever
End Sub
Set Timer = CreateObject("mytimer")
WScript.ConnectObject Timer, "Timer_"
Timer.Start 10
WScript.Sleep 1000

There are many IDispatch objects here and we'll look at three of them. First, there's the source, that is, the timer object. Then there's the sink, owned by the host and created when the source is connected to it. Finally there is the script engine itself, upon which the host can dispatch calls to global functions such as the event handler.

ConnectObject is given a source object. It has no idea what early-bound outgoing interface is on that object, and it certainly doesn't have an implementation of such an interface even if there is one, so it is going to have to do a late bound sink. It creates an object to act as the sink, gets an IDispatch-enabled connection point from the source, and advises it.

Now the script engine calls Start (via IDispatch on the source) and goes to sleep.

Pretty soon the timer invokes the sink, passing in the dispatch identifier for Tick.

The sink needs to know which sub to call in the script engine.

Fortunately, it has an IDispatch pointer to the source, and a dispatch identifier.

Unfortunately, though IDispatch provides a function which maps from name to id, it provides no function that maps the other way.

Fortunately, IDispatch does provide a function that enables the host to obtain a type information structure which contains a list of all the methods and what their dispatch identifiers are. Therefore we can search all the methods in the type info and check to see which has the desired dispatch identifier. That then gives us the method name, so we know which event handler function to dispatch on the script engine.

Unfortunately there is a major design flaw in IDispatch -- it gives you the type info for the incoming interface -- the ITimer. But there is no way to take the ITypeInfo for ITimer and say "give me the type info for the class as a whole so that I can obtain the type info for the outgoing interface".

Fortunately, IProvideClassInfo was invented. IPCI gives you back the "root" type info for an object, from which you can obtain a type info for the default outgoing interface, from which you can map the dispatch identifier to the name.

Unfortunately, many objects do not implement IProvideClassInfo, and therefore cannot have their events hooked up by ConnectObject.

(I feel a little bit like the ending of Dr. Strangelove here. Fortunately, they stop General Ripper in time. Unfortunately, the secret code died with him. Fortunately they figure out the code. Unfortunately, the radio is broken. Fortunately, the bomb bay doors are stuck. Unfortunately, they fix them, and the Doomsday Device destroys the world. Bummer.)

The moral of this story is simple: for late bound events to work, at some point the class of the source must be known. And therefore:

"automagic event binding" in IE only works on "named items" present in the script engine. (The script engines require that the host provide a coclass type info for named items, and build the sinks very early on, before other code runs.)
WScript.CreateObject can always hook up events. The class type info is known because CreateObject had to create an instance of the class.
WScript.ConnectObject can hook up events only if the object implements IProvideClassInfo

I hope that clears up any confusion about what IProvideClassInfo is for!

250% of what, exactly?

Eric Lippert — Thu, 01 Sep 2005 20:13:00 GMT

I just got a question this morning about how to take two collections of items and determine how many of those items had the same name. The user had written this straightforward but extremely slow VBScript algorithm:

For Each Frog In Frogs
For Each Toad In Toads
    If Frog.Name = Toad.Name Then
      SameName = SameName + 1
      Exit For
    End If
Next
Next

There were about 5000 frogs, 3000 toads and 1500 of them had the same name. Every one of the 3500 unsuccessful searches checked all 3000 toads, and the 1500 successful searches on average checked 1500 toads each. Each time through the inner loop does one loop iteration, two calls to the Name property, one string comparison. Add all those up and you get roughly 50 million function calls to determine this count.

This code has been somewhat simplified – the actual user code was doing more work inside the loop, including calls to WMI objects. The whole thing was taking 10+ minutes, which is actually pretty fast considering how much work was being done. Each individual function call was only taking a few microseconds, but fifty million calls adds up!

Now, we've been down this road before in this blog (here, here and here) and so of course I recommended building a faster lookup table rather than doing a full search through the collection every time.

Set FrogLookup = CreateObject("Scripting.Dictionary")
For Each Frog In Frogs
FrogLookup(Frog.Name) = Frog
Next
For Each Toad In Toads
If FrogLookup.Exists(Toad.Name) Then SameName = SameName + 1
Next

Which is much, much faster. That's only about 16 thousand function calls. Now, this is maybe not an apples-to-apples comparison of function calls, but we at least have good reason to believe that this is going to be several times faster.

And indeed it was. But I'm still not sure how much, which brings me to the actual subject of today's blog. The user reported that the new algorithm was "250% better". I hear results like this all the time, and I always have to ask to clarify what it means. You can't just say "n% better" without somehow also communicating what you're measuring.

(UPDATE: This reported result understandably confused some readers. Clearly the new loop here is thousands of times faster, not a mere 250% faster. As I said before, the sketch above is highly simplified code. The real-world code included many thousands of additional calls to WMI objects which were not eliminated by this optimization. Eliminating these 50 million function calls helped -- you should always eliminate the slowest thing first! But doing so also exposed a new "hot spot" that needed further optimization. However, the point of this article is not the benefits of using lookup tables, but rather that using unexplained percentages to report performance results is potentially misleading. The result above is merely illustrative. See the comments for more details.)

Allow me to illustrate. Suppose I have a web server that is serving up ten thousand pages every ten seconds. I make a performance improvement to the page generator so that it is now serving up fifteen thousand pages every ten seconds. I can sensibly say:

performance has improved by 50%, because we are now serving up 5000 more pages every ten seconds, and 5000 is 50% of the original 10000. In this world, any positive percentage is good.
performance is now 150% of original performance because 15000 is 150% of 10000. In this world, 0%-100% worse or the same, 100%+ is good.
We've gone from 1000 microseconds per page to 667 microseconds per page, saving 333 microseconds per page. 333 is 33% of 1000, so we've got a 33% performance improvement. In this world, 0% is bad, 100% is perfect, more than 100% is nonsensical.

If I can sensibly say that performance is better by 50%, 150% and 33%, all referring to exactly the same improvement, then I cannot actually be communicating any fact to my listeners! They need to know whether I'm measuring speed or time, and if speed, whether I'm comparing original speed to new speed or original speed to the difference.

So what is "250%" better? Clearly not raw timings. If this loop took 10 minutes to run before, 250% of 10 minutes is 25 minutes, but surely it is not running in -15 minutes now! I assume that the measurement is speed -- say, number of loops runnable per hour. If before it took ten minutes and therefore we could run this loop six times per hour, and now it is 250% better, 250% of 6 is 15, and this is "better", so we need to add. So that's 21 loops per hour, or about 3 minutes per loop. Had the user accidentally said "250% faster" but meant to say "250% of previous performance" then we'd conclude that 250% of 6 per hour is 15 per hour, so now we've got 4 minutes per loop.

And of course, this is assuming that the person reporting the improvement is actually trying to communicate facts to the audience. Be wary! Since 33%, 50% and 150% are all sensible ways to report this improvement, which do you think someone who wants to impress you is going to choose? There are opportunities here for what Kee Dewdney calls "percentage pumping" and other shyster tricks for making weak numbers look more impressive. Professor Dewdney's book on the subject, "200% of Nothing", is quite entertaining.

The moral of the story is: please do not report performance improvements in percentages. Report performance improvements in terms of raw numbers with units attached, such as "microseconds per call, before and after change", or "pages per second, before and after change". That gives the reader enough information to actually understand the result.

(And the other moral is, of course, lookup tables are your friends.)

VBScript Default Property Semantics

Eric Lippert — Tue, 30 Aug 2005 20:15:00 GMT

Here’s a question I recently got about VBScript, where by "recently" I mean August 28th, 2003. This code works just fine:

Set myObject = CreateObject("myObject")
myObject.myName = "Eric"
WScript.Echo myObject ' myName is the default property, so prints "Eric"

But myObject = "Robert" doesn't set the default property, it sets the variable to the string. Why does reading work but writing fail? This works in VB6, why not VBScript?

In a strongly typed language such as VB6 the compiler can look at the compile-time types of the left and right sides of an assignment and determine that the type of the left hand side is an object, the right hand side is a string, and the object has a default property which is a string. The VB6 compiler can then generate the appropriate code to assign the string to the default property.

But what if you wrote a VB6 program using only variants? When the compiler sees foo = bar typed as variants it cannot tell whether this means "set the default property of foo to bar" or "set foo to bar". The VB compiler chooses the latter every time. VBScript is a weakly typed subset of VB -- in VBScript, everything is a variant. So in VBScript, all assignments are treated as though the value is actually being assigned to the variable, not the default property. Therefore this is by design - this is for compatibility with VB6's behaviour. Had we written the same program in weakly-typed VB6, we'd get the same result.

I hear you exclaiming "The fact that a difference in available type information leads to a difference in run-time behaviour violates a basic principle of programming language design! Namely, the principle that late-bound calls have exactly the same semantics as early-bound calls!"

Indeed, as I've mentioned before, VB6 and VBScript violate this principle in several places. Default properties were, in my opinion, a bad idea all around, for this and other reasons. They make the language harder to parse and hence harder to understand. In particular, parameterless default properties make very little sense in VBScript. However, we are stuck with them now.

However, I should call out that there are some things we can do at runtime. Suppose blah is an object with a property fred that is an object, and fred has a default property which is a string. If you say foo = blah.fred then foo is assigned the default value of fred even though we lack the compile-time type information. foo isn't set to the object fred. Of course, in this case we know to fetch the default property at runtime because we know the runtime type of fred and also know that there is no Set keyword. These two facts are sufficient to cause the runtime engine to always fetch the default property.

Or suppose you have an object Baz with a property Table, Table has a default parameterized property Item which returns an object that has a string property Blah: Then this works just fine:

x = Baz.Table(1).Blah

But why? In this case we do not have enough compile-time information to determine that we really should call Baz.Table.Item(1).Blah. The script engine has every reason to believe that Table is a function or parameterized property of Baz. This problem is solved by pushing it off to the implementation! The rule for implementers of IDispatch::Invoke is if all of the following are true:

the caller invokes a property
the caller passes an argument list
the property does not actually take an argument list
that property returns an object
that object has a default property
that default property takes an argument list

then invoke the default property with the argument list. Strange but true.

Perhaps unsurprisingly, not very many people know about that rule! This is yet another reason to never, ever write your own implementation of IDispatch::Invoke. It also leads to some mind-numbingly complex code in the VBScript IntelliSense engine that Visual Studio uses. Making default property IntelliSense work with such a dynamically typed language was a piece of work, lemme tell ya. As I mentioned earlier, there are in fact some odd corner cases that are slightly different between VB6 and VBScript IntelliSense. I also talked a bit about left-hand-side default property semantics earlier.

Scripting Type Library Constant Injection Performance Characteristics, Part One

Eric Lippert — Wed, 22 Jun 2005 17:00:00 GMT

(Sorry about the title. I work for Microsoft; we like nouns.)

Over a year ago now a reader noted in a comment to this posting that defining named constants using Const in VBScript or var in JScript is way, way faster than importing a type library.

My empirical JScript testing showed that loading constants from a typelib is one or two orders of magnitude slower than preloading an engine with script which defines those same constants.

I responded

I wrote all the type library importation code -- you are correct, it is NOT FAST compared to simply defining a bunch of consts/vars. There are good reasons why. I'll blog about them at some point.

At long last I'll get around to explaining why. There are interesting design tradeoffs here, and as often happens in the scripting world, performance was traded off against ease-of-use. We're built for comfort, not speed. As I have said many times before, if you care about maximizing performance, using a late-bound unoptimized bytecode-interpreted dynamically-typed language is probably a bad choice.

Let's go through the design process first, and then get into the implementation details and what the performance costs are.

Let's make up a little syntax to describe the type libraries that I'm talking about. Note that this is not VBScript, I just need a concise way to describe the enumerated type contents of a type library. (String constants in modules work the same way and nothing else in the library is relevant, so without loss of generality I'll be considering only enumerated types in this series.) I'll make it a bracy language to emphasize that this is not VBScript.

typelib Video {
enum Color {
    Red = 1
    Green = 2
    Blue = 3
    Black = 4
    White = 5
}
}

typelib Food {
enum Cheese {
    Green = 1
    Blue = 2
    Yellow = 3
    White = 4
}
}

What does this VBScript program do if those type libraries have been added to the script engine namespace?

foo.bar = Blue

We've got a bit of a collision problem here. An obvious technique to solve this problem is to allow qualification:

foo.bar = Color.Blue ' OK, it's 3

Which works great until we also add this type library:

typelib Magic {
enum Color {
White = 1
Black = 2
}
}

Now it's not clear what foo.bar = Color.White means. We need two levels of disambiguation:

foo.bar = Video.Color.White ' five

Our major design decision here is to decide whether full qualification is a requirement all the time, or only when there is a collision. Or, somewhere in between; C# requires that enumerations be qualified by at least the type name but makes explicit full qualification optional via the "using" statement. C doesn't allow qualification at all -- you get a collision, that's your problem. (This is why you see so many enums in C that look like enum MagicColor { magicColorWhite, magicColorBlack }; in an attempt to avoid collisions.

We gave this a lot of thought and decided that the likely scenario was that collisions would be rare. Therefore users should not be forced to always qualify their enumerations at either the enumeration or library level. However, since collisions might occur, we should not act like C. In short, full, partial and no qualification should all be legal.

There are other design constraints that might not be immediately obvious. For example, full qualification must always work even in a world where type information is not known at compile time. Suppose you have the three type libraries above plus

typelib Cheese {
enum Blue{
Stilton = 1
Gorgonzola = 2
}
}

The code foo.bar = Cheese.Blue.Stilton must succeed -- we can't decide at runtime that this really means the nonsensical Food.Cheese.Blue.Stilton and raise an error. But we cannot do any analysis before the code runs either, because for all we know, someone used Execute to introduce a local object variable called Cheese, which would then win over either of those options. Furthermore, we might not even have the type libraries available when the compiler runs: the host can choose to add them after the script has been compiled. (Though that would be irksome, it is legal.) We can't make any assumptions at code generation time -- we have to assume that this is a series of object property calls like any other.

Next time we'll revisit how the script engines handle host-supplied objects, and see how that mechanism neatly solves our design problem. Then we'll see why this isn't fast compared to declaring the constant yourself.

Bad Recursion Revisited

Eric Lippert — Thu, 28 Apr 2005 21:12:00 GMT

We have internal email lists for questions about programming languages. Here's one that came across recently that I thought illustrated a good point about language design.

An interview candidate gave the following awful implementation of the factorial function. (Recall that factorial is notated "n!", and is defined as the product of all the integers from 1 to n. 0! is defined as 1. So 4! = 4 x 3 x 2 x 1 = 24.)

If you note that n! = n x ((n-1)!) then a recursive solution comes to mind. When asked to implement the factorial function in C, an interview candidate came up with this:

int F(int x){
return (x > 1) ? (x * F(--x)) : x;
}

Now, leaving aside for the moment the fact that this badly-named function does no bounds checking on the inputs, potentially consumes the entire stack, returns a wrong or nonsensical answer for inputs less than one, and is a recursive solution to a problem that can easily be solved with a simple lookup table, it has another big problem -- it doesn't even return the correct answer for any input greater than one either! F(4) will return 6, not 24, in Microsoft C.

And yet the seemingly equivalent C#, JScript and VBScript programs return 24.

The question was "What the heck is up with that?"

This looks perfectly straightforward. If x is 4, then we evaluate the consequence of the trinary operator. 4 * F(3) = 4 * 3 * F(2) = 4 * 3 * 2 * F(1) = 4 * 3 * 2 * 1 = 24, right? What is broken with C?

Page 52 of K&R has the answer.

"C, like most languages, does not specify the order in which the operands of an operator are evaluated. (The exceptions are &&, ||, ?: and ','.) For example, in a statement like x = f()+g(); f may be evaluated before g or vice versa; thus if either f or g alters a variable on which the other depends, x can depend on the order of evaluation."

The Microsoft C compiler actually evaluates the function call first, which causes x to be decremented before the multiplication. So this is actually the same as 3 * F(3) = 3 * 2 * F(1) = 3 * 2 * 1 = 6.

Why does the specification call out that the compiler can choose any order for subexpression evaluation? Because then the compiler can choose to optimize the order so that it consumes a small number of stack or register slots for the results.

Of course, C was written back in the dark ages -- the 1970's -- when indeed most languages were pretty ill-specified and full of terrible "gotchas" like this. JScript, VBScript, C#, and most modern languages do guarantee that functions will be evaluated in left-to-right order. In all these languages,

x = f() + g() * h();

will evaluate f, then g, then h.

As K&R notes "The moral is that writing code that depends on order of evaluations is a bad programming practice in any language." Amen to that!

How Do Script Engines Implement Object Identity?

Eric Lippert — Tue, 26 Apr 2005 21:49:00 GMT

I've talked a few times in this blog about the semantics of the equality operators in various languages. (Such as here, and here.) Recently a reader asked me how JScript implements object identity. That is, given two objects, how do we know if they are "the same object" or not?

For the remainder of this discussion I'm going to assume that we've got two non-null objects that we're comparing for identity. The details of how objects are compared to strings, numbers, etc, will have to wait for another blog entry.

Note also that in VBScript, the comparison operator for objects is Is, not =. Also, == and === in JScript behave identically if both arguments are objects; essentially the implementation of Is, == and === are all the same as far as object comparison goes.

Behind the scenes, everything in a VBScript/JScript program is represented as a variant. Most of the time, variants containing objects will have their variant type field set to VT_DISPATCH, and their pdispVal field set with a pointer to the object. (Occasionally we'll have a VT_UNKNOWN with a punkVal field instead, but lets ignore that corner case for now.) Suppose we want to compare two such objects for equality. We could pass the pdispVal of each to a helper function and compare the pointers.

bool AreObjectsEqual(IDispatch * pdisp1, IDispatch * pdisp2)
{
if (pdisp1 == pdisp2)
return true;
return false;
}

And we're done. Wow, that was easy.

Except that it's wrong. In VBScript, those objects could be two different dispatch pointers to the same object because VBScript supports non-default dispatches. JScript does not support non-default dispatches, but in JScript there are also situations in which you can have two different dispatch pointers to the same object; we'll come to that later.

If its not clear why the two pointers to the same object could be numerically different, read Raymond's article on how COM objects are laid out in memory, and you'll see why. There can be many dispatch vtables.

How can we ever determine when two COM objects are the same? We have to rely upon one of the most important rules of COM: if an object multiply inherits IUnknown then calling QueryInterface for IUnknown on any of those implementations must always give back the same pointer. As you can see in Raymond's example of multiple inheritance, there are two IUnknown vtables. It is illegal for each of them to return a pointer to itself when QI'd for IUnknown; the implementation must consistently pick one of them every time.

So now it's pretty clear what we must do:

bool AreObjectsEqual(IDispatch * pdisp1, IDispatch * pdisp2)
{
IUnknown * punk1 = NULL;
IUnknown * punk2 = NULL;
if (pdisp1 == pdisp2)
    return true;
if (pdisp1 == NULL || pdisp2 == NULL)
    return false;
pdisp1->QueryInterface(IID_IUnknown, (void**)&punk1);
pdisp2->QueryInterface(IID_IUnknown, (void**)&punk2);
// This should never fail, but better safe than sorry.
if (punk1 != NULL)
    punk1->Release();
if (punk2 != NULL)
    punk2->Release();
// We're not dereferencing the pointers, so it is OK to use the
  // locals even after their release.
if (punk1 == punk2)
    return true;
return false;
}

Unfortunately, that's not the whole story either.

For security reasons, Internet Explorer sometimes creates proxy objects -- little objects that detect if they are being used safely, and if they are, then they forward the call to the real implementation, on a separate object. You can run into situations in IE where you have two proxy objects to the same "real" object -- as far as COM is concerned, all three are different objects, but as far as IE is concerned, that's an implementation detail of the security system which we would like to hide from the script users.

To solve this problem, IE implements a special interface that knows when two IUnknown objects are both proxies for the same object. The script engines actually do something like this:

bool AreObjectsEqual(IDispatch * pdisp1, IDispatch * pdisp2) {
IUnknown * punk1 = NULL;
IUnknown * punk2 = NULL;
IObjectIdentity * pObjectIdentity;
bool fRet = false;
HRESULT hr;
if (pdisp1 == pdisp2)
    fRet = true;
else if (pdisp1 != NULL && pdisp2 != NULL) {
    // This must always succeed.
    pdisp1->QueryInterface(IID_IUnknown, (void**)&punk1);
    pdisp2->QueryInterface(IID_IUnknown, (void**)&punk2);
    if (punk1 == punk2)
      fRet = true;
    else if (punk1 != NULL && punk2 != NULL) { // This should never be false.
      hr = punk1->QueryInterface(IID_IObjectIdentity, (void **)&pObjectIdentity);
      if (SUCCEEDED(hr)) && pObjectIdentity != NULL) {
        hr = pObjectIdentity->IsEqualObject(punk2);
        if (hr == S_OK)
          fRet = true;
      }
    }
}
if (pObjectIdentity != NULL)
    pObjectIdentity->Release();
if (punk1 != NULL)
    punk1->Release();
if (punk2 != NULL)
    punk2->Release();
return fRet;
}

Which is a heck of a lot of code just to determine if two pointers are equal, but that's the tax you pay for object-oriented programming. Of course, you don't need to do any of this rigamarole with IObjectIdentity unless you're writing code that messes around with IE objects.

Binary Files and the File System Object Do Not Mix

Eric Lippert — Wed, 20 Apr 2005 21:47:00 GMT

OK, back to scripting today.

But before I get back to scripting issues, one brief correction. An attentive reader noted that "The Well-Tempered Clavier" was in fact designed to sound good on a "well tempered" instrument, not an "equally tempered" instrument. The difference is that a "well" temperament is designed so that every key sounds good, but is allowed to have some badly-out-of-tune intervals that must be avoided. (Traditionally these are called "wolf intervals".)

There was considerable controversy when equal temperament was introduced in Europe. I suppose it was the "what is the One True Bracing Style?" ridiculous issue of the day.

Another commenter pointed out that you could translate my wav-writing program into VBScript by using the File System Object to write out the bytes. To simplify their code down to a program that writes out individual bytes:

' DO NOT DO THIS
Set FSO=CreateObject("Scripting.FileSystemObject")
Set File=FSO.CreateTextFile("c:\test.bin", True)
For i = 0 to 255
File.Write Chr(i)
Next
File.Close

And sure enough, this writes out a binary file consisting of those bytes.

Please don't do that. See that line that says "CreateTextFile"? We wrote that method to create a text file, not a binary file. Though this code might appear to work, it actually does not. Text files are more than just binary files that can be interpreted as text. Text files have to conform to certain rules to ensure that they can be sensibly interpreted as text in the local code page. If that's not 100% clear to you, read Joel's article on the subject before we go on.

Let me give you an example that clearly fails. What does this program do?

Set FSO=CreateObject("Scripting.FileSystemObject")
Set File=FSO.CreateTextFile("c:\test.bin", True)
For i = 0 to 255
File.Write Chr(&hE0)
Next
File.Close

If you said "it writes out a binary file consisting of 256 E0 bytes," bzzt! Sorry, try again. The correct answer is "it writes out a binary file consisting of 256 E0 bytes on any operating system where the user's default ANSI code page does not define E0 as a lead byte in a DBCS encoding, like, say, Japanese, in which case it writes out 256 zeros."

In the Japanese code page, just-plain-chr(E0) is not even a legal character, so Chr will turn it into a zero.

If I were whipping up a little one-off program on my own to write out a binary file -- well, I'd personally do it in C, but I can see how some people might want to do it in script. But there's a big difference between writing a one-off program that you're going to delete in five minutes, and writing a general-purpose utility program that you expect people around the world will use. That's an entirely different standard of robustness and portability. Do not use the FSO to read/write binary files, you're just asking for a world of hurt as soon as someone in DBCS-land runs your code.

I have been asked many times over the years if I know of a scriptable object that can read-write true binary files in all locales. I do not. Anyone have any suggestions? I would have thought given the number of people that have asked me, that some third party would have come up with something decent by now.

VBScript Quiz Answers, Parts Eleven and Twelve

Eric Lippert — Tue, 05 Apr 2005 21:25:00 GMT

11) X and Y are both Booleans. Which of the following always assigns True? Why?

(a) Z = Not X Or Y = X Imp Y
(b) Z = X Imp Y = Not X Or Y
(c) Z = X Eqv Y = Not X XOr Y
(d) Z = Not X XOr Y = X Eqv Y

(a) assigns False if X and Y are both False. The others assign True no matter what X and Y are. However, these are not as straightforward as you might think.

Imp is the "logical implication" operator. X Imp Y means "If X is True then Y is True". It is always True unless X is True and Y is False.

Eqv is the "logical equivalence" operator. X Eqv Y is True if X and Y are both True or both False.

Why do we need Eqv -- isn't that the definition of "equals"? The difference is that if given numbers instead of Booleans, these do the same logic on corresponding pairs of bits.

If two logical expressions are always equal no matter what the values of their variables are, then they are logically identical. Such a relationship is called an "identity". These are clearly identities:

(a) Z = ((Not X) Or Y) = (X Imp Y)
(b) Z = (X Imp Y) = ((Not X) Or Y)
(c) Z = (X Eqv Y) = (Not (X XOr Y))
(d) Z = (Not (X XOr Y)) = (X Eqv Y)

No matter what Boolean values X and Y are, Z will always be True. The trick here though is the operator precedence. Just like 1 + 2 * 3 is 7, not 9, these guys all have precedence. To show the precedence with brackets, the expressions I actually asked about are equivalent to

(a) Z = (((Not X) Or (Y = X)) Imp Y)
(b) Z = (X Imp ((Y = (Not X)) Or Y))
(c) Z = (X Eqv ((Y = (Not X)) XOr Y))
(d) Z = (((Not X) XOr (Y = X)) Eqv Y)

It is much less clear now that the last three are identities. The moral of the story is to be very careful with operator precedence when using logical operators -- equals binds tighter than Or!

12) Consider this program:

s = "Eric read Æsop's Fables"
arr = Split(s, "e", -1, 1)

arr is an array with how many elements?

(a) 2
(b) 3
(c) 4
(d) 5

It's (c). The parts are "", "ric r", "ad Æsop's Fabl", "s"

The 1 means "go case insensitive". I agonized over whether or not Split should split on matches to ligature characters. It gave me grief because this does work:

print Instr(1, "xxxÆxxx", "e", 1)

That finds the "e" inside the ligature, so why shouldn't Split split on the ligature? Basically, we just decided that it was too weird to split a string in the middle of a ligature. I was never very happy with this decision, and if I had to do it again, I probably would decide the other way. But it's too late now!

Wow, that took longer than I expected. Coming up next, a short break from scripting.

VBScript Quiz Answers, Parts Nine and Ten

Eric Lippert — Wed, 30 Mar 2005 17:25:00 GMT

9) Which of these statements is syntactically legal? Why?

(a) Const Z = -+-+-+-10
(b) Const Y = 2 + 2
(c) Const X = .1e310
(d) Dim W(+10)

(a) is legal. The rest are illegal.

(a) is legal because you can put as many unary positive or negative operators on top of a constant as you want, so long as it's a numeric constant. (b) is illegal; as I've discussed before, we do not support "constant folding" in VBScript. (c) is illegal because it's outside the range of a float. (d) is illegal because, bizarrely enough, dimension lists must be just straight integers, no operators whatsoever.

Clearly VBScript is inconsistent insofar as how unary operators and constants work together. These inconsistencies are one those unfortunate corner cases that got missed when the parser was designed.

10) Which of these statements is syntactically legal? Why?

(a) If Blah Then Foo Else Bar
(b) If Blah Then If Foo Then Bar
(c) If Blah Then Foo Bar Else Bar Foo
(d) If Blah Then Foo Bar End If

(a) is illegal, the rest are legal.

The VBScript single-line If-statement parser is all screwed up. Again, some of these were bugs in the first version that we couldn't fix without breaking compatibility, and some are just plain bugs that never got fixed for no particular reason.

(b) is perfectly legal, nothing odd here.

(a) is legal in VB6 but not in VBScript because of a mistake. The parser sees IF ID THEN ID ELSE... and tries to parse "ID ELSE" as a statement. The statement processor (correctly) thinks that the id is a procedure call and it tries to parse the argument list. The argument list processor should detect the Else and figure oh, I'm in an If and the statement is done, but it doesn't figure that out, it just creates a syntax error. (c) is legal because in this case the argument list parser finds an argument list before the Else and doesn't freak out.

A number of people pointed out that having both Foo Bar and Bar Foo in the same program is a little weird. Yes, it is! But the question was what is syntactically legal, not what makes sense. And in fact, it is possible for this to work:

Class ABC
Public Default Sub DEF(X)
Msgbox X.Name
End Sub
Public Name
End Class
Set Foo = New ABC
Foo.Name = "Foo"
Set Bar = New ABC
Bar.Name = "Bar"
Blah = True
If Blah Then Foo Bar Else Bar Foo

(d) is another mistake. The VB6 parser does not allow this, but VBScript does. As with #8, we couldn't fix it when we found it because it would have broken existing ASP page and web pages. The End If is ignored.

Funny story: I actually fixed this one for a beta release of the engines and a certain influential news site was broken by the fix. Bizarrely enough they had a web page with this syntax in it. They said they would give the new version of IE a bad review if upgrading broke even a single one of their hundreds of thousands of pages! Backwards compatibility is very, very important in the scripting world. Of course we rolled the change back, and there's a comment in the code now cautioning future maintenance programmers to never change it.