Fabulous Adventures In Coding : COM Programming

Human sacrifice, dogs and cats living together, mass hysteria! and thread model errors!

Eric Lippert — Tue, 08 May 2007 02:22:00 GMT

Reader Shaka comments on my post about error messages that "catastrophic failure" really does take the cake as being a terrible error message.

I fondly remember the first time I saw "catastrophic failure" as an error message. I was an intern, running the build lab for Visual Basic for Applications, and I was trying to get the PowerPC cross-compiler working so that we could build PPC binaries on x86 hardware. (Note that this was in the early 1990's. The VS build lab is now an entire team of people who have dedicated circuits just for the cooling equipment for all their computers. We've come a long way from an intern, a closet, and six machines named after opponents of Godzilla (*).)

Anyway, the compiler had a bug which caused it to run out of stack while compiling a particular module and the error was "catastrophic failure". This amused me to no end at the time. I laughed out loud, literally. I mean, fatal, sure. That process is going down. But "catastrophic"? I expect catastrophes to at least result in my hard disk becoming unreadable. I want bridges collapsing and floods and mass hysteria!

I have therefore always been rather embarrassed that the error message you get when you call the script engine on the wrong thread is "catastrophic failure". That, unfortunately, is the somewhat breathless and overstated string associated with E_UNEXPECTED. Basically we want to say "you called us in a completely unexpected way which thoroughly violates the engine contract, please don't do that". But "catastrophic failure" is what you get. I wish now that we'd declared a new HRESULT just for "you've violated the engine contract". But it's far, far too late now.

(*) In the interests of total accuracy: the OLE Automation build machines were named after Godzilla's opponents: Mothra, Monster Zero, etc. The VBA build machines were named after species of marine mammals: sei, humpback, etc.

Chained user-defined explicit conversions in C#, Part Three

Eric Lippert — Fri, 20 Apr 2007 17:00:00 GMT

Jeroen Frijters knew the answer to my challenge of last time: how is it that Foo foo = new Foo(); can cause a runtime conversion failure? And how is it that Bar bar = (Bar)(new Baz()); can succeed even if there is no user-defined conversion or built-in implicit conversion between Baz and Bar?

The answer is that it is, as Stuart Ballard editorialized, “a goofy COM interop thing that the language should never have supported directly”.

In COM programming you never talk to a object directly, as a class. Rather, the object has a number of public interfaces; the client of a particular object asks the object for the interface it wants, and it talks to the object on that interface. COM provides a special function called CoCreateInstance which gets the ball rolling; it knows how to create an instance of a specific class and provides a pointer to a given interface on that object.

Suppose you need to interoperate with an existing COM object in C#. We could have made you do all the work you’d do in a C++ program: call CoCreateInstance, get the interface, blah blah blah. But that’s kind of gross and ugly. Instead, we take advantage of the fact that the vast majority of the time that you are talking to a COM object is via an interop library created with the type library importer. The type library importer knows from the information in the type library which interfaces are associated with which coclasses. The typical pattern of interop classes created by the type library importer would look something like this if we wrote out the code in C#:

using System.Runtime.InteropServices;
[ComImport]
[Guid("12345678-1234-1234-1234-123412341234")]
[CoClass(typeof(InteropClass))]
public interface IMyInterface
{ /* whatever */}
public class InteropClass : IMyInterface {/*whatever*/}

So the C# compiler knows that IMyInterface is associated with the coclass InteropClass. We therefore allow this:

IMyInterface x = new IMyInterface();

as is a syntactic sugar for

IMyInterface x = (IMyInterface)(new InteropClass());

So this is yet another place where we introduce an explicit cast on your behalf.

This means that it is legal to say

InteropClass x = (InteropClass)(new IMyInterface());

And it will not fail at run time, even though there is no user-defined conversion or built-in implicit conversion from IMyInterface to InteropClass!

Unfortunately, there’s a bit of a bug in the C# compiler. We assume two things. First, that the type library importer always generates this pattern correctly, and second, that no one other than the type library importer pulls shenanigans like this. The first assumption is probably good, but there’s nothing stopping anyone from pulling shens like:

The compiler does not check to see if InteropClass actually implements IMyInterface! Therefore this guaranteed-to-fail explicit cast gets through at compile time and fails at run time.

Obviously this is really gross. I don’t think we’ll get to fixing this in C# 3.0, but perhaps in a later version of the language we will tighten this up and make it illegal to have a coclass which does not actually implement the given interface.

Next time, one more place where poorly specified compiler behaviour causes us to insert an explicit cast.

Do Not Call IsBadFooPtr, Indeed

Eric Lippert — Wed, 27 Sep 2006 21:16:00 GMT

Here’s a story that I said a long time ago that I was going to tell you all, and then promptly forgot about it. Raymond Chen’s blog entry today reminded me of it, because this is the story of how I found out the hard way that IsBadFooPtr is bad, bad, bad. Those of you who have been following releases of the script engines incredibly carefully may remember this story, but hopefully we caused and then fixed the problem sufficiently quickly that the vast majority of customers never noticed.

But I’m getting ahead of myself.

One of the most basic and important rules of COM programming is that you never, ever, EVER change an interface without changing its GUID. Or, to put it another way, you simply never change an interface; if you need to, you create an entirely new interface.

You may have wondered why is COM so strict about this rule. It’s because when you write a C++ program that uses an interface, the compiler will generate code that depends upon the particular pattern of stack frame layouts for passing arguments remaining exactly the same even when the user installs a new version of the COM object on their machine. Because COM objects and their “clients” can be versioned independently, the GUID has to be associated with a particular “binary contract” describing how the two pieces of code communicate.

Now, the script engines always talk to scriptable objects via IDispatch or IDispatchEx, so if a scriptable object accidentally breaks its contract, the script engines don’t notice a bit. They let the dispatch layer take care of all those irksome details about stack frame layout, and the dispatch layer figures all this stuff out from the type library data in the object itself.

Perhaps you see where this is going.

The Replace method of the VBScript regular expression object originally took two “in” arguments, both strings, and returned a string. This was the IRegExp interface. Later we realized that we wanted to be able to pass things other than strings for the second argument, so we created a new IRegExp2 interface where the Replace method took a string and a variant, and returned a string. So far, so good.

Then we ported the script engines to the IA64 processor. During the port for some unknown reason one of the developers temporarily changed the second argument from a VARIANT to a VARIANT*, put a comment in the code saying that this needed to be changed back before it was checked in, checked it in, and, uh, we kinda shipped it like that.

We thoroughly violated the rules of COM, but none of our automated tests showed this, because of course, all of the scripting automated tests were written – you guessed it – in script. We had no tests which used the scriptable objects from early-bound code. (We do now!)

Unfortunately, some of our developer customers were using the VBScript regular expression object from C++ and other early bound languages, so as soon as their customers upgraded to the latest version of scripting, all hell broke loose. The caller was fulfilling their end of the contract by putting a sixteen byte variant on the stack, and the callee was suddenly expecting a four byte pointer to a variant, so it was both crashing and misaligning the stack.

We immediately started getting bug reports, of course. About half the bug reports said “please change this back ASAP, our customers are broken”, and the other half said “we have already shipped a patch of our code to our customers using the new stack layout, please do not change it back, ever, we do not want to ship another patch”.

Obviously we realized that we had badly screwed up and were now in a hideous bind. We came up with three options:

First, we could revert to the previous layout, and inconvenience the small number of customers who had already shipped patches to their customers. (And of course, inconvenience all of those people as well, who would have to apply a second patch in as many days.)

Second, we could keep the current layout and tell the rest of our affected customers that they needed to patch our mistake.

Third, we could revert to the previous layout and yet NOT inconvenience anyone.

Obviously the third option is best, but how to do it?

I ended up writing heuristics into the Replace method that examined the stack frame, and then rerouted the call to a special helper method if we detected a VARIANT* on the stack when we were expecting a VARIANT. The helper method did all the necessary magic to adjust the stack pointer, and so on, so that from the caller’s perspective, everything looked perfectly normal.

Now, remember, we expected that in the vast majority of cases, a VARIANT would be passed, not a VARIANT*. My heuristic detected which case we were in by extracting four bytes from the middle of the stack frame and calling IsBadReadPtr on it. If that was a valid pointer then I’d cast it to a VARIANT*, check to see whether the variant type field was a string (by far the most likely argument to Replace), and then call IsBadWritePtr on the bstrVal (because BSTR pointers are almost always writeable). There were a few other details to the heuristic, but the point is that in the vast majority of cases, IsBadReadPtr was expected to fail. Only in the rare cases where we were in an early-bound and third-party patched program should it succeed.

We shipped the fixed version with the heuristics out immediately.

At the time I did not realize how IsBadReadPtr was implemented. It actually puts a try-except around an attempt to read the pointer! If it gets an exception then it eats the exception and says yeah, it's bad. Essentially what I had done was inserted an almost-always-thrown exception into every call to Replace.

My goal of not inconveniencing anyone turned out to not quite work out as planned. The Active Server Pages team went nuts that afternoon. Their tests assume that all exceptions, whether handled or not, are flaws somewhere in ASP or scripting. Their test machines are all configured to track and report all first-chance exceptions, and suddenly every compatibility and stress test machine they had that happened to do a regular expression replace – that is, approximately all of them – suddenly started reporting hundreds of first-chance exceptions per minute when they upgraded to the latest released version.

So just when I thought that I could breathe easy again for a few minutes I started getting calls from the now highly vexed ASP team. I ended up writing my own versions of IsBadFooPtr which do the right thing – they call VirtualQuery to ask the operating system what the protection state of a given address is, rather than simply trying it out and throwing a first-chance exception on failure.

Long story short: don’t change an interface, don’t call IsBadFooPtr, and run your changes by all your important consumers before you ship a tricky fix!

Checking For Script Syntax Errors, This Time With Code

Eric Lippert — Wed, 12 Oct 2005 19:37:00 GMT

A number of people asked me to clarify yesterday's entry. Rather than try to talk you through it, I think the code is straightforward enough to speak for itself. Here's a little skeleton that I just whipped up.

#include <stdio.h>
#include <activscp.h>
#include <new>

const GUID CLSID_VBScript = {0xb54f3741, 0x5b07, 0x11cf, {0xa4, 0xb0, 0x00, 0xaa, 0x00, 0x4a, 0x55, 0xe8}};
const GUID CLSID_JScript = {0xf414c260, 0x6ac0, 0x11cf, {0xb6, 0xd1, 0x00, 0xaa, 0x00, 0xbb, 0xbb, 0x58}};

class MySite : public IActiveScriptSite {
private:

ULONG m_cref;
virtual ~MySite() {}

public:

  MySite() {
   this->m_cref = 1;
  }

STDMETHOD_(ULONG, AddRef)() {
return ++this->m_cref;
}

STDMETHOD_(ULONG,Release)() {
    --this->m_cref;
  if (this->m_cref == 0) {
     delete this;
      return 0;
  }
    return this->m_cref;
  }

  STDMETHOD(QueryInterface)(REFIID iid, void ** ppv) {
   if (ppv == NULL)
      return E_POINTER;
    if (IsEqualIID(iid, IID_IUnknown))
      *ppv = (IUnknown*)this;
    else if (IsEqualIID(iid, IID_IActiveScriptSite))
   *ppv = (IActiveScriptSite*)this;
    else {
      *ppv = NULL;
      return E_NOINTERFACE;
    }
    this->AddRef();
    return S_OK;
}

STDMETHOD(GetLCID)(LCID * plcid) {
return E_NOTIMPL;
}

  STDMETHOD(GetItemInfo)(
  LPCOLESTR pstrName,
   DWORD dwReturnMask,
    IUnknown ** ppunkItem,
    ITypeInfo ** ppti) {
    return E_NOTIMPL;
  }

STDMETHOD(GetDocVersionString)(BSTR * pbstrVersion) {
return E_NOTIMPL;
}

STDMETHOD(OnScriptTerminate)(
   const VARIANT * pvarResult,
   const EXCEPINFO * pexcepinfo) {
    return S_OK;
  }

STDMETHOD(OnStateChange)(SCRIPTSTATE state) {
return S_OK;
}

  STDMETHOD(OnEnterScript)() {
   return S_OK;
}

STDMETHOD(OnLeaveScript)() {
    return S_OK;
  }

STDMETHOD(OnScriptError)(IActiveScriptError * perror) {
    EXCEPINFO excepinfo;
    LONG column = 0;
    ULONG line = 0;
    DWORD context = 0;
    BSTR bstrLine = NULL;
    memset(&excepinfo, 0x00, sizeof excepinfo);
   perror->GetExceptionInfo(&excepinfo);
    if (excepinfo.pfnDeferredFillIn != NULL)
     excepinfo.pfnDeferredFillIn(&excepinfo);
    perror->GetSourceLineText(&bstrLine);
   perror->GetSourcePosition(&context, &line, &column);

   wprintf(L"Error on line %ld column %ld\n", line, column);
   if (bstrLine != NULL)
      wprintf(L"Line: %s\n", bstrLine);
    if (excepinfo.bstrDescription != NULL)
     wprintf(L"Description: %s\n", excepinfo.bstrDescription);
    if (excepinfo.bstrSource != NULL)
     wprintf(L"Source: %s\n", excepinfo.bstrSource);
    if (excepinfo.bstrHelpFile != NULL)
     wprintf(L"Help: %s\n", excepinfo.bstrHelpFile);

    SysFreeString(bstrLine);
    SysFreeString(excepinfo.bstrDescription);
    SysFreeString(excepinfo.bstrSource);
    SysFreeString(excepinfo.bstrHelpFile);

return S_OK;
}
};

void main() {
HRESULT hr = S_OK;
HRESULT hrInit;
IClassFactory * pfactory = NULL;
IActiveScript * pscript = NULL;
IActiveScriptParse * pparse = NULL;
IActiveScriptSite * psite = NULL;

hr = hrInit = OleInitialize(NULL);
if (FAILED(hr))
goto LError;

hr = CoGetClassObject(CLSID_VBScript, CLSCTX_SERVER, NULL,
IID_IClassFactory, (void**)&pfactory);
if (FAILED(hr))
goto LError;

hr = pfactory->CreateInstance(NULL, IID_IActiveScript, (void**)&pscript);
if (FAILED(hr))
goto LError;

psite = new(std::nothrow) MySite();
if (psite == NULL) {
hr = E_OUTOFMEMORY;
goto LError;
}

hr = pscript->SetScriptSite(psite);
if (FAILED(hr))
goto LError;

hr = pscript->QueryInterface(IID_IActiveScriptParse, (void**)&pparse);
if (FAILED(hr))
goto LError;

hr = pparse->ParseScriptText(L"Function Foo \n Foo = 123 \n End Funtcion \n",
NULL, NULL, NULL, 0, 1, 0, NULL, NULL);

if (FAILED(hr))
goto LError;

LError:

if (FAILED(hr))
printf("%0x\n", hr);

if (pparse != NULL)
pparse->Release();

if (psite != NULL)
psite->Release();

if (pscript != NULL) {
    pscript->Close();
   pscript->Release();
  }

if (pfactory != NULL)
pfactory->Release();

if (SUCCEEDED(hrInit))
OleUninitialize();
}

As you would expect, this program prints out the information about the error, and ParseScriptText returns SCRIPT_E_REPORTED to indicate that there was an error but it has already been reported. Had there been no error, the script would not have actually run; the engine is not started, just initialized.

Error on line 3 column 5
Line: End Funtcion
Description: Expected 'Function'
Source: Microsoft VBScript compilation error
80020101

Implementing Event Handling, Part Two

Eric Lippert — Wed, 21 Sep 2005 21:51:00 GMT

It's been an insanely busy month for me, between having multiple out-of-town guests, throwing a party for the people who couldn't make it to the wedding, and oh yeah, getting up to speed on the C# compiler and trying to understand the implications that LINQ features are going to have on the current implementation. Not much time for blogging. Hopefully October will be a little bit more under control.

Anyway, I was talking about early-bound event binding. Basically the idea is that the source and the sink agree upon an interface that the source can call on the sink whenever the source wishes to fire an event. Before we get into that more, I want to talk briefly about early- and late-bound code.

At an implementation level, what is the difference between early- and late- bound code? In the early-bound world, a particular function on an interface gets called when the caller dereferences the callee's virtual function table and transfers control directly to the function by changing the instruction pointer on the chip. In the late-bound world, the caller calls IDispatch::Invoke and passes in a magic number that tells the callee what function it should be calling, and the callee is then responsible for dispatching the function appropriately. (Hence the name "IDispatch".)

The script engines use IDispatch all the time to call out to functions on objects. When calling an object late-bound, you don't know until you actually try the call whether or not it is going to succeed, because you do not know the interface the object implements. So you call GetIdsOfNames to get the magic number associated with a particular function, Invoke on that magic number, and let the callee sort it out. In this situation, the caller (the script engine) knows the name of the function it wants to call, and the callee (the object) can map the name to the appropriate identifier.

Now consider how this works if IDispatch is the interface over which the source (the object) is calling back the sink (the script host). This appears to be the same situation, but in fact it is completely different. It appears the same because from the perspective of how COM actually manages all the calls, its exactly the same. IDispatch is an interface like any other, and the source can call the sink's IDispatch to its heart's content if that's the interface that they agree to talk over.

But look at it from the point of view of the sink: the caller knows what the magic number means and the callee does not, but it's the callee who is being asked to do the dispatching! You're sitting there sinking events of who knows what object, and every now and then you get a call on IDispatch::Invoke with some unknown dispid. What the heck are you supposed to do with that? In the late-bound world, instead of getting a nice direct call on "Tick()... Tick()... Tick()..." you're getting "12... 12... 12..." and you have no idea what "12" means.

Let's tie this in to scripting. Suppose you're in Windows Script Host and you do something like:

Sub Timer_Tick()
'whatever
End Sub
Set Timer = CreateObject("mytimer")
WScript.ConnectObject Timer, "Timer_"
Timer.Start 10
WScript.Sleep 1000

There are many IDispatch objects here and we'll look at three of them. First, there's the source, that is, the timer object. Then there's the sink, owned by the host and created when the source is connected to it. Finally there is the script engine itself, upon which the host can dispatch calls to global functions such as the event handler.

ConnectObject is given a source object. It has no idea what early-bound outgoing interface is on that object, and it certainly doesn't have an implementation of such an interface even if there is one, so it is going to have to do a late bound sink. It creates an object to act as the sink, gets an IDispatch-enabled connection point from the source, and advises it.

Now the script engine calls Start (via IDispatch on the source) and goes to sleep.

Pretty soon the timer invokes the sink, passing in the dispatch identifier for Tick.

The sink needs to know which sub to call in the script engine.

Fortunately, it has an IDispatch pointer to the source, and a dispatch identifier.

Unfortunately, though IDispatch provides a function which maps from name to id, it provides no function that maps the other way.

Fortunately, IDispatch does provide a function that enables the host to obtain a type information structure which contains a list of all the methods and what their dispatch identifiers are. Therefore we can search all the methods in the type info and check to see which has the desired dispatch identifier. That then gives us the method name, so we know which event handler function to dispatch on the script engine.

Unfortunately there is a major design flaw in IDispatch -- it gives you the type info for the incoming interface -- the ITimer. But there is no way to take the ITypeInfo for ITimer and say "give me the type info for the class as a whole so that I can obtain the type info for the outgoing interface".

Fortunately, IProvideClassInfo was invented. IPCI gives you back the "root" type info for an object, from which you can obtain a type info for the default outgoing interface, from which you can map the dispatch identifier to the name.

Unfortunately, many objects do not implement IProvideClassInfo, and therefore cannot have their events hooked up by ConnectObject.

(I feel a little bit like the ending of Dr. Strangelove here. Fortunately, they stop General Ripper in time. Unfortunately, the secret code died with him. Fortunately they figure out the code. Unfortunately, the radio is broken. Fortunately, the bomb bay doors are stuck. Unfortunately, they fix them, and the Doomsday Device destroys the world. Bummer.)

The moral of this story is simple: for late bound events to work, at some point the class of the source must be known. And therefore:

"automagic event binding" in IE only works on "named items" present in the script engine. (The script engines require that the host provide a coclass type info for named items, and build the sinks very early on, before other code runs.)
WScript.CreateObject can always hook up events. The class type info is known because CreateObject had to create an instance of the class.
WScript.ConnectObject can hook up events only if the object implements IProvideClassInfo

I hope that clears up any confusion about what IProvideClassInfo is for!

Implementing Event Handling, Part One

Eric Lippert — Sat, 10 Sep 2005 01:35:00 GMT

Back in February I posted a bit about how script hosts such as Windows Script Host dynamically hook up event sources (objects) to event sinks (chunks of script that run when the event fires.) It has become apparent from some questions I've received recently that it would be helpful to have a more detailed explanation of how event handling works in COM, and how that applies to late-bound scripting languages.

We'll start in the early-bound world and then move into the scripting world next week.

Suppose you've got an object Timer that has methods Start and Stop, and event Tick. The methods are straightforward. We define an interface that has these methods on it:

interface ITimer : public IUnknown
{
public:
virtual void Start(int interval) = NULL;
virtual void Stop() = NULL;
}

When users want to call those methods, they just get a pointer to the IUnknown of the object, QueryInterface it for ITimer, and then call the appropriate method on the virtual function table.

But the user doesn't just want to call the timer. Not much point in that! They want the timer to inform some other piece of code whenever a tick event occurs. Somehow the sink - the listener function - is going to have to be called. This means that the timer code has to be running and it has to call some arbitrary code that it knows nothing about.

How the timer code manages to run itself is a topic for another day; today we'll just worry about getting the call from the source to the sink. The sink is going to have to somehow register itself with the source, and they'll negotiate what interface the source will call on the sink when the event occurs. In this case the sink wants to know when a tick happens, so the sink should implement something like

interface ITick : public IUnknown
{
public:
virtual void Tick(ITimer * pSource) = NULL;
}

OK, so suppose we have a source that implements ITimer and a sink that implements ITick. Somehow they have to get hooked up. (Error checking removed.)

pSource->QueryInterface(IID_IConnectionPointContainer, &pContainer);
pContainer->FindConnectionPoint(IID_ITick, &pConnection);
pConnection->Advise(pSink, &cookie);

The sink says to the source "do you contain connection points that can call me back on ITick?" The source says "sure, here's an object that you can register yourself with that will call you back". The sink then says "Super, here's a pointer back to me, and please give me a 'cookie' number that I can use to uniquely identify this relationship so that I can shut it down later."

The source now has a pointer to the sink, so when a tick event occurs, the source can call the sink.

Next time we'll explore what happens in the scripting world where the sink only speaks IDispatch.

VBScript Default Property Semantics

Eric Lippert — Tue, 30 Aug 2005 20:15:00 GMT

Here’s a question I recently got about VBScript, where by "recently" I mean August 28th, 2003. This code works just fine:

Set myObject = CreateObject("myObject")
myObject.myName = "Eric"
WScript.Echo myObject ' myName is the default property, so prints "Eric"

But myObject = "Robert" doesn't set the default property, it sets the variable to the string. Why does reading work but writing fail? This works in VB6, why not VBScript?

In a strongly typed language such as VB6 the compiler can look at the compile-time types of the left and right sides of an assignment and determine that the type of the left hand side is an object, the right hand side is a string, and the object has a default property which is a string. The VB6 compiler can then generate the appropriate code to assign the string to the default property.

But what if you wrote a VB6 program using only variants? When the compiler sees foo = bar typed as variants it cannot tell whether this means "set the default property of foo to bar" or "set foo to bar". The VB compiler chooses the latter every time. VBScript is a weakly typed subset of VB -- in VBScript, everything is a variant. So in VBScript, all assignments are treated as though the value is actually being assigned to the variable, not the default property. Therefore this is by design - this is for compatibility with VB6's behaviour. Had we written the same program in weakly-typed VB6, we'd get the same result.

I hear you exclaiming "The fact that a difference in available type information leads to a difference in run-time behaviour violates a basic principle of programming language design! Namely, the principle that late-bound calls have exactly the same semantics as early-bound calls!"

Indeed, as I've mentioned before, VB6 and VBScript violate this principle in several places. Default properties were, in my opinion, a bad idea all around, for this and other reasons. They make the language harder to parse and hence harder to understand. In particular, parameterless default properties make very little sense in VBScript. However, we are stuck with them now.

However, I should call out that there are some things we can do at runtime. Suppose blah is an object with a property fred that is an object, and fred has a default property which is a string. If you say foo = blah.fred then foo is assigned the default value of fred even though we lack the compile-time type information. foo isn't set to the object fred. Of course, in this case we know to fetch the default property at runtime because we know the runtime type of fred and also know that there is no Set keyword. These two facts are sufficient to cause the runtime engine to always fetch the default property.

Or suppose you have an object Baz with a property Table, Table has a default parameterized property Item which returns an object that has a string property Blah: Then this works just fine:

x = Baz.Table(1).Blah

But why? In this case we do not have enough compile-time information to determine that we really should call Baz.Table.Item(1).Blah. The script engine has every reason to believe that Table is a function or parameterized property of Baz. This problem is solved by pushing it off to the implementation! The rule for implementers of IDispatch::Invoke is if all of the following are true:

the caller invokes a property
the caller passes an argument list
the property does not actually take an argument list
that property returns an object
that object has a default property
that default property takes an argument list

then invoke the default property with the argument list. Strange but true.

Perhaps unsurprisingly, not very many people know about that rule! This is yet another reason to never, ever write your own implementation of IDispatch::Invoke. It also leads to some mind-numbingly complex code in the VBScript IntelliSense engine that Visual Studio uses. Making default property IntelliSense work with such a dynamically typed language was a piece of work, lemme tell ya. As I mentioned earlier, there are in fact some odd corner cases that are slightly different between VB6 and VBScript IntelliSense. I also talked a bit about left-hand-side default property semantics earlier.

Caching Dispatch Identifiers Is A Bad Idea

Eric Lippert — Thu, 25 Aug 2005 01:22:00 GMT

About two years ago I wrote a bit about when it was legal to cache a dispatch identifier so that you didn’t have to look it up a second time. I was reminded of this today because the scripting sustaining engineering team was working on a bug that involved a customer incorrectly caching a dispatch identifier. I’d like to take this opportunity to expand a bit on the rules I mentioned two years ago.

Suppose you've got two script engines that are totally identical – same script code added in the same order, similarly the same named items – running on two different threads. I’m sure you can see why that would be useful; think of an ASP server serving up two pages on two threads, for example. The customer needed to call a particular script function by invoking each script engine’s global dispatch. They were getting the dispatch identifier once, caching it and using it for calling both engines. Depending on what build of the script engine they were using, that either worked fine or died horribly.

Unfortunately it turns out that dying horribly is both legal and expected. A given dispatch identifier can only be cached in the context of a particular dispatch object. Two identical dispatch objects with identical functions are allowed to give different dispatch identifiers for a given method!

The script engines have changed their algorithm for how dispatch identifiers are generated several times over the years. In some builds of the script engines, dispatch identifiers are generated sequentially, so that the first function added is one, the second is two, and so on. In those builds, two identical engines that have had the same functions added in the same order will have identical dispatch identifiers for a given name. In other builds of the script engines, the dispatch identifiers are globally unique, so that two engines with two functions will always have different dispatch identifiers even if everything is identical otherwise.

Unfortunately the customer built and tested their code against the former kind of engine and then upgraded to a more recent version using the latter algorithm, which broke them. I regret the problem -- I'm the guy who screwed around with the algorithm -- but still, you've got to follow the rules. Calling GetIdsOfNames is almost always very fast compared to Invoke, so please don’t cache dispatch identifiers for performance reasons. We don’t do so in the script engines, as I mentioned two years ago, because the trouble you get into when you do it wrong is not worth the time you save.

While I’m on the subject, a quick word of advice for implementers too: If you are implementing IDispatch on an expando object it is very tempting to say "I’ll use a pointer to my function object as the dispatch identifier for that function". Just cast the pointer to int and you’ve guaranteed uniqueness and have an easy way to map back to the function when invoked, right? Please do not do this. First off, it’s not portable to 64 bit machines. Second, it’s dangerous – a hostile or broken caller can make you invoke pointers to garbage. Third, on processes with 3GB of user-accessible memory, pointers can be cast to negative integers and negative integers are illegal dispatch ids. Come up with some safer way to guarantee uniqueness of your dispatch ids, such as maintaining a global counter and lookup table.

How Do Script Engines Implement Object Identity?

Eric Lippert — Tue, 26 Apr 2005 21:49:00 GMT

I've talked a few times in this blog about the semantics of the equality operators in various languages. (Such as here, and here.) Recently a reader asked me how JScript implements object identity. That is, given two objects, how do we know if they are "the same object" or not?

For the remainder of this discussion I'm going to assume that we've got two non-null objects that we're comparing for identity. The details of how objects are compared to strings, numbers, etc, will have to wait for another blog entry.

Note also that in VBScript, the comparison operator for objects is Is, not =. Also, == and === in JScript behave identically if both arguments are objects; essentially the implementation of Is, == and === are all the same as far as object comparison goes.

Behind the scenes, everything in a VBScript/JScript program is represented as a variant. Most of the time, variants containing objects will have their variant type field set to VT_DISPATCH, and their pdispVal field set with a pointer to the object. (Occasionally we'll have a VT_UNKNOWN with a punkVal field instead, but lets ignore that corner case for now.) Suppose we want to compare two such objects for equality. We could pass the pdispVal of each to a helper function and compare the pointers.

bool AreObjectsEqual(IDispatch * pdisp1, IDispatch * pdisp2)
{
if (pdisp1 == pdisp2)
return true;
return false;
}

And we're done. Wow, that was easy.

Except that it's wrong. In VBScript, those objects could be two different dispatch pointers to the same object because VBScript supports non-default dispatches. JScript does not support non-default dispatches, but in JScript there are also situations in which you can have two different dispatch pointers to the same object; we'll come to that later.

If its not clear why the two pointers to the same object could be numerically different, read Raymond's article on how COM objects are laid out in memory, and you'll see why. There can be many dispatch vtables.

How can we ever determine when two COM objects are the same? We have to rely upon one of the most important rules of COM: if an object multiply inherits IUnknown then calling QueryInterface for IUnknown on any of those implementations must always give back the same pointer. As you can see in Raymond's example of multiple inheritance, there are two IUnknown vtables. It is illegal for each of them to return a pointer to itself when QI'd for IUnknown; the implementation must consistently pick one of them every time.

So now it's pretty clear what we must do:

bool AreObjectsEqual(IDispatch * pdisp1, IDispatch * pdisp2)
{
IUnknown * punk1 = NULL;
IUnknown * punk2 = NULL;
if (pdisp1 == pdisp2)
    return true;
if (pdisp1 == NULL || pdisp2 == NULL)
    return false;
pdisp1->QueryInterface(IID_IUnknown, (void**)&punk1);
pdisp2->QueryInterface(IID_IUnknown, (void**)&punk2);
// This should never fail, but better safe than sorry.
if (punk1 != NULL)
    punk1->Release();
if (punk2 != NULL)
    punk2->Release();
// We're not dereferencing the pointers, so it is OK to use the
  // locals even after their release.
if (punk1 == punk2)
    return true;
return false;
}

Unfortunately, that's not the whole story either.

For security reasons, Internet Explorer sometimes creates proxy objects -- little objects that detect if they are being used safely, and if they are, then they forward the call to the real implementation, on a separate object. You can run into situations in IE where you have two proxy objects to the same "real" object -- as far as COM is concerned, all three are different objects, but as far as IE is concerned, that's an implementation detail of the security system which we would like to hide from the script users.

To solve this problem, IE implements a special interface that knows when two IUnknown objects are both proxies for the same object. The script engines actually do something like this:

bool AreObjectsEqual(IDispatch * pdisp1, IDispatch * pdisp2) {
IUnknown * punk1 = NULL;
IUnknown * punk2 = NULL;
IObjectIdentity * pObjectIdentity;
bool fRet = false;
HRESULT hr;
if (pdisp1 == pdisp2)
    fRet = true;
else if (pdisp1 != NULL && pdisp2 != NULL) {
    // This must always succeed.
    pdisp1->QueryInterface(IID_IUnknown, (void**)&punk1);
    pdisp2->QueryInterface(IID_IUnknown, (void**)&punk2);
    if (punk1 == punk2)
      fRet = true;
    else if (punk1 != NULL && punk2 != NULL) { // This should never be false.
      hr = punk1->QueryInterface(IID_IObjectIdentity, (void **)&pObjectIdentity);
      if (SUCCEEDED(hr)) && pObjectIdentity != NULL) {
        hr = pObjectIdentity->IsEqualObject(punk2);
        if (hr == S_OK)
          fRet = true;
      }
    }
}
if (pObjectIdentity != NULL)
    pObjectIdentity->Release();
if (punk1 != NULL)
    punk1->Release();
if (punk2 != NULL)
    punk2->Release();
return fRet;
}

Which is a heck of a lot of code just to determine if two pointers are equal, but that's the tax you pay for object-oriented programming. Of course, you don't need to do any of this rigamarole with IObjectIdentity unless you're writing code that messes around with IE objects.

Why does WScript.ConnectObject not always work?

Eric Lippert — Tue, 15 Feb 2005 19:54:00 GMT

I've had a tiny handful of entries in my VBScript quiz. I know from my logs that 2600+ people have read it. Come on people, there are fabulous prizes at stake here! Go for it!

I'm going to keep the contest open until Friday morning, at which point I'll start posting answers and analysis.

****************************************************

Earlier today someone asked me why it is that sometimes WScript.ConnectObject does not connect an object to an event sink even if the object sources events. That reminded me that back in June I said that I'd explain this at some point, and I never did.

WScript.ConnectObject only works if the object implements IProvideClassInfo or IProvideMultipleClassInfo. Not all objects do.

Why?

An object can implement multiple interfaces both incoming and outgoing. Incoming interfaces are the interfaces you call an object on. Outgoing interfaces are interfaces that they call you on. That's how events work: you tell the object that you want a particular method to be called on a particular outgoing interface when a particular event occurs.

Each interface supported by an object, whether incoming or outgoing, can have its own interface typeinfo describing it. For a given object all these interfaces are "summarized" by one "parent" type info called the "coclass type info". Once you have the coclass type info, you can figure out everything about the object's incoming and outgoing interfaces.

Clearly, in order to build an event sink for an object, you need to know what events the event source is going to source! You need the outgoing interface typeinfo, or, equivalently, the coclass typeinfo.

The flaw in the design of the automation interfaces is this: given an IDispatch object, there is no way to discover what its coclass typeinfo is! You can get a type info for the dispatch interface -- that is, the incoming call interface -- by calling GetTypeInfo. But there is no way to get the coclass type info, and therefore no way to determine what the outgoing interfaces are on the object, and therefore no way to bind its events.

When that omission was realized, IProvideClassInfo was invented. (And for aggregate objects, which may have aggregated multiple coclasses, IProvideMultipleClassInfo was invented.) IProvideClassInfo returns the coclass type info. That's why WScript.ConnectObject requires IProvideClassInfo -- it just has the object and knows nothing about its class.

You may then wonder how it is that WScript.CreateObject does event binding on objects that do not implement IProvideClassInfo. That's easy -- WScript.CreateObject is given enough information to find the coclass type info. WScript.CreateInstance is passed the progid. It uses that to look up the class id in the registry. It obtains the incoming interface typeinfo from the newly-created dispatch object, obtains the typelibrary from the typeinfo, and searchs the typelibrary for a coclass type info which matches the class id.

This also explains how it is that Visual Basic can do event binding on objects which do not implement IProvideClassInfo. VB has compile-time information about the progid of the object and can look up the outgoing interface in the same way.

We considered creating another version of WScript.ConnectObject which took a classid or progid, but we decided that it wouldn't be useful in very many situations, and would be confusing and hard to document well.

VBScript Terminators, Part Two

Eric Lippert — Thu, 30 Dec 2004 03:47:00 GMT

You guys came up with good answers to three of my four questions, which is about what I expected; question 2 was pretty hard.

To sum up:

QUESTION #1: Why does the termination logic go terminate, terminate, terminate, clear, clear, clear, instead of terminate and clear, terminate and clear, terminate and clear?

Because if the second object to be terminated has a terminator that accesses a property of the first object to be terminated and cleared, it will fail, which seems bad. We want to run all the terminators while the objects are still in good shape, and then blow them all away.

QUESTION #3: Why do we want to ensure that terminators don’t run twice?

Imagine a terminator which writes a "logging complete" message to a log file; you don't want it to run twice.

QUESTION #4: Why do we run the garbage collector at the end of every statement, instead of only at the end of every procedure/global block?

Because, though local variables are not going to go out of scope, temporary anonymous slots are. Consider

x = MyFunc("a" + s, new Foo)

That's going to allocate one temporary slot for the string and one for the object. The temporary string can be cleared whenever, but the temporary object should be released ASAP so that it's terminator runs ASAP. We therefore clean up all temporaries after every statement.

That leaves

QUESTION #2: In what scenario can a bad implementation crash the process and/or terminate the object twice?

When I first wrote the termination logic, the release code looked like this:

ULONG VBSClassInstance::Release(){
--this->m_cRef;
if (this->m_cRef == 0)
{
    this->RunTerminator();
    delete this;
    return 0;
}
return this->m_cRef;
}

Looks like a perfectly straightforward implementation, right? But what if some bozo does this?

Dim Global
Class Foo
Private Sub Class_Terminate()
Set Global = Me
End Sub
End Class
Sub Blah
Dim Local
Set Local = New Foo
End Sub
Blah
Set Global = Nothing

When Local goes out of scope, the terminator runs and sets the Global variable to the object which has just been terminated! Therefore it must live. We must write the VBSClassInstance::RunTerminator method to ensure that the terminator doesn't run twice, but that's the least of our problems. Look at the implementation of Release above carefully. When the terminator runs, the ref count will go back up to one, but we still delete the object! The script engine now has a global variable containing a pointer to deleted memory; this will crash the process, corrupt the heap, who knows what?

OK, so what if we go

ULONG VBSClassInstance::Release(){
--this->m_cRef;
if (this->m_cRef == 0)
{
    this->RunTerminator();
}
  if (this->m_cRef == 0)
{
    delete this;
    return 0;
}
return this->m_cRef;
}

Is that better? Well, sure, it's better, but it's still wrong. Forget globals; consider this:

Private Sub Class_Terminate()
Dim TermLocal
Set TermLocal = Me
End Sub

Now what happens? The "final" release when Local goes out of scope sets the ref count to zero and calls the terminator. The terminator increases the ref count to one by assigning it. Then when TermLocal goes out of scope, it calls Release on the object. We've now got a re-entrant Release method! The "inner" Release detects that the ref count has gone to zero and deletes the object. Then the "outer" Release reads from the now-invalid "this" pointer. Assuming that doesn't crash, it then probably corrupts the heap by releasing the object a second time.

The correct logic looks something like this:

ULONG VBSClassInstance::Release(){
--this->m_cRef;
if (this->m_cRef == 0)
{
    ++this->m_cRef;
    // protects against re-entrant final release
    this->RunTerminator();
    --this->m_cRef;
    if (this->m_cRef == 0)
    {
      delete this;
      return 0;
    }
  }
return this->m_cRef;
}

Writing correct shutdown logic is surprisingly tricky!

T4: VBScript and the Terminator

Eric Lippert — Wed, 22 Dec 2004 23:42:00 GMT

I had some free time on the flight to Canada and my laptop with me, so maybe I will do a little blogging on my vacation after all.

A week ago or so a reader asked me to talk a bit about how class termination works in VBScript. Let’s start by looking at a simple case:

Class Foo
Public Name
Public Other
Private Sub Class_Terminate
MsgBox Name & ": Goodbye, world!"
End Sub
End Class

As you can see, the class has an “event handler” which is executed when an instance is just about to be released. (I put scare quotes on there because, though it has the form of an event handler, in fact the underlying mechanism that runs the terminator is completely distinct from the other event handling code in VBScript.)

JScript, as you may know, has a nondeterministic mark-and-sweep garbage collector. It’s “nondeterministic” because it runs pretty much when it feels like it. There is a scheduling algorithm, but from the perspective of the program, it’s pretty much random; it’s quite tricky to predict when it’s going to run. And it’s “mark and sweep” because every collection, it runs through every script-allocated memory block, checks if it is still accessible by the script, and if it isn’t, the memory is freed. This makes it mostly immune to the “circular reference” problem, where two or more otherwise dead objects reference each other.

VBScript’s garbage collector is completely different. It runs at the end of every statement and procedure, and does not do a search of all memory. Rather, it keeps track of everything allocated in the statement or procedure; if anything has gone out of scope, it frees it immediately. We can see this in action by watching terminators run.

Let’s consider a simple example.

Sub Bar
   Dim X
   Set X = New Foo
   X.Name = "X"
End Sub
MsgBox "Starting Bar"
Bar
MsgBox "Done Bar"

outputs, as you’d expect:

Starting Bar
X: Goodbye, world!
Done Bar

As soon as Bar’s End Sub runs, X goes out of scope. Since that’s its last reference, the terminator runs.

It’s that “last reference” part that’s the rub. Objects are reference counted in COM – every object keeps track of how many other objects are keeping it alive, and when that drops to zero, the object deletes itself.

The trouble is, what if two objects are each keeping a ref on each other, but are otherwise dead? The “circular reference” problem wasn’t so much a problem before we added classes to VBScript. After all, without the ability to define new objects in script, the only objects you really need to worry about are external ActiveX objects – and not even JScript can break circular references involving ActiveX objects, because the ActiveX objects know nothing about JScript’s garbage collector, and vice versa.

But with objects in VBScript, suddenly it becomes quite bad. You can very easily write scripts that consume memory in the form of objects that reference each other circularly, and they won’t go away when they go out of scope:

Sub XYZ
Dim Alpha, Bravo
Set Alpha = New Foo
Alpha.Name = "Alpha"
Set Bravo = New Foo
Bravo.Name = "Bravo"
Set Alpha.Other = Bravo
Set Bravo.Other = Alpha
End Sub

MsgBox "Starting XYZ"
XYZ
MsgBox "Done XYZ"

outputs

Starting XYZ
Done XYZ

Uh oh. Neither of those guys was terminated when their variables went out of scope. All the memory associated with them is still reserved by the heap. But they’re not reachable now. There are no variables that reference them. There’s nothing the script programmer can do about it now.

It would be unfortunate indeed if such circular references really did live until the end of the process, when the whole heap is cleaned up, both because we’d leak memory, and because those terminators might be doing something interesting.

Fortunately, we can do slightly better than that. If you run the code above in Windows Script Host, you’ll see that the terminators run as soon as the script ends; they are cleaned up eventually. In IE, the terminators run when the page is torn down. And in ASP, the terminators run when the page is served. (Though of course in ASP you don’t want to be putting up message boxes, or accessing the ASP object model – the page is just about to get served, don’t try to mess with it!)

How was this implemented? And how does this solve the memory leak problem?

It’s pretty straightforward. Every time a new instance of a VBScript class is created, it adds itself to a special list maintained by the engine, and whenever one is destroyed, it removes itself from the list. When the VBScript engine itself is closed by the host, the engine runs down the list and runs all the terminators of all the VBScript objects left on the list. Then it runs down the list a second time and clears every field of the object. This breaks the circular reference. Better late than never.

In Windows Script Host, this really isn’t much help because the engine isn’t closed until the process is about to be torn down anyway. But in IE and even more importantly, ASP, script engines are closed much more aggressively. It was very important that we not add a new feature to the language that made it really easy to write ASP pages which leaked memory and forced you to reboot your server frequently.

There are some wrinkles that we ran into when implementing the shutdown logic. Let’s see if you can deduce what they are; I’ll give the answers in my next post.

QUESTION #1: When I first wrote the VBScript class code, that’s not exactly the shutdown sequence I wrote. The original logic ran down the list of objects once, going terminate, clear, terminate, clear, terminate, clear. Soon after that I rewrote the shutdown sequence to the present terminate, terminate, terminate, clear, clear, clear. Why did I make the change to a two-pass implementation? What was wrong with the original one-pass implementation?

QUESTION #2: The code that implements VBScript classes checks to see if the instance has been terminated already before it calls the Class_Terminate method. It should be clear from the foregoing why that is – if an object is in a circular reference and manages to survive until the engine shuts down, then the object will probably be terminated while there are still references held on it. The later “clear all fields” phase of the shutdown sequence could then release all the references, triggering the “we’ve just released the last reference, so call the terminator” logic. We therefore have to protect against the terminator running twice.

There’s another way that an object can be terminated twice that we protect against. In this scenario, not only can the object be terminated twice, but an incorrect implementation of the object deletion code can crash the process. What is it?

Hint: the scenario does not involve circular references. It has just a single object being terminated because it is going out of scope and has no other references.

QUESTION #3: Speaking of which, why do we want to ensure that terminators don’t run twice?

QUESTION #4: This is not a question about termination per se, but there is a termination angle in the answer. Why do we run the garbage collector at the end of every statement, instead of only at the end of every procedure/global block? After all, no local variable is going to go out of scope until the end of its procedure! Doesn’t that seem like a lot of work for no possible gain?

"I'll be back" in the new year -- have a good festive holiday season everyone!

Why do built-in JScript functions not appear in the typeinfo?

Eric Lippert — Mon, 06 Dec 2004 21:08:00 GMT

A reader sent me an email over the weekend asking about some odd behaviour in the guts of the JScript engine. Unfortunately, they didn't include an email address that worked; I just got a bounce message back, Ben. Therefore I'll just answer the question here.

To briefly describe the problem: the user fetches the JScript "Math" object from the script engine by calling ParseScriptText to evaluate the expression "Math". This returns an IDispatch object. The dispatch object can be queried for dispatch ids for "sin" and "cos" and all that. It can be successfully invoked. Everything appears to be good. But when the user calls GetTypeInfo, the type description structure says that the object has zero functions. What's up with that?

Clearly the typeinfo is out of step with reality. The user has in fact found two bugs. The first is that the typeinfo building code only adds to the typeinfo those functions which have already been dereferenced. So, had you called "sin" once already, "sin" would be added but nothing else.

However, the second bug trumps the first one; built-in functions are automatically skipped when adding function descriptions to the typeinfo. I'm not sure what I was thinking when I wrote that logic; it was a long time ago. So even if you have already dereferenced a built-in function object, it won't get added to the typeinfo.

User-defined functions should be always correctly added to the typeinfo; they are automatically dereferenced when they are created.

I wrote almost all the typelibrary code, and I regret the errors. However, it's unlikely that either of these flaws will ever be fixed, and clearly both would have to be fixed in order to make the given scenario work correctly. Both fixes would touch a lot of complex code and are therefore quite risky. We try to not apply risky fixes to fix a problem that, to my knowledge, only one person has ever noticed in the last seven or eight years! If there's a massive outcry to make typeinfos for built-in objects work correctly, I'll pass that along to the sustaining engineering team; I wouldn't hold my breath waiting for a fix if I were you.

Thanks for bringing this to my attention.

Wherefore IDispatchEx?

Eric Lippert — Thu, 07 Oct 2004 17:32:00 GMT

Quite a while back I said that I'd write a post describing why we invented IDispatchEx. I already covered the part about using it to probe the stack for security reasons, but what other reasons did we implement that thing? The documentation is pretty much accurate, but sparse in places. (According to the doc, GetNameSpaceParent, uh, gets the namespace parent.) I won't go into the flag-for-flag level of detail that the doc does well, but I thought I might describe what features motivated the various methods on IDispatchEx. The features are:

Expando objects

In most objects, the methods you've got are the ones you get. But JScript objects and some IE DOM objects can have arbitrary, user-defined properties and methods added to them at will. We call such objects "expando" objects.

Strictly speaking, IDispatchEx is not required to implement an expando object. You could have an implementation of IDispatch which added a new named field every time GetIdsOfNames was called, and everything would work just fine. What IDispatchEx gives you is the ability to pass in a flag (fdexNameEnsure) that specifically requests expando semantics. An object model could choose to, say, only create new fields when the flag was passed in and error out if an unknown name was passed in without the flag. This would then allow the caller to probe for the existence of a property without creating it as a side effect. (Those semantics are up to the callee though -- callers cannot assume that not passing the flag means no expando semantics!)

Use the GetDispID method to access this feature.

Delete members

JScript has a delete operator which does not work at all like C++'s delete operator. Rather, JScript's delete operator removes an expando field from an object. It's pretty useless actually, because doing so does not free up any memory beyond simply clearing the field. Why not? Because implementers must ensure that if the property is re-added that it gets the same dispatch identifier the second time; someone might be caching the dispid. That means that the property bucket and its name has to be kept around, and worse, that every property bucket needs a flag that marks whether it's deleted or not. (The implementer must also ensure that property enumeration continues to work even if the enumerator -- see below -- is presently sitting on a property that was just deleted.) Use DeleteMemberByName or DeleteMemberByDispID in the unlikely event that you want to use this feature.

Case sensitivity

Visual Basic is case-insensitive, and IDispatch was built by the VBA team back in the day, so they never implemented support for case-sensitive languages. JScript is case sensitive, so IDispatchEx lets you pass in flags (fdexNameCaseSensitive, fdexNameCaseInsensitive) on GetDispID that control the case-sensitivity of the lookup.

Property enumeration

In the non-expando world, the property set of an object is stable, so there's little need to enumerate it. If you did need to enumerate it, you could always just look at the static typeinfo. But constructing dynamic typeinfos in a world with expando objects is difficult and expensive, so instead we added the ability to enumerate the valid dispatch identifiers (GetNextDispID) and turn the identifier back into a name (GetMemberName). This is how JScript's for-in loop is implemented.

Constructors

JScript functions may be called as constructors, which is a little weird and has different semantics from calling a regular function. IDispatchEx adds a DISPATCH_CONSTRUCT flag that can be passed to InvokeEx to let the callee know that it is being invoked as a constructor.

Namespace chaining

JScript supports some pretty intense scope resolution semantics, well beyond the local/class/global scoping rules in VBScript. Think closures, or with blocks, which allow construction of scope chains of arbitrary length. But that's not all -- it gets even more fun.

To make the this object semantics that you guys were arguing about the other day work, if you call a function that has a this argument, and it was invoked with the fdexImplicitParents flag, then we run up the GetNameSpaceParent chain to get all the parent scope objects and stuff them into the scope chain for the function invocation. Don't try this at home, kids; I barely remember how any of this stuff works.

Debugger support

Finally IDispatchEx has a GetMemberProperties method which is never used by JScript. Rather, it's used by the script debugger so that the debugger can tell you whether a given field is a method, property, etc.

So there you go -- a whole lot of new features that were difficult to shoehorn into IDispatch, so we implemented a new interface.

Use vs. Mention in JScript Doesn't Come For Free

Eric Lippert — Mon, 20 Sep 2004 17:16:00 GMT

Before today's entry, a quick note. Work has just gotten insanely busy as we push towards getting VSTO ready for the Whidbey release. I likely won't have much time to blog over the next couple weeks, so the blog refresh rate is going to go down for a while. I have a collection of pre-written articles -- such as this one -- that I'll dip into every now and then when I have a few spare minutes, but I likely won't be very topical or responsive for a bit.

OK, onward; a while back a reader asked what the difference was between

document.write("hello");

and

foo = document.write;
foo("hello");

from the point of view of what actually happens "behind the scenes" in the IDispatch calls. The reader noted in particular that though this trick works in IE, it doesn't work in WSH:

foo = WScript.Echo; // Nope, sorry.
foo("hello");

Here's the deal. The IE object model has been specially designed with JScript in mind. Unlike VBScript, JScript makes a distinction between naming a function and calling a function. When you say

abc = blah.baz();

that calls the baz function and results in whatever it returns. But

abc = blah.baz;

assigns the baz function object itself to abc; it essentially makes an alias.

The IDispatch interface takes flags which determine whether the caller wants to fetch the named property or call the named function. Many object models do not honour those flags though, because no language before JScript really took advantage of the distinction. In fact, object models designed to be called from VB or VBScript often don't even check the flag -- if it says "fetch the property" or "call the method", well, it just calls the method either way.

So here's what happens when you say foo = document.write; First, JScript attempts to resolve "document". It can't find a local or global variable called that, so it asks the global window object for the document property. IE gives back the document object. JScript then asks the document object to give back the value of the write property. IE creates an object which has a default method. The default method calls the write function, but no one calls the method yet -- we just have an object which, when invoked, will call the mehtod. JScript assigns the object to foo. Then when you call foo("hello"); JScript invokes the default method on the object, which calls the write method.

The WSH object model was not designed with this in mind. It does not make a distinction between naming a function and calling it, so you can't use this trick. WScript.Echo; does not give back a function object that can be invoked later.