HD DVD / Randomness... : Script

Fun Friends for Functions, Part 2: Handling Exceptions

ptorr — Sat, 02 Feb 2008 04:24:40 GMT

Previously we looked at how you can bind a function to a set of arguments and then call the resultant function at some time in the future. This technique is very powerful, but as-is it is a ticking bomb. As Amy points out in her blog post on exception handling, you always need to be prepared to handle an exception in your code. Failure to catch an exception usually leads to playback stopping and an error diagnostic from your player.

The trouble is that when you start using bound functions, you don't always have an easy way to deal with exceptions. Typically, a bound function is used as part of an asynchronous callback, and in those cases you don't have any functions further up the stack to save you from unhandled errors. Simple, you might think, I'll just wrap the contents of my function in a try-catch and be done with it.

Not so fast.

Let's look at the example function we were using in the previous blog post:

// function to set a backgroundImage
function SetBackgroundImage(markupId, url)
{
document[markupId].style.backgroundImage = url;
}

This could throw for several reasons – for example, an element with the markupId doesn't exist, or url isn't a correctly-formatted property value. Simply wrapping this up in a try-catch would result in something like this:

// function to set a backgroundImage
function SetBackgroundImage(markupId, url)
{
  try
  {
    document[markupId].style.backgroundImage = url;
  }
  catch(ex)
  {
    // What do we do here?!?
  }
}

But there's a problem – what do we do when we catch the exception? I don't know. And that's a key thing about exception handling:

Functions should only handle exceptions if they are actually prepared to deal with them. If a function doesn't know what to do in an error condition, it should let its caller figure it out. The exception [groan] to this rule is that your top-level code should always swallow exceptions (unless of course things are so out of whack that you think the player should stop immediately).

In this case, the SetBackgroundImage function has no idea what a failure means. It might be perfectly OK if the background image can't be set correctly, or it could be totally disastrous. It all depends on the context, so the SetBackgroundImage function should remain silent on the topic of exceptions and let the calling function figure out what (if anything) to do with them.

In our particular case, let's say that the exception is benign; we are simply updating some eye-candy graphics and it doesn't matter if they fail to appear. So then: maybe we should write a wrapper function, SafeSetBackgroundImage, that does this for us?

// function to set a backgroundImage, ignoring exceptions
function SafeSetBackgroundImage(markupId, url)
{
  try
  {
    SetBackgroundImage(markupId, url);
  }
  catch(ex) { }
}

I actually like to use this approach if I am going to be calling the 'Safe' version in a lot of different places, but if it is only in one or two places that seems like too much overhead to someone lazy like me. Anytime you update the parameter list of the wrapped function, you have to update the wrappers, too. It can also hide the intent of your code by sprinkling endless wrapper functions all over the place, and this also has the productivity-reducing effect of polluting IntelliSense with unnecessary function names. Instead, we can use the same kind of nested-function trick we used in the argument-binding case to get a "safe" version of any function:

// return a version of a function that is wrapped in try-catch
// you can optionally provide an errorFunc, which will be
// called with the 'tag' parameter in case of an exception
function Function_WrapInTryCatch(errorFunc, tag)
{
// stash the target function
var func = this;

// return the nested function
return wrappedFunction;

Function.prototype.WrapInTryCatch = Function_WrapInTryCatch;

In this example, we also allow the caller of the WrapInTryCatch function to provide an optional error-handling function that will be called if an error occurs. This allows you to do things like channel all your async exceptions to the same error handling function, which can figure out what to do based on the value of the tag parameter. Now we can re-write the last couple of lines of the example as:

// Call the download function, passing in the set-image function as the callback handler
var setBackgroundHandler = SetBackgroundImage.BindToArguments("button1", "file:///filecache/bar.png");
var wrappedHandler = setBackgroundHandler.WrapInTryCatch();
DownloadFile("http://foo/bar.png", "file:///filecache/bar.png", wrappedHandler);

This helper has the same limitations as the BindToArguments function (specifically: you can't pass any arguments to the returned function, and any tracing information is lost if 'errorFunc' is called) but this is also a solvable problem.

Why the ‘arguments’ object is bad, Exhibit #3,745

ptorr — Thu, 31 Jan 2008 00:46:52 GMT

Having just written a post with an example of how to use the arguments object, you might question my choice of title for this blog post. In my not-so-humble opinion, the 'arguments' object in ECMAScript is only useful for one thing:

The 'arguments' object should only be used by infrastructure functions that simply perform book-keeping on behalf of other functions. Any function that actually operates on its arguments in a meaningful way should never access the 'arguments' object.

The reason for this is plain and simple: code should be easy to read, easy to debug, and easy to maintain. Saving yourself a small amount of effort while writing code is a very bad trade-off if it negatively impacts these other three important tasks. Using the 'arguments' object in any normal function might save you a few keystrokes in the short-term, but will come back to bite you later on.

The most obvious problem with the arguments object is this: You have some code you wrote a few [days|weeks|years] ago and now you have to go and update it. You see some function calls like this in source code:

SomeBadFunction(42);
// ...

SomeBadFunction(i, current);
// ...

SomeBadFunction(next, data, OtherFunction(), moreData);
// ...

You think the problem might be related to one of the parameters being passed to SomeBadFunction, so you decide you need to check it out:

function SomeBadFunction()
{
// 200 lines of code
}

Oh dear. The definition of SomeBadFunction gives us absolutely no idea how many parameters it expects, what order they should be in, nor what values they might represent. Maybe there's a comment at the top of the function that explains what you should pass it, but we all know that comments get out of date very quickly and the code is the only real way to know what is going on. So now you have to trawl through several hundred lines of code, trying to reverse-engineer what kind of parameters the function expects. This could take a long time, and you might get it wrong.

Let's imagine that the real intent of SomeBadFunction is to update an internal table. The first parameter is the index of the item in the table to update, and the second parameter is the value to place in the table. The function supports rudimentary format strings (like printf() in C) and so if the value contains a format-specifier like "%d" or "%s" then the remaining parameters are used to fill in the blanks. Finally, if the value is omitted then the table entry is removed.

In this case, I would define the function something like this:

function SomeBadFunction(index, value, formatArgs)
{
// probably fewer than 200 lines of code
}

Now it is clear that SomeBadFunction expects three arguments, the first should probably be a number, the second should be some generic value, and the third should be an array of values (by using the naming convention ...Args; you could also call it formatArgArray or paramarray or varargs or something else if you feel that is clearer).

ECMAScript already allows for omitting function arguments, and code is typically written to have some "obvious" behaviour if parameters end up being undefined, so simply dropping value or formatArgs from a function call is quite OK. Now the only thing you need to change when you call the function is that if – if – you want to use the special format-string ability of SomeBadFunction, you simply encase the trailing arguments in square brackets to put them in an Array:

SomeBadFunction(next, data, [OtherFunction(), moreData]);

Anyway, I promised you an exhibit of why 'arguments' is bad – a bona-fide bug in ECMAScript, not just the rambling opinions of some Microsoft fuddy-duddy. Take this code:

function foo(x, y)
{
  print("x: " + x + ", y: " + y);
  Array.prototype.shift.call(arguments);
  print("x: " + x + ", y: " + y);
}

foo(42, 123);

What do you expect it to display? Don't ruin the fun by running it and posting the actual answer – just post what you think it is.

I say this is a bug in ECMAScript – rather than a bug in JScript – because section 10.1.8 of the ECMA 262 spec doesn't completely define how the arguments object interacts with the parameters of the function, and so you get some strange results. Note that both IE and Firefox give you the same result (replace 'print' with 'alert') but I doubt both companies independently came up with the same unexcpected result. I don't know if this is a case of IE being bug-compatible with the original Netscape Navigator code, or Firefox being bug-compatible with IE code, or a bit of both. I wasn't around when this was happening :-)

Fun Friends For Functions

ptorr — Wed, 30 Jan 2008 03:45:34 GMT

And now for a more techy post. In ECMAScript, functions are considered first-class objects. "What," you might ask, "differentiates a 'first-class' object from a merely 'economy-class' object?" Well, the main thing is that functions in ECMAScript are treated just like any other type of data (strings, numbers, arrays, etc.) in that they can be assigned to variables, they can be passed as function arguments, and they can have properties and methods of their own. They also have a prototype object, just like all other ECMAScript objects. The upshot of this is that you can do fun things with functions in ECMAScript that you can't do in other languages such as C or Java.

In this entry I will introduce four types of "helper" functions that are ideally suited to HDi development. They are:

Binding a function to a set of arguments and / or a 'this' object
Wrapping an existing function in an exception handler
Chaining function calls together one after the other
Calling a function after a specified period of time

In all cases, you write the actual function (the one that does the interesting work) just as you normally would, but when it comes time to call that function you do something funky with it to make some magic happen. Exactly why you might want to do this will become clearer soon, but it mostly comes down to how you write event handlers and callbacks.

A common theme in this post will be that you want to avoid closures as much as possible, and using these helpers should get you a long way to doing that (even though these helper functions rely on closures, they do so in a "safe" manner). And although there are various ad-hoc ways to accomplish the same tasks, generally these require writing specialised wrapper functions or following ugly coding practices (which I might blog about in the future) when what I would prefer to focus on is writing highly readable, highly re-usable code and utilising some clever, general-purpose helper functions to take care of all the tricky dirty work.

Binding to arguments

Many operations in HDi use a callback function to notify you when a specific task is completed (such as a File I/O operation or an event handler). Oftentimes you will want to associate more information with your callback than the HDi API actually allows for, and there are a couple of ways to accomplish this. But first, an example:

Say you have an application where you want to dynamically download some images from the network and display them inside a markup page. The basic code you want to write is probably like this (the code is simplified for clarity; obviously there is some additional code to convert the onStateChange callback into a single "the download is complete" call, etc.):

// function to download an image
function DownloadImage(src, dest)
{
  var httpClient = Network.createHttpClient(src, dest);
  httpClient.onStateChange = SetBackgroundImage;
  httpClient.send();
}

// function to set a backgroundImage
function SetBackgroundImage(markupId, url)
{
document[markupId].style.backgroundImage = url;
}

// call the download function... but what next?
DownloadImage("http://foo/bar.png", "file:///filecache/bar.png");

Note that both functions have been written to be "highly readable, highly re-usable" and do one thing in a very straight-forward manner. The problem, of course, is that this code won't work because there is no way for the callback to SetBackgroundImage to communicate what markupId or the final download url is supposed to be.

One common way of solving this problem is to use a nested function, like this:

// function to download an image and set it as the background URL
function DownloadBackgroundImage(src, dest, markupId)
{
  var httpClient = Network.createHttpClient(src, dest);
  httpClient.onStateChange = SetBackgroundImage;
  httpClient.send();

  // nested function to set a backgroundImage
  function SetBackgroundImage()
  {
    document[markupId].style.backgroundImage = dest;

    // Clean up the memory
    httpClient = null;
  }
}

// call the download function and update the element 'button1'
DownloadBackgroundImage("http://foo/bar.png", "file:///filecache/bar.png", "button1");

Note that even though we've done work to ensure the closure won't memory leak memory, there is still at big problem in that neither the download function nor the background setting function are generic any more. If you want to (eg) download an XML file, or if you want to set the backgroundImage to a file on persistent storage, you will need to either add some more hacks to this function (making it less readable and less maintainable) or you will need to duplicate the code into a similar-but-not-entirely-the-same function.

The solution I prefer is to use the fact that functions are first-class objects and solve this problem using "argument binding." In this case, the code would be re-written as follows:

// function to download a file and then call 'handler' when it is done
function DownloadFile(src, dest, handler)
{
  var httpClient = Network.createHttpClient(src, dest);
  httpClient.onStateChange = handler;
  httpClient.send();
}

// function to set a backgroundImage
function SetBackgroundImage(markupId, url)
{
document[markupId].style.backgroundImage = url;
}

// Call the download function, passing in the set-image function as the callback handler
var setBackgroundHandler = SetBackgroundImage.BindToArguments("button1", "file:///filecache/bar.png");
DownloadFile("http://foo/bar.png", "file:///filecache/bar.png", setBackgroundHandler);

Now both the download function and the set-image function are still written in a clear, simple, re-usable fashion, but we can easily chain them together to get the desired result. Even the call that makes this possible (the BindToArguments call) is straight-forward and understandable. All that is missing is the magic BindToArguments helper function... which looks like this:

// helper function to return a copy of the target function, but
// bound to the original arguments
function Function_BindToArguments()
{
  // store a reference to the target function and its arguments
  var storedFunction = this;
  var storedArguments = arguments;

// return the nested function
return boundFunction;

  // nested function simply invokes the stashed values
  function boundFunction()
  {
    storedFunction.apply(null, storedArguments);
  }
}

// make the helper available to all functions
Function.prototype.BindToArguments = Function_BindToArguments;

As noted in the comments, this helper function simply stores a reference to the "target" function and the arguments that are passed to the BindToArguments call. Then it returns a nested function, which when called will invoke the stored function (SetBackgroundImage) with the stored arguments ("button1" and "file:///filecache/bar.png"). Note that the apply function is a built-in function of ECMAScript Function objects that invokes a given function with the specified 'this' object and arguments array.

This is the most basic form of a "binding" helper, and the resultant function is one that has no 'this' object and that cannot take any arguments of its own. It also can't participate in a typical logging infrastructure because much of the required metadata (original function name, number of expected arguments, etc.) is missing. The good news is that all this can be added with the help of more code, and hopefully that's something I'll be posting soon.

As usual, this is longer (and took longer) to write, so I'll have to save the other "friends" for later. The depressing thing is that actually writing the code is the easy part (and it's all done already); it's wrapping it all up in a post and explaining how things work that take forever!

Maintaining Callback Semantics

ptorr — Wed, 24 Oct 2007 21:29:55 GMT

One of the key programming concepts in HDi is the use of asynchronous processing with callback functions. The HDi platform exposed to title authors is not multi-threaded (which helps reduce the complexity of writing applications), but many of the functions of the player are asynchronous in nature. For example, all the I/O in HDi (both local and network) is asynchronous so that the on-screen experience isn't degraded while the script waits for a long I/O call to complete. Instead, the script asks the player for a resource and then just keeps on plugging away. At some future point in time the resource will become available, the player will notify the script via a callback function, and the script can act on the resource without any further waiting.

As is common in any programming environment, many developers have built "helper functions" around some of the built-in HDi functions in order to centralise error handling, simplify high-level tasks, or for other various reasons. This is all well and good, but I have seen some developers use a programming pattern for these helpers that will lead to bugs down the road. I'm going to show an example of this bad pattern (and how to avoid it), but first here's an example of many people's first attempt at asynchronous programming:

// Global variables to hold state
var done = false;
var result = 0;

// Copy a file and return when it is done
function CopyFileHelper(src, dest)
{
// Copy the file, providing a simple callback function
FileIO.copy(src, dest, true, IOFinishedHandler);

  // Wait until the callback is called
  while (!done)
    ; // nothing

return result;
}

// Function to handle the callback and stop the loop
// inside CopyFileHelper
function IOFinishedHandler(errorInfo)
{
result = errorInfo;
done = true;
}

This code will never work, so luckily nobody has ever shipped a title like this (that I know of!). Nevertheless, it is interesting to learn why it doesn't work. The basic idea is to have a sentinel value (in this case, done) that is initially false. The FileIO operation is called and is provided a callback handler that will (i) stash the result of the I/O function for later use; and (ii) set the sentinel to true. The main function then performs a busy wait until the done variable is set to true by the callback, at which point the while loop ends and the function returns.

Although this kind of programming might work on a natively multi-threaded platform like .NET or Java, it won't work in HD DVD because of the way application time is scheduled. (Note it would also still be a bad idea on .NET or Java, but that's another story). In HD DVD, application code (including markup timing and script execution) is executed in a tick-based fashion. At the start of a tick, the necessary script code for that tick is executed (including event handlers, timers, and other callbacks) and then execution is passed to the markup layer. The markup layer performs timing and layout operations, and then the script is given another chance to run if it needs to handle any markup events. Finally, the markup page is generated and the system starts over again on a new tick. This all happens on a single thread. At the same time, other operations like A/V decoding and the FileIO APIs are happening on different threads.

What happens in this case is that the FileIO.copy function is called, a new thread is created to handle the copy operation, and then the script goes into the busy-wait loop. At some point in the future, the copy operation will finish and player will schedule the callback to happen on the next tick. But because the script code is busy waiting for done to become true, the current tick never ends and so your player's menu appears to hang. This is why nobody actually ships this code :-)

OK, so that's the impossibly-bad way of doing async IO. Here's the promised programming pattern that you should look out for and fix:

// Copy a file and do some common logging, etc
function CopyFileHelper(src, dest, clientCallbackHandler)
{
  // Perform some common, useful stuff...
  // ...
  // ...

  // Copy the file, providing the caller's callback handler
  // so they can continue when it is done
  try
  {
    FileIO.copy(src, dest, true, clientCallbackHandler);
  }
  catch (ex)
  {
    // Oops! Got an exception, so log it, convert to a FAILED result,
    // and let the caller know via their callback

    LogError("Exception while copying " + src + " to " + dest, ex);
    clientCallbackHandler(FileIO.FAILED);
  }
}

In this case, at least we're not trying to break the laws of physics. In most cases, this code will work fine. If the copy operation doesn't throw an exception, the client's callback is called and everyone is happy. If the copy operation does throw, we log the error and convert it into a failed result for their callback. This makes programming much easier for the client since they don't have to worry about exceptions anymore; they just treat all failures the same way... right?

Wrong.

Although it looks like you've made life simpler, you've just introduced a subtle bug that may not bite you until sometime in the future. The problem is that you've broken the semantics of the client's callback, which is that callbacks are always called asynchronously. You should never call a callback synchronously because the client code may not be expecting it.

Imagine the client has code like this:

// Perform a sequence of operations involving a file copy
function ClientCode()
{
DoFirstThing();

CopyFileHelper(src, dest, DoThirdThing);

DoSecondThing();
}

function DoFirstThing() { /* ... */ }
function DoSecondThing() { /* ... */ }
function DoThirdThing(result) { /* ... */ }

Under normal circumstances (no exceptions), the order of function calls will be DoFirstThing, DoSecondThing, and then at some future tick, DoThirdThing. Even if there is an error while copying, the code will progress in this sequence and the client can happily deal with any copy failures in DoThirdThing, secure in the knowledge that DoSecondThing has already completed. But what happens when there is an exception (which may not have happened during testing)? Now the order of execution is DoFirstThing, DoThirdThing, DoSecondThing, all in the same tick. You have reversed the order of the last two steps!

If DoSecondThing performs a critical task that is needed before DoThirdThing can succeed, the code is now broken and won't operate properly even though DoThirdThing may be coded robustly to deal with the copy error. The solution is to simply schedule the client's callback to happen at a later time, thus preserving the execution order of the client's code. This is easily done with a single-frame, non-repeating timer. Here's an example of how you might do it:

  // ... same code as before ...
  catch (ex)
  {
    // Oops! Got an exception, so log it, convert to a FAILED result,
    // and let the caller know via their callback

    LogError("Exception while copying " + src + " to " + dest, ex);
    ScheduleCallback(ExceptionCallback);
  }

  // Closure used to pass arguments to the callback
  function ExceptionCallback()
  {
    try
    {
      clientCallbackHandler(FileIO.FAILED);
    }
    catch (ex)
    {
      // Client callback failed; nowhere to go from here
      LogError("Fatal exception while calling client callback", ex);
    }
  }
}

// Call the function on the next tick. If you need to pass arguments to
// the function, then pass a closure (nested function) to do the work
function ScheduleCallback(callback)
{
  var t = createTimer("00:00:00:01", 1, callback);
  t.autoReset = false;
  t.enabled = true;
}

In this new code, if an exception occurs then rather than calling the callback directly, we schedule it to be called on the next tick via a couple of helper functions. We need the ExceptionCallback helper function in order to pass arguments to the callback (timers never pass arguments themselves), and we use the ScheduleCallback helper to avoid closures. This ensures that the client's code always executes in the order expected, and thus reduces your chances of having weird bugs in the future.

Sliced bread ain’t got nuthin’ on this!

ptorr — Tue, 02 Oct 2007 21:49:21 GMT

Forget being the best thing since sliced bread – this is the best thing ever. Well maybe not quite that good, but it's still darn cool!

Finally – FINALLY! – you can get decent IntelliSense for JScript inside Visual Studio!

Holy hotcakes, Batman! This is so awesome you have to download Visual Studio 2008 Beta 2 right now and try it out.

Back when I was on the script team, we couldn't get any features into Visual Studio (much to the chagrin of our customers) because all the hype was around ASP.NET. The rich client was dead. It was all about the server. Nobody needed client-side script any more. Everything was a web form post-back.

But now that AJAX and Silverlight and all these other buzzwords are suddenly important, people taking notice of the little script language that could.

Congrats and thanks to the Visual Web Developer and JScript teams – at last, JScript is getting some of the respect it has always deserved! :-)

Using Timers Effectively in HDi

ptorr — Thu, 30 Aug 2007 05:21:47 GMT

The use of timers in HDi applications is very common, and there are a few things you should keep in mind when using them to avoid performance issues when playing back your title.

I will categorise these as:

The autoReset problem
The closure problem
The re-use problem
The general-abuse problem

The AutoReset Problem

This problem stems from the fact that, by default, timers created in HDi are repeating timers, meaning that they will fire over and over and over again until you tell them to stop. Sometimes this is what the content author wants, but more often than not what they really need is a single notification after the timeout period has elapsed (what we call a "one-shot" timer). It's essentially the difference between setTimeout and setInterval in the browser DOM.

If you want a one-shot timer, the following pattern should be avoided:

// Create a timer; note it auto-repeats by default
var t = createTimer(TIME, CLOCK, callback);
t.enabled = true;

function callback()
{
// do stuff...

// Now disable the timer; sometimes people forget this step!
t.enabled = false;
}

Instead, consider the more straightforward (and less bug-prone) version:

// Create a timer and make sure it only fires once
var t = createTimer(TIME, CLOCK, callback);
t.autoReset = false;
t.enabled = true;

function callback()
{
// do stuff...
}

Sometimes, the problem is compounded because the author assumes the timer is a one-shot timer, and that in order to get repeating semantics they need to re-create the timer every time. That results in code like this:

// Create a timer; note it auto-repeats by default
var t = createTimer(TIME, CLOCK, callback);
t.enabled = true;

function callback()
{
// do stuff...

  // Now re-create the timer to ensure we get called again
  t = createTimer(TIME, CLOCK, callback);
  t.enabled = true;
}

Now what happens is that every time the timer fires, a new timer is created to fire on the same schedule! So after the first callback, there are two timers. After the second callback, there are four timers. After the third callback, there are eight timers, and so on. Eventually the old timers will get garbage collected by the script engine (and stop firing), but this could take a long time to happen depending on what else your script is doing. This is clearly a waste of resources and should be avoided; if you need a repeating timer, leave autoReset at its default value and don't re-create the timer later on (although I prefer explicitly setting autoReset to true to convey the intent that it really should repeat). Otherwise, set autoReset to false at the time you create the timer, and you will be happy.

The Closure Problem

In ECMAScript, a "closure" is a geeky name for a nested function. The reason we call them closures is to do with the way they affect the lifetime of other objects. I don't want to go into great detail now, but essentially a nested function maintains a reference to all the local variables of its containing function, thereby "closing them off" from being released by the garbage collector. Normally this isn't a problem for ECMAScript, because sooner or later the closure will die and thus the other objects it has been keeping references to will also die.

For example:

function MakeAClosure()
{
// Gobble up some memory
var bigObj = CreateECMAScriptObjectThatUsesLotsOfMemory();

// Deliberately set up a circular reference
bigObj.circularReference = Closure;

// Return the closure
return Closure;

  // Define the nested function (closure)
  function Closure()
  {
    // doesn't really matter what Closure does;
    // it still keeps a referenceto bigObj
  }
}

function foo()
{
  // Get the closure
  var closure = MakeAClosure();
  // maybe use it, maybe not...
}

// Call foo
foo();

In this example, we have an outer function (MakeAClosure), a nested function (Closure), another global function (foo), and some global code that calls foo. Here's what happens during execution:

foo gets called
foo calls MakeAClosure
MakeAClosure creates a large object that uses up some memory, and stashes it in bigObj
- The nested function Closure gets a reference to bigObj for reasons that I won't get in to here (check sections 10.2.3, 13, and 13.2 of ECMA-262 for technical details... or I can post more later :-))
MakeAClosure explicitly makes bigObj hold a reference to Closure so they both refer to each other – a classic circular reference problem
MakeAClosure returns the Closure Function object back to Foo
- Note that this includes the reference to bigObj
foo stores the Function object inside the closure variable
foo exits
Sooner or later, the garbage collector runs and sees that bigObj is used by closure, and closure is used by bigObj, but there is no way to access either of them (they are not "reachable") so it can safely kill both of them

Here everything is fine and dandy. But if bigObj where a non-ECMAScript object (eg, a Host object), the memory would never be reclaimed in Step 8 because the garbage collector cannot prove that bigObj is unreachable – maybe some other part of the system still relies on it – and so it has to assume that closure is still reachable, too. And since closure holds on to bigObj, the native object is never going to release itself, and the memory is leaked.

Perhaps it's clearer with an example.... using Timers, of course :-)

// Perform two animations, one after the other
function DoTwoPartAnimation()
{
// Perform some animation over 1 second
document.foo.animateProperty("y", "0px;100px", 1);

  // Setup a callback to do the other animation after the first is done
  var t = createTimer("00:00:01:00", 1, DoPartTwo);
  t.autoReset = false;
  t.enabled = true;

  function DoPartTwo()
  {
    // Perform second part of animation
    document.bar.animateProperty("x", "0px;100px", 1);
  }
}

This simple example illustrates a common pattern in HDi – chaining together two or more script-based animations by having a timer fire after the first animation is complete (this is unnecessary with markup animations, but they're not always practical).

In this case, DoTwoPartAnimation creates a local Timer variable, t, and gives it a reference to the nested function DoPartTwo as the callback function. As we learnt above, DoPartTwo maintains a reference to all of DoTwoPartAnimation's variables, including the timer t. So even after the timer has fired, and DoPartTwo has executed, we will leak the timer object (and the Function object for DoPartTwo) because the circular reference can't be broken.

The good news is that there are at least three ways to solve this problem:

Avoid closures altogether
Explicitly break the cycle
Avoid making the timer a local variable

The first is the simplest, and in the case above we can solve the problem by simply making DoPartTwo another global function instead of a nested function. Then although the timer t has a reference to DoPartTwo, DoPartTwo does not have a reference back to t and so the timer will be garbage collected sooner or later (which in turn will release DoPartTwo). The problem here is that often you need to use a nested function in order to correctly perform the callback operation – for example, the callback needs to know some of the parameters that were originally passed to the containing function in order to complete its task.

In that case, another simple solution is to explicitly break the cycle by manually releasing the reference to the timer. For example, we could change the DoPartTwo function slightly to look like this:

  function DoPartTwo()
  {
    document.bar.animateProperty("x", "0px;100px", 1);

    // Explicitly break the circular reference
    t = null;
  }

Now, as with the 1^st work-around, the script engine can eventually garbage collect t, which in turn will release DoPartTwo. Nevertheless, the problem with this solution – as with all solutions that require that the programmer religiously follow some strict rules all the time – is that you might forget to add this line to your callback, or you might have a return statement near the top of the function, or you might throw an exception, or... there are various ways your program might miss this opportunity to break the cycle.

So that leaves us with... my preferred approach, which is simply to make your own helper function – much like the web's setTimeout – that will create the timer for you. Here's a simple example:

// Call callback after timecode has elapsed
function SetTimeout(timecode, callback)
{
  var t = createTimer(timecode, 1, callback);
  t.autoReset = false;
  t.enabled = true;
}

Now you can simply replace all the pesky createTimer / autoReset / enabled code with a single call to SetTimeout, and avoid all the closure issues, too!

// Perform two animations, one after the other
function DoTwoPartAnimation()
{
// Perform some animation over 1 second
document.foo.animateProperty("y", "0px;100px", 1);

// Setup a callback to do the other animation after 1s
SetTimeout("00:00:01:00", DoPartTwo);

  function DoPartTwo()
  {
    // Perform second part of animation
    document.bar.animateProperty("x", "0px;100px", 1);
  }
}

The closure issues are avoided because DoPartTwo has zero knowledge of the Timer object t, which is a local variable of the SetTimeout function. Although the timer has a reference to DoPartTwo, the opposite is not true and so there is no circular reference.

Note about HTTPClient

Note that this "closure problem" also exists with HTTPClient objects and the onStateChange function. The same solutions exist for breaking the circular reference though – avoid closures, null out the object once it is finished (eg, STATE_COMPLETED, STATE_ERROR, or STATE_ABORT), or use a helper to create the object.

In general it doesn't affect other callback-based APIs like FileIO.openTextStream or XMLParser.parse because these are global objects, not local instances created by the enclosing function. The API will release the callback function as soon as it has been used (so it won't leak), and it doesn't really matter how many references there are to global objects; they will always exist.

The Re-Use Problem

Timer re-use is something that can be done correctly, but is often done incorrectly. Sometimes this is done by mistake (eg, forgetting to put var in front of a variable declaration, making it global), but other times it is deliberately done and leads to brittle code.

Here's a canonical example:

// Create timer; assume it is enabled
var timer = createTimer(TIME, CLOCK, CALLBACK);

function EventHandler()
{
// Clobber global timer; assume it is enabled
timer = createTimer(TIME2, CLOCK2, CALLBACK2);
}

function CancelTheTimer()
{
// Which timer are you cancelling?
timer.enabled = false;
}

The code is pretty much self explanatory:

A global timer is created
An event handler clobbers the first timer and creates a new one
A cancel function disables the timer

The problem is that depending on the order in which EventHandler and CancelTheTimer are called, different things may happen. If CancelTheTimer is called before EventHandler, it will cancel the global timer, but if the order is reversed then the global timer will fire and the event-handler timer will be disabled instead!

Even if you think you know the order in which events will occur, they may not always fire in exactly the same order – especially if they rely on user input or some asynchronous processes such as the progression of the title timeline. Better to always assume that the worst will happen, and program accordingly.

The simple solution here is to always use a unique name for your timers, or ensure they are local variables that won't conflict with other timers. (I would say to use the SetTimeout function from above, but that won't allow you to cancel the timer).

The General-Abuse Problem

The "general abuse" category basically covers any use of a timer that is unnecessary given the feature set of HDi. Sometimes authors abuse timers because they are not aware of HDi's built-in features (in which case it is generally easy to change from an abusive timer to use the right feature), or sometimes it is because the application's design has been built around timers rather than some more elegant solution (in which case it can be much harder to re-work the code to use fewer timers).

Two prime examples of the former include:

Performing many set operations on a repeating 1-frame timer, rather than using the <animate> element in markup or the animateProperty API
Constantly polling currentTitle.elapsedTime to check for a specific timecode, rather than using an <event> in markup or a PauseAt in the playlist

Examples of the latter are harder to come by, but (with no intention of bashing the other format) we have seen authors trying to replicate the behaviour of BD-J applications on HD DVD by an over-zealous use of timers, since they don't necessarily realise that the HDi system will do a lot of the heavy lifting for them.

Final Thought

An astute observer might ask the question "what happens if a timer is garbage-collected before it fires?" Good question. It is entirely possible that the local variable t used in these examples will be collected before the timer fires, especially if the timeout is quite long (say, several minutes). The answer is that the HD DVD specification requires timers to fire at least once, even if the script engine has released all references to them.

Simple ACA dumper

ptorr — Fri, 06 Jul 2007 00:36:00 GMT

Quick one today – if you have an ACA with some files in it, you can use the createaca tool (provided as part of the Jumpstart Kit) to list the files inside it. Trouble is that the output is not very helpful; it contains too much information.

So here is a simple JScript file you can run on the output of createaca -r to just dump the filenames (and it checks for duplicates, too!). Just copy to dumpaca.js (or the file of your choice) and run it from a command prompt. It expects a file named filelist.txt to exist in the same folder. Modifications to do whatever you want to do should be trivial...

// Open file named "filelist.txt"
‎var fso = new ActiveXObject("Scripting.FileSystemObject");
var stream = fso.OpenTextFile("filelist.txt");
var contents = stream.ReadAll();

// Regular expression to search for DATA_FNAME followed by a name
var re =/DATA_FNAME : ((?:\w|\.)+)/g;

// Lists of result data
var names = {};
var nameList = [];
var result;
var dupes = [];

// Search for every instance of the regular expression
while(null != (result = re.exec(contents)))
{
var name = result[1];

  // Record dupes
  if (names[name] != null)
  {
    dupes.push(name);
  }
  // else record new values
  else
  {
    names[name] = nameList.length;
    nameList.push(name);
  }
}

// Print results
print("---------- Complete file list ----------");

for (var i in nameList)
print(nameList[i]);

// Print dupes
if (dupes.length > 0)
{
print("---------- Duplicate file names ----------");

for (var j in dupes)
print(dupes[j]);
}

// Helper
function print(s) { WScript.Echo(String(s)); }

Helpful Tracing Routines

ptorr — Wed, 27 Jun 2007 11:37:00 GMT

Last time we looked at conditional compilation, and how using it with HDiSim could aid with interactive debugging of runtime exceptions. But as useful as interactive debugging is, there are three major cases where it is impractical or impossible:

When the bug is timing related, and using a debugger masks the bug; or
Reproducing the bug requires running a large amount of code it's impractical or laborious to step through by hand; or
You're trying to track down an issue on a real player (which you can't debug on).

In these cases, it is very useful to have some robust tracing primitives built in to your code to give you a trace (or log) of everything that the player is doing up until the time the bug occurs. I already posted a while back on red box debugging, and how that relates to printf debugging; now we're going to add some more printf-like tracing to our HDi toolkit to help diagnose issues on retail players or to trace errors through the system in critical timing sections.

The general approach is to pepper our code with calls to a trace() function that will either write the trace info to a log file, or display it on the screen, or both. Usually the trace function will be called when execution enters a new function, when an exception is handled, and at any other point in time where it is useful to know the result of a test or the value of a particular variable or property. Now, the key to any good trace log is the ability to identify the function that was active at the time a given log entry was made. Whilst it is possible to hard-code the name of the function into the trace call (for example, trace("Inside function 'foo'")), this is problematic because any time you re-name a function (or copy-and-paste it to a new function) you need to update the function name in the script. It would be much better if you could somehow automatically get the function name...

Enter the GetFunctionName function, defined like this:

// Returns the name of a function
function GetFunctionName(func)
{
try
{
var re = /function\s+(\w+)/;
var s = re.exec(func.toString())[1];

    // Based on naming convention of 'object_FunctionName'
    if (s.indexOf("_") >= 0)
      return s.substring(s.indexOf("_"));

return s;
}
catch(ex) { ; }

return "AnonymousOrUnknownFunction";
}

Given a Function object as its only argument, this function will return the name of the function (or the string "AnonymousOrUnknownFunction" if the function name is not available – for example, you pass in a function expression or a native method). It does this by creating a Regular Expression re that looks for the word "function" followed by some whitespace \s+ and then a word (\w+). If the word is found, it is "captured" (because it is in parentheses) and returned by the expression re.exec(...)[1]. The toString call inside of exec simply converts the Function object into an implementation-defined representation of the function body, but ECMA 262 defines that at a minimum this contains the function name (ie, it is a FunctionDeclaration).

OK, that's kind of useful, but how do you get the right Function object to pass to the GetFunctionName function? Good question. The answer is that you use the arguments object, and in particular the callee property; it holds a reference to the function being called. Neato, eh?

I want to take a quick detour here to say that I personally think that relying on the arguments object as a way of passing parameters to a function is a bad programming practice and should be avoided for normal development; I won't bore you with all the details, but basically it makes your code harder to read (and thus harder to understand) if all the expected parameters aren't immediately obvious from the function's declaration. Nevertheless, the arguments object is great for tracing, and so we shall use it for this.

So while we're on the subject of the arguments object, it's also worth noting that it's a great way to trace the arguments passed to a function in a generic way (again, avoiding the copy-and-paste problem, or having to keep up with added or removed parameters). To do this, we'll write another function (and a little helper) that can be used to generate a string holding the value of all the arguments passed to a function. It looks like this:

// Returns a string representing the arguments list of a function
function GetArgumentsString(args)
{
var s = "(";
var first = true;

for (var i = 0; i < args.length; i++)
{
if (first != true)
s += ", ";

first = false;

var arg = args[i];
s += ObjectToString(arg);
}

s += ")";
return s;
}

// Returns a string representation of an object;
function ObjectToString(obj)
{
try
{
if (typeof(obj) == "function")
return "function " + GetFunctionName(obj);

if (typeof(obj) == "string")
return "\"" + obj + "\"";

return String(obj);
}
catch(ex)
{
return "[native Object]";
}
}

Together these two functions will take an arguments object and return a string that contains all their values, including special handling for functions and native (host) objects. Now we can build a simple function that is used to trace function entry like this:

// Global to define whether to do tracing or not;
// Tracing can be expensive so you might not want to do it on a released title
var _DO_TRACE = true;

// Trace function name and argument list
function TraceFunctionEntry(args)
{
// Bail immediately if tracing is not desired
if (_DO_TRACE != true)
return;

var name = GetFunctionName(args.callee);
var argString = GetArgumentsString(args);
TraceString(name + argString);
}

And we can use the TraceFunctionEntry function like this:

function MyFunction(a, b, c)
{
TraceFunctionEntry(arguments);

// Do some actual work...
}

Now we just need to define the TraceString function; but that's the easy part ;-)

// Global startup code to open the diagnostics file, if necessary.
if (_DO_TRACE)
{
try
{
Diagnostics.listeners.add("file");
Diagnostics.trace.autoFlush = true;
} catch (ex) { }
}

// Function to trace a message to the diagnostics log and, if available,
// an on-screen input.
function TraceString(msg)
{
if (_DO_TRACE != true)
return;

// Write out the message to the diagnostics log
try
{
Diagnostics.trace.writeLine(msg, "Trace", 1);
} catch (ex) { }

// Try to add the message to an <input> named 'trace', if one exists
try
{
    var node = document.trace;
    if (node != null && node.nodeName == "input")
      node.state.value = node.state.value + " | " + msg;

} catch (ex) { }
}

And there you have it! I have attached a sample project that includes a couple of other helpful tracing aids (one for errors, one for informational tracing). As usual, you need to copy a file named font.ttf into the ADV_OBJ folder in order for it to work (I use the "Kooteny" sample font from the HDiSim download).

Using Conditional Compilation in HDi

ptorr — Sat, 02 Jun 2007 05:40:00 GMT

One of the cardinal rules for developing HD DVD applications is, "let no exception go unhandled." The reason for this is simple – if you fail to handle an exception, the player will kill your application and stop the disc. This leads many developers to wrap entire functions in try / catch blocks, which solves the problem but introduces some of its own.

To me, one of the biggest problems with using try / catch during development is that it makes debugging script with Visual Studio virtually impossible. For a start, you can't rely on "just-in-time" (JIT) debugging to work because you never have any unhandled exceptions. Additionally, even if you manually attach the debugger and step through your code line by line, as soon as there is an exception it will dump you out at the catch handler, losing all the context information you might need to fix the bug (stack trace, local variables, even the location in the source file).

As an example, try this script (run it under WSH with the command-line cscript //x test.js) and step through each line by hitting F11. Step over the PretendThisIsAPlatformAPI function by hitting F10 instead.

// Hit F11 to step INTO each line
try
{
foo(1)
}
catch(ex)
{
WScript.Echo(ex)
}

function foo(x)
{
// Imagine lots of complex code here...
var y = x - 1
bar(y)
}

function bar(x)
{
// More complex code here...
var y = x / 2
// Hit F10 to step OVER this function
PretendThisIsAPlatformAPI(y)
}

function PretendThisIsAPlatformAPI(x)
{
if (x <= 0)
throw("bad parameter value")
}

Note that everything is fine until you call that fake platform API. Then you are dumped back into the catch block of the global code with a message saying "bad parameter value." But what was the parameter value I passed to the function? I can't find out, because all the context (the stack frames for bar and foo) has been removed.

Is the solution to remove the try / catch block during development, then add it back at the last minute before shipping?

NO!

That is a recipe for disaster for at least two reasons:

You might forget to put it back in
You will never have tested your exception-handling code

There's a better approach, and it relies on a feature of Microsoft's JScript implementation that lets you "hide" parts of the script from JScript whilst making it clearly visible to normal ECMAScript engines. This allows you to toggle exception handling on or off during development (with HDiSim), whilst never risking having it turned off when you ship.

This feature is called conditional compilation, and here's how we use it:

The first thing to do is to activate conditional compilation. We do that like this – note how it is in an ECMAScript comment, so that other script engines will ignore the command:

//@cc_on

Next, we define a conditional-compilation variable that will determine whether or not we want exception handling (note: you can name the variable anything you want; this isn't some magic value):

//@set @enable_exceptions = true

Perhaps obviously, we set this to true if we want to handle exceptions with try / catch, and false if we do not.

Finally, we wrap our try and catch blocks with a conditional statement:

//@if(@enable_exceptions)
try
{
//@end

foo(1)

//@if(@enable_exceptions)
}
catch(ex)
{
WScript.Echo(ex)
}
//@end

Now, depending on the value of @enable_exceptions, JScript either sees this source code (when true):

try
{
foo(1)
}
catch(ex)
{
WScript.Echo(ex)
}

Or this source code (when false):

foo(1)

Non-JScript engines will always see the former source code, because they simply treat the @if statements as comments. That means they will always have try / catch turned on, and is why I say it is impossible to "forget" to turn exception handling back on when you ship.

During normal testing and development, you should leave @enable_exceptions set to true, because you need to test your exception-handling code. But when you are trying to debug a problem caused by a thrown exception, you can set it to false and use the JIT feature of Visual Studio to jump right to the line that is throwing the exception. You should then turn it back to true again after you've finished debugging, because it is possible that some players on Windows might use the JScript engine. Even then, this approach is better than manually adding and removing the exception-handling code from your applications since there's only one place in your code to do it, rather than every single location where you use try / catch.

You can try this out now, by running the sample script again with exception handling turned on and off. In this case, instead of stepping through the file with F11, just hit F5 to run the program. When exception handling is turned on, you don't get any help from the VS debugger. When it's turned off, VS will break at the offending line so you can investigate the problem and determine the right fix.

Marvellous!

Typically I have a single file, cc.js, that I always include as the first <Script> element in my manifest. It contains the @cc_on statement and the @enable_exceptions variable, along with any others I might decide to use. As part of your shipping process, you could decide to always remove this file (or replace it with one that forces @enable_exceptions to true) "just in case"...

A Game of Chance that Pirates like to Play

ptorr — Thu, 17 May 2007 01:52:00 GMT

Ahoy there me laddies! Yarrrr you better be readin' this blog then matey, or you'll be feelin' the lick o' the cat come nightfall!

No, it's not Talk Like a Pirate Day; it's Let's Post Another HDi Game Day. The other week I posted an HDi sample based of the old arcade game Space Invaders. Today it's a (somewhat prettier) example that implements the old game Liar's Dice. Some of you may wonder how I chose these two games to demo as HDi applications; the rest of you already know this is to show just how easy it is to get great interactivity on a next-generation high-definition optical disc without expensive replication, without expensive programmers, and without an expensive games console to play it on. This was all written by little old me in a couple of hours, is about 130k in size (most of which are PNGs), and will play on any sub-$300 HD DVD player without first running through a performance test application. [Ed: I am currently travelling without access to a real player so have been unable to test it properly; it works on HDiSim and if there are any issues with up-to-date players I will resolve them next week]

Here's a screenshot of the game:

As with the last game, this was thrown together very quickly and doesn't necessarily represent "best practices" (I know, I know, I should be demonstrating those too). Nevertheless the concepts should be pretty easy to understand. If not, post a comment! :-)

The download is here. Enjoy.

Red Box Debugging

ptorr — Thu, 01 Feb 2007 23:07:01 GMT

Today I have a simple debugging tip.

Sometimes when you have an obscure bug, you need to answer the question "is my code even getting executed?" Often you can find that really obscure bugs are caused simply by a bad conditional, a typo, an unexpected exception, or some other flow-control issue that bypasses your code altogether.

Whilst you can use an interactive debugger such as Microsoft Visual Studio for tracking down some of these things, debuggers aren't always available (eg, if you are on a hardware emulator), and if the bug is timing-related then they hamper your ability to reproduce the problem. Worse still, if your bug is in a declarative timing section (rather than in a script file) there's currently no way to debug that at all.

So you need something a bit less impactful.

In the good old days of C programming, people used to use "printf debugging" to display a message on the console indicating that a particular line of code had executed, and possibly to show the value of a variable. In the web world, this was changed to "alert debugging" where a popup window was used to the same effect.

For HD DVD, I like to use "red box debugging". Basically I create a big red div in my markup file and set style:display="none". Then at the point in my script where I want to "prove" that it gets executed or not, I simply say document.whateverTheIdIs.style.display = "auto" and see if the red box appears. And if your bug is timing related, you simply has a cue that does a <set style:display="auto" /> and similarly proves (or disproves...) that your program is working as planned.

That's all J

Simple Pre-Processor for HDi

ptorr — Wed, 10 Jan 2007 11:07:00 GMT

Hope you all had a good break. I haven't blogged for a while due to travel (both business and pleasure) and laziness… but I thought I'd start off today with simple pre-processor for HDi.

One of the things that is missing from HD DVD is a good pre-processor; anyone familiar with C will know of all the good (and evil) things you can do with a few #define macros. Anyone not familiar with C, well, you don't know what you're missing out on :-)

Anyway, what I am basically talking about is the ability to define macros (symbolic constants; shortcuts; expansions; whatever) that you can use in your markup files and script files to simplify development.

Here's a common problem scenario (at least for me):

Lay out some simple markup
Create a button foo with (eg) an x position of 100px
Write some animation that moves foo from 100px to 200px (this could be in script or markup)
Realise you need to shift foo to 110px to accommodate a stylistic change
Now figure out all the places where you need to change 100px to 110px, noting that you can't do a global search-and-replace because 100px could refer to other things as well

Traditionally this kind of problem (known as a "magic number") is solved by using a symbolic constant – we define a placeholder value named (eg) FOO_X and we define it to be 100px. Then everywhere we want to refer to the x location of foo, we use FOO_X rather than 100px. Then if we want to update foo to be at 110px later on, we simply change the definition of FOO_X to be 110px and our work is done.

This is easily done in script (where you have variables) but not so easily done in markup. So, I have written a very simple pre-processor that will take a bunch of macro definitions and replace them in your markup files AND give you a nice script file containing matching declarations to go along with it.

In addition to simple text expansion (FOO_X = 100px) you can also do arithmetic (FOO_X = BAR_X + 100) and indeed perform any kind of processing in your macros because they are all eval-ed at "compile" time. This has made my life a lot easier, especially when I am just testing things out and I need to re-lay-out stuff over and over again.

Attached is a simple project, along with preprocess.js. You run preprocess.js like this:

cscript.exe preprocess.js <defs> <infile> <outfile> [-d]

where:

<defs> is the name of the definitions file, eg "my_definitions.ini"
<infile> is the name of the input file, eg "markup_with_symbolic_names.xmu"
<outfile> is the name of the output file to create, eg "markup.xmu"
The optional –d switch produces a .JS file with the same base name as <defs> containing JScript declarations for all the macros

The definitions file is a simple text file that contains four types of lines:

Empty lines or lines beginning with a semi-colon are ignored (eg, use for comments)
Lines of the format NAME <space> TEXT define NAME to be exactly TEXT
Lines of the format NAME <space> '=' CODE define NAME to be the evaluation of CODE, which can contain references to functions or other NAMEs
Lines of the format '#' CODE run CODE directly without creating any new names (it is assumed the function being called adds names implicitly)

If a macro definition looks like a number, the system will automatically create an additional macro with the same name with "_PX" on the end that has the same value with "px" on the end. An example of a simple file:

; Here is a simple file
; These lines are ignored
; This creates two macros
; The first is called "BUTTON_X" and has the value 100
; The second is called "BUTTON_X_PX" and has the value "100px"

BUTTON_X 100

; This also creates two macros by doing simple arithmetic
; Given the definition of BUTTON_X above, this will produce
; both "BUTTON_Y" with a value of 200
; and "BUTTON_Y_PX" with a value of "200px"

BUTTON_Y = BUTTON_X + 100

; This runs the theoretical function DoSomethingCool, which you
; could add to preprocess.js

#DoSomethingCool()

Note that preprocess.js is not necessarily the best code in the world, nor is it a secure application (it will eval arbitrary input code, so DO NOT USE IT on untrusted definition files!). It was something I threw together and have been iterating on in the normal hacker fashion. It was an itch that needed scratching, not a well-designed and thought-out application.

Also note that whilst using the –d switch will output a script file, it is not big-endian as the HD DVD spec requires! This is due to a limitation in the FileSystemObject. You need to open the JS file in Notepad and re-save it with Big Endian encoding. That's why it isn't generated by default…

So, on to the enclosed example HDi program, it is very simple and very ugly. All it shows you is a simple defs file and a simple input markup file. Running the batch file will create the output markup file and then run the result (assuming HdiSim.exe is in your PATH). Clicking on the blue square makes it grow larger.

AMAZING!

But what is "cool" about this is that if you edit the defs.ini file and change (eg) the SMALL_W_ANIMATED value to 500 (to make the blue button grow larger) and re-run the batch file, the red button will automatically shift as well. No need to go and tweak its values to compensate for changes in the blue button.

AMAZING!

Well, not so much, but in more complicated projects this is a real time-saver.

Script Execution Order in HD DVD

ptorr — Thu, 09 Nov 2006 00:28:00 GMT

An HDi application can use multiple script files so that you can better manage your code (more manageable files; easier source control; etc.). You do this simply by listing the files inside <Script> elements inside the manifest (and of course add the appropriate <Resource> elements if they're not all in an ACA file). Whether or not script code in one file can "see" the script in another file is a common question that I'll address here.

Spec Behaviour

The HD DVD specification says that you take all the script files, load them in the order they are listed in the manifest, and then start running the script engine. This means that you really have one big "virtual file" that is the concatenation of all the individual files, and so all the code can "see" all the other code.

This doesn't mean you're home and dry, of course, because there are some other things to be wary of that we'll get to soon.

HDiSim Behaviour

The spec behaviour was clarified in a fairly recent supplemental change, and so HDiSim currently uses a different approach – it loads each script file and executes it in the order listed in the manifest, similar to how a web browser would load and execute multiple <script> blocks. What this means is that in HDiSim, code can only "see" code in the files that were listed before it in the manifest.

Note that this only applies to "global code" that executes as soon as the script is loaded; basically any code outside of any functions (and the code that it calls, etc.) is subject to this rule, but because ECMAScript is a late-bound language, once all your script files have been loaded and your setMarkupLoadedHandler has been called, all your code can "see" all the other code anyway.

So it's only during initial startup that you have this problem.

Caveats

There are still several things to think about though. First let's look at what happens when an ECMAScript interpreter gets hold of a script. It basically does the following:

Scan the file for variable declarations, and make them available (but with the initial value undefined)
Scan the file for function declarations, and make the Function objects available
Start running program code

This means that you can always call functions declared in the same file, and you can always access variables declared in the same file, but if you access variables before they are initialised you get undefined.

Here's a quick example, using Windows Script Host as an example:

var above = "you see this"
print(above)
print(below)
var below = "you don't see this"
function print(s) { WScript.Echo(String(s)) }

Running this program will print "you see this" and "undefined" on the screen. Note that if you comment out the line "var below=..." you will get a runtime error because below is not declared. ECMAScript lets you assign values to undeclared variables, but you cannot read undeclared variables (in broad terms, they can be l-values but not r-values).

Another HD DVD-specific caveat is that the document object is not available when your script first executes; it doesn't become valid until the markup document has loaded and your setMarkupLoadedHandler function has been called. Because of this, you probably want to avoid running too much code at startup anyway; sooner or later you're going to try and access document, and crash.

Finally, ECMAScript lets you re-define functions (since they are just objects like anything else). Within the same file, the lexically last definition will win, but in the case where you load multiple files independently (ala HDiSim) you could end up running one version of the function from global code inside one file, then re-defining it in a later file and running different code. Don't do that.

The Long and the Short of it

So basically what this means is that for maximum compatibility with the spec and with HDiSim, you should just make sure that your script files are listed in a "reasonable" order in the manifest, and that you don't try to do too much work before setMarkupLoadedHandler is called. At that stage, everything will be available to you.

Also, never re-define functions unless you really know what you are doing.

How do I get started with iHD?

ptorr — Tue, 18 Jul 2006 00:50:03 GMT

Many DVD authoring professionals may be wondering how they can get started with HD DVD. Probably the first thing to do is to download the iHD Jumpstart kit and get yourself a good XML editor (like the free Visual Studio Web Developer edition) and a graphics program like Photoshop or the Microsoft Expressions beta. This will allow you to create iHD content and play it back on your PC.

But the tools aren't much good if you don't know how to use them. You can look at the very basic samples in the iHD download, or look at some of the samples on my blog, but the best way to go from zero to hero (*cringe*) is to get a good book on HTML programming for the web.

iHD's concepts are very similar to those used on the web -- you have some content elements (like divs, buttons, and paragraphs) and then you make them look pretty by giving them styles (like CSS -- Cascading Style Sheets). You then make things happen by writing small blocks of script code, just like a web page. The only thing that doesn't have a good parallel with web programming is the XPath-based timing engine, but hopefully I will be writing more about that in the not-too-distant future. You don't really need to know much about XML to write iHD; learning HTML is probably enough to get started, and any decent XML editor will help write the correct XML syntax for you (or, at least, tell you where you went wrong :-) ).

So if you know nothing about web development, or if you haven't really learnt anything new since 1995, I would suggest picking up a book on HTML programming -- just make sure it covers CSS and some basic JavaScript! Don't pick up one of those really thick books that contain a full reference to every DHTML object, property, and method, since that will not do you any good. You want to know about the basics of HTML structure, the basics of CSS and styling, and a little bit about script (which you might be better off getting a different book for -- again, just make sure it isn't full of HTML object references, because that's a waste of your time).

There are also lots of references on the web for HTML, CSS, and Script... unfortunately I don't have any personal recommendations right now; let me know if you find something particularly good!

Inheritance in JavaScript - Part 2

ptorr — Tue, 20 Jun 2006 05:38:00 GMT

Last time we took a (very) long-winded look at a simple JavaScript program that used basic inheritance. This time, we'll expand on that a little bit and improve on the techniques.

Calling the Constructor Directly

First of all, let's look back at the Point constructor again:

function Point(id, x, y)
{
this._id = id;
this._x = x;
this._y = y;
}

Last time we noted that we just stash the value if id into the _id member because that's the only thing that the Thing constructor does with it. But what if Thing did some more complex processing, or took a larger number of arguments? We could keep copy-and-pasting the code from Thing into Point, but that is a tiresome process known as the "double maintenance problem" and will very quickly lead to bugs.

Much better is to simply let Thing do its, err, thing, for us, by calling into it directly:

function Point(id, x, y)
{
Thing.call(this, id);
this._x = x;
this._y = y;
}

The only potentially weird bit here is that rather than just calling the Thing function directly, we call a function off the Thing function to do some special house-keeping work for us. How so, you ask? Remember that in JScript, functions are first-class objects and can have properties and methods of their own, just like any other object. They just happen to have some internal properties like [[Call]] and [[Construct]] that make them cleverer than the average object.

So, the Thing function has some functions of its own, one of which is called call. What the call method does is call the function that it belongs to and passes in a random object as the this object. In ECMAScript you don't usually have to ask yourself "How does my function get a this object anyway?" because it just works. It's only when it doesn't work that you get confused.

Well, when you call a function as a constructor (i.e., as part of a new expression), JavaScript creates a new object for you and then passes it in as the this object. So you can write stuff like this._id = id and the this object just happens to be there for you. And then typically you assign the result of the constructor to a variable -- something like var thing = new Thing(42) -- and now the thing variable holds onto that object. When you call a method off an object, JScript once again figures out the this object for you (it's the thing holding the method), and passes it in for you. If you just call a function as if it were a normal method -- i.e., not as part of a new expression, or not off an existing object -- then the script engine will just use the global object as the this object.

So what if you want to call a method on an object you already have? That's the case here -- we already have our this object inside the Point constructor, and we want to ask Thing to do some work for us. But we don't want to create a new Thing -- we're deriving from it, remember -- and we don't want it to operate on the global object either; we want it to operate on our currently existing this object. And that's where the call method (and it's cousin apply) comes in -- it lets you call a method and supply an arbitrary object that will become the this object inside the method call. So we just call Thing.call and supply our this object as Thing's this object, and everything is peachy. (The apply method is basically the same as call, but deals better with a variable number of arguments -- a programming practice you should try to avoid).

OK, so now we have our Point object calling into Thing to do the work for it, and we have saved ourselves the cost and complexity if maintaining the same code in two places. Are we done?

Nope!

Two things are still pretty ugly about this: this method is still too expensive, and it's not conducive to copy-and-paste programming.

Reducing Duplicate Function Calls

Let's look at the code for Point again:

function Point(id, x, y)
{
Thing.call(this, id);
this._x = x;
this._y = y;
}

Point.prototype = new Thing();
Point.prototype._className = "Point";
Point.prototype.constructor = Point;

Notice something fishy going on here? If we create n Point objects, we actually call the Thing constructor n + 1 times -- that "+ 1" is caused by the initial call to set the Point.prototype property. In this specific example, that's not too bad, because Thing doesn't really do much. But in a more complicated scenario, it could cause problems. For a start, if the Thing constructor is expensive, we will waste some CPU on that first initialisation (hopefully not a huge concern, but you never know).

Secondly, if Thing actually does anything "interesting" in the constructor (and by "interesting" I mean interacting with the world outside of this -- incrementing a global counter, printing information on the screen, creating a row in a database, or whatever), then you will have an additional bogus interaction from that Point.prototype = new Thing() call. Unlike the first point, this is more likely to be of concern, so it's something we should try to avoid.

Thirdly, we construct Thing without any arguments -- so if the constructor expects to receive meaningful parameters, it is likely to throw an exception or perform some other unpleasant acts because it got a bunch of undefined values instead of real data. One way around this would be of course to supply Thing with the parameters it needed, but since they are "throw away" parameters (they aren't really used for any specific Point instance, because we're going to call the constructor with real values later on), it kind of defeats the purpose. And, again, it could put us into unchartered waters further down the road, depending on the complexity of the Thing constructor.

We can easily avoid all these pitfalls by using a different coding pattern that is tuned more to the way ECMAScript works and less to the way C++ works (unless you write COM objects for a living). Rather than doing any work inside the constructor, I make the constructor do the bare minimum initialisation steps (basically set everything to null / zero / empty string) and then do the real work inside an Initialise method. This avoids duplicating effort inside that initial call to setup the prototype, but still leaves the object in a bare-bones usable state should someone forget to call Initialise before using it. (You could leave the constructor completely empty if you like, choosing to do all the initialisation during the Initialise call, but personally I think that setting up all the properties in the constructor provides better documentation -- you only need to look at the constructor to find out what members the object has).

So, using this technique, the code is modified thusly (only new things are re-printed here, not the whole script):

function Thing()
{
this._id = "";
}

Thing.prototype.Initialise = function Thing_Initialise(id)
{
this._id = id;
};

var thing = new Thing();
thing.Initialise('thing');

//////////////////////////////////////////////////

function Point()
{
this._x = -1;
this._y = -1;
}

Point.prototype.Initialise = function Point_Initialise(id, x, y)
{
Thing.prototype.Initialise.call(this, id);
this._x = x;
this._y = y;
};

var point = new Point();
point.Initialise('point', 0, 0);

Whilst this style might offend some of the more hard-core C# / Java / etc. enthusiasts, and it is a new coding convention that takes some getting used to, it definitely helps curtail the one-too-many-constructor-calls problem that will come up to bite you in the proverbial sooner or later. I recommend you consider it.

Making Code Copy-And-Paste-able

Everyone knows that a lot of "programming" consists of copying your last project (or someone else's :-) ) and then tweaking it to do whatever you need to do next. In support of this, we should look for coding practices that make it easy to copy-and-paste code in such a way that the "programmer" doesn't have to go over every line with a fine-toothed comb to make sure there are no bugs. Because of JScript's loose typing (and very forgiving semantics), it is possible to copy and paste code that happens to "work" by mistake but isn't doing what you think it is. Days or weeks later you make a (seemingly) unrelated change elsewhere, and suddenly things stop working.

Argh!

So let's look at the code again:

Point.prototype.Initialise = function Point_Initialise(id, x, y)
{
Thing.prototype.Initialise.call(this, id);
this._x = x;
this._y = y;
};

If we're going to copy-and-paste this code a lot of times, we really want it to be as loosely-coupled to Thing as possible... and yet here we have a direct call into a Thing.prototype method right smack bang in the middle of our function (and imagine this is a large code-base where you have lots of methods that make calls to their base class' methods all over the place). The scary thing is that copied code will still work, even if the object you copied it into doesn't derive from Thing, because there exists a Thing and it has an Initialise method. Thing will just happily dork around with the this object, creating new expando properties if it needs to, or reading undefined out of ones that don't exist, and it probably won't crash or do anything terribly noticeable. But the fact is that it won't be doing what you intended it to do, and you might never know that it's not really doing what you expected until it's too late.

So how do we solve it? We remove any reference to Thing from inside our functions. And how do we do that? I'm glad you asked:

function Point()
{
this._x = -1;
this._y = -1;
}

Point.prototype = new Thing();
Point.prototype._className = "Point";
Point.prototype.constructor = Point;
Point._base = Thing.prototype;

Point.prototype.Initialise = function Point_Initialise(id, x, y)
{
Point._base.Initialise.call(this, id);
this._x = x;
this._y = y;
};

Aha! We add a new _base property to the Point object itself (remember, Point is an object too!) that points to the base object (in this case, Thing.prototype) and then we call that instead of Thing.prototype. OK, OK, so you may be saying "But Peter you twit, this didn't solve anything -- when I copy-and-paste my code, I could still end up with Point._base.prototype instead of Thing.prototype and the same problems will still ensue!"

How true! The difference, dear reader, is that I expect you are more likely to do a search-and-replace for "Point" when you copy this code than you are to search-and-replace "Thing" (remember, you're copying code from Point to YourNewObject), and so the reference to Point._base will be updated to YourNewObject._base without any hassles. Of course if you don't do any kind of search-and-replace, you have the same problem you always had... life is tough.

Why Being Smarter Won't Work

The "obvious" solution to the last problem is to put the _base property onto the this object -- rather than the constructor function itself -- then you can always refer to this._base.prototype and never have to worry about referring to the wrong thing ever again, right?

Wrong.

Exactly why it's wrong is a subject for another post (or perhaps a reader can comment...). Try it out for yourself, and see what happens (hint: you'll need more than two classes to see why it doesn't work).

Coming Up...

In the next instalment: a feature that some people claim is impossible with JScript. Just you wait and see!