One of the key programming concepts in HDi is the use of asynchronous processing with callback functions. The HDi platform exposed to title authors is not multi-threaded (which helps reduce the complexity of writing applications), but many of the functions of the player are asynchronous in nature. For example, all the I/O in HDi (both local and network) is asynchronous so that the on-screen experience isn't degraded while the script waits for a long I/O call to complete. Instead, the script asks the player for a resource and then just keeps on plugging away. At some future point in time the resource will become available, the player will notify the script via a callback function, and the script can act on the resource without any further waiting.

As is common in any programming environment, many developers have built "helper functions" around some of the built-in HDi functions in order to centralise error handling, simplify high-level tasks, or for other various reasons. This is all well and good, but I have seen some developers use a programming pattern for these helpers that will lead to bugs down the road. I'm going to show an example of this bad pattern (and how to avoid it), but first here's an example of many people's first attempt at asynchronous programming:

// Global variables to hold state
var done = false;
var result = 0;

// Copy a file and return when it is done
function CopyFileHelper(src, dest)
{
  // Copy the file, providing a simple callback function
  FileIO.copy(src, dest, true, IOFinishedHandler);

  // Wait until the callback is called
  while (!done)
    ; // nothing

  return result;
}

// Function to handle the callback and stop the loop
// inside CopyFileHelper
function IOFinishedHandler(errorInfo)
{
  result = errorInfo;
  done = true;
}

 

This code will never work, so luckily nobody has ever shipped a title like this (that I know of!). Nevertheless, it is interesting to learn why it doesn't work. The basic idea is to have a sentinel value (in this case, done) that is initially false. The FileIO operation is called and is provided a callback handler that will (i) stash the result of the I/O function for later use; and (ii) set the sentinel to true. The main function then performs a busy wait until the done variable is set to true by the callback, at which point the while loop ends and the function returns.

Although this kind of programming might work on a natively multi-threaded platform like .NET or Java, it won't work in HD DVD because of the way application time is scheduled. (Note it would also still be a bad idea on .NET or Java, but that's another story). In HD DVD, application code (including markup timing and script execution) is executed in a tick-based fashion. At the start of a tick, the necessary script code for that tick is executed (including event handlers, timers, and other callbacks) and then execution is passed to the markup layer. The markup layer performs timing and layout operations, and then the script is given another chance to run if it needs to handle any markup events. Finally, the markup page is generated and the system starts over again on a new tick. This all happens on a single thread. At the same time, other operations like A/V decoding and the FileIO APIs are happening on different threads.

What happens in this case is that the FileIO.copy function is called, a new thread is created to handle the copy operation, and then the script goes into the busy-wait loop. At some point in the future, the copy operation will finish and player will schedule the callback to happen on the next tick. But because the script code is busy waiting for done to become true, the current tick never ends and so your player's menu appears to hang. This is why nobody actually ships this code :-)

OK, so that's the impossibly-bad way of doing async IO. Here's the promised programming pattern that you should look out for and fix:

// Copy a file and do some common logging, etc
function CopyFileHelper(src, dest, clientCallbackHandler)
{
  // Perform some common, useful stuff...
  // ...
  // ...

  // Copy the file, providing the caller's callback handler
  // so they can continue when it is done
  try
  {
    FileIO.copy(src, dest, true, clientCallbackHandler);
  }
  catch (ex)
  {
    // Oops! Got an exception, so log it, convert to a FAILED result,
    // and let the caller know via their callback
    
    LogError("Exception while copying " + src + " to " + dest, ex);
    clientCallbackHandler(FileIO.FAILED);
  }
}

 

In this case, at least we're not trying to break the laws of physics. In most cases, this code will work fine. If the copy operation doesn't throw an exception, the client's callback is called and everyone is happy. If the copy operation does throw, we log the error and convert it into a failed result for their callback. This makes programming much easier for the client since they don't have to worry about exceptions anymore; they just treat all failures the same way... right?

Wrong.

Although it looks like you've made life simpler, you've just introduced a subtle bug that may not bite you until sometime in the future. The problem is that you've broken the semantics of the client's callback, which is that callbacks are always called asynchronously. You should never call a callback synchronously because the client code may not be expecting it.

Imagine the client has code like this:

// Perform a sequence of operations involving a file copy
function ClientCode()
{
  DoFirstThing();

  CopyFileHelper(src, dest, DoThirdThing);

  DoSecondThing();
}

function DoFirstThing() { /* ... */ }
function DoSecondThing() { /* ... */ }
function DoThirdThing(result) { /* ... */ }

 

Under normal circumstances (no exceptions), the order of function calls will be DoFirstThing, DoSecondThing, and then at some future tick, DoThirdThing. Even if there is an error while copying, the code will progress in this sequence and the client can happily deal with any copy failures in DoThirdThing, secure in the knowledge that DoSecondThing has already completed. But what happens when there is an exception (which may not have happened during testing)? Now the order of execution is DoFirstThing, DoThirdThing, DoSecondThing, all in the same tick. You have reversed the order of the last two steps!

If DoSecondThing performs a critical task that is needed before DoThirdThing can succeed, the code is now broken and won't operate properly even though DoThirdThing may be coded robustly to deal with the copy error. The solution is to simply schedule the client's callback to happen at a later time, thus preserving the execution order of the client's code. This is easily done with a single-frame, non-repeating timer. Here's an example of how you might do it:

  // ... same code as before ...
  catch (ex)
  {
    // Oops! Got an exception, so log it, convert to a FAILED result,
    // and let the caller know via their callback
    
    LogError("Exception while copying " + src + " to " + dest, ex);
    ScheduleCallback(ExceptionCallback);
  }

  // Closure used to pass arguments to the callback
  function ExceptionCallback()
  {
    try
    {
      clientCallbackHandler(FileIO.FAILED);
    }
    catch (ex)
    {
      // Client callback failed; nowhere to go from here
      LogError("Fatal exception while calling client callback", ex);
    }
  }
}

// Call the function on the next tick. If you need to pass arguments to
// the function, then pass a closure (nested function) to do the work
function ScheduleCallback(callback)
{
  var t = createTimer("00:00:00:01", 1, callback);
  t.autoReset = false;
  t.enabled = true;
}

 

In this new code, if an exception occurs then rather than calling the callback directly, we schedule it to be called on the next tick via a couple of helper functions. We need the ExceptionCallback helper function in order to pass arguments to the callback (timers never pass arguments themselves), and we use the ScheduleCallback helper to avoid closures. This ensures that the client's code always executes in the order expected, and thus reduces your chances of having weird bugs in the future.