HD DVD / Randomness...

What Now?

ptorr — Wed, 20 Feb 2008 22:18:46 GMT

I have had some readers ask what is to become of this blog. Well, it started out discussing Office Development, then it covered Threat Modelling, and most recently it has been about HDi. There have been several posts about the Pet Shop Boys and then there was this little post, too. That's the cool thing about Microsoft; you work on one thing for a while, and if it doesn't work out (or if you decide you want to try something new) then you just go do it.

One thing's for sure – I'm not about to start posting about BD-J authoring :-)

I still have a few script-based posts to finish, but after that I don't know how often I will update the blog because most likely I will be working on the Next Big Thing, and most likely that won't be public (and thus not blog-able). Anyway, thanks for reading and if you choose to stick around then maybe I'll post something cool again in a few months...

http://www.hddvdforever.com/

Peter.

Fun Friends for Functions, Part 2: Handling Exceptions

ptorr — Sat, 02 Feb 2008 04:24:40 GMT

Previously we looked at how you can bind a function to a set of arguments and then call the resultant function at some time in the future. This technique is very powerful, but as-is it is a ticking bomb. As Amy points out in her blog post on exception handling, you always need to be prepared to handle an exception in your code. Failure to catch an exception usually leads to playback stopping and an error diagnostic from your player.

The trouble is that when you start using bound functions, you don't always have an easy way to deal with exceptions. Typically, a bound function is used as part of an asynchronous callback, and in those cases you don't have any functions further up the stack to save you from unhandled errors. Simple, you might think, I'll just wrap the contents of my function in a try-catch and be done with it.

Not so fast.

Let's look at the example function we were using in the previous blog post:

// function to set a backgroundImage
function SetBackgroundImage(markupId, url)
{
document[markupId].style.backgroundImage = url;
}

This could throw for several reasons – for example, an element with the markupId doesn't exist, or url isn't a correctly-formatted property value. Simply wrapping this up in a try-catch would result in something like this:

// function to set a backgroundImage
function SetBackgroundImage(markupId, url)
{
  try
  {
    document[markupId].style.backgroundImage = url;
  }
  catch(ex)
  {
    // What do we do here?!?
  }
}

But there's a problem – what do we do when we catch the exception? I don't know. And that's a key thing about exception handling:

Functions should only handle exceptions if they are actually prepared to deal with them. If a function doesn't know what to do in an error condition, it should let its caller figure it out. The exception [groan] to this rule is that your top-level code should always swallow exceptions (unless of course things are so out of whack that you think the player should stop immediately).

In this case, the SetBackgroundImage function has no idea what a failure means. It might be perfectly OK if the background image can't be set correctly, or it could be totally disastrous. It all depends on the context, so the SetBackgroundImage function should remain silent on the topic of exceptions and let the calling function figure out what (if anything) to do with them.

In our particular case, let's say that the exception is benign; we are simply updating some eye-candy graphics and it doesn't matter if they fail to appear. So then: maybe we should write a wrapper function, SafeSetBackgroundImage, that does this for us?

// function to set a backgroundImage, ignoring exceptions
function SafeSetBackgroundImage(markupId, url)
{
  try
  {
    SetBackgroundImage(markupId, url);
  }
  catch(ex) { }
}

I actually like to use this approach if I am going to be calling the 'Safe' version in a lot of different places, but if it is only in one or two places that seems like too much overhead to someone lazy like me. Anytime you update the parameter list of the wrapped function, you have to update the wrappers, too. It can also hide the intent of your code by sprinkling endless wrapper functions all over the place, and this also has the productivity-reducing effect of polluting IntelliSense with unnecessary function names. Instead, we can use the same kind of nested-function trick we used in the argument-binding case to get a "safe" version of any function:

// return a version of a function that is wrapped in try-catch
// you can optionally provide an errorFunc, which will be
// called with the 'tag' parameter in case of an exception
function Function_WrapInTryCatch(errorFunc, tag)
{
// stash the target function
var func = this;

// return the nested function
return wrappedFunction;

Function.prototype.WrapInTryCatch = Function_WrapInTryCatch;

In this example, we also allow the caller of the WrapInTryCatch function to provide an optional error-handling function that will be called if an error occurs. This allows you to do things like channel all your async exceptions to the same error handling function, which can figure out what to do based on the value of the tag parameter. Now we can re-write the last couple of lines of the example as:

// Call the download function, passing in the set-image function as the callback handler
var setBackgroundHandler = SetBackgroundImage.BindToArguments("button1", "file:///filecache/bar.png");
var wrappedHandler = setBackgroundHandler.WrapInTryCatch();
DownloadFile("http://foo/bar.png", "file:///filecache/bar.png", wrappedHandler);

This helper has the same limitations as the BindToArguments function (specifically: you can't pass any arguments to the returned function, and any tracing information is lost if 'errorFunc' is called) but this is also a solvable problem.

Why the ‘arguments’ object is bad, Exhibit #3,745

ptorr — Thu, 31 Jan 2008 00:46:52 GMT

Having just written a post with an example of how to use the arguments object, you might question my choice of title for this blog post. In my not-so-humble opinion, the 'arguments' object in ECMAScript is only useful for one thing:

The 'arguments' object should only be used by infrastructure functions that simply perform book-keeping on behalf of other functions. Any function that actually operates on its arguments in a meaningful way should never access the 'arguments' object.

The reason for this is plain and simple: code should be easy to read, easy to debug, and easy to maintain. Saving yourself a small amount of effort while writing code is a very bad trade-off if it negatively impacts these other three important tasks. Using the 'arguments' object in any normal function might save you a few keystrokes in the short-term, but will come back to bite you later on.

The most obvious problem with the arguments object is this: You have some code you wrote a few [days|weeks|years] ago and now you have to go and update it. You see some function calls like this in source code:

SomeBadFunction(42);
// ...

SomeBadFunction(i, current);
// ...

SomeBadFunction(next, data, OtherFunction(), moreData);
// ...

You think the problem might be related to one of the parameters being passed to SomeBadFunction, so you decide you need to check it out:

function SomeBadFunction()
{
// 200 lines of code
}

Oh dear. The definition of SomeBadFunction gives us absolutely no idea how many parameters it expects, what order they should be in, nor what values they might represent. Maybe there's a comment at the top of the function that explains what you should pass it, but we all know that comments get out of date very quickly and the code is the only real way to know what is going on. So now you have to trawl through several hundred lines of code, trying to reverse-engineer what kind of parameters the function expects. This could take a long time, and you might get it wrong.

Let's imagine that the real intent of SomeBadFunction is to update an internal table. The first parameter is the index of the item in the table to update, and the second parameter is the value to place in the table. The function supports rudimentary format strings (like printf() in C) and so if the value contains a format-specifier like "%d" or "%s" then the remaining parameters are used to fill in the blanks. Finally, if the value is omitted then the table entry is removed.

In this case, I would define the function something like this:

function SomeBadFunction(index, value, formatArgs)
{
// probably fewer than 200 lines of code
}

Now it is clear that SomeBadFunction expects three arguments, the first should probably be a number, the second should be some generic value, and the third should be an array of values (by using the naming convention ...Args; you could also call it formatArgArray or paramarray or varargs or something else if you feel that is clearer).

ECMAScript already allows for omitting function arguments, and code is typically written to have some "obvious" behaviour if parameters end up being undefined, so simply dropping value or formatArgs from a function call is quite OK. Now the only thing you need to change when you call the function is that if – if – you want to use the special format-string ability of SomeBadFunction, you simply encase the trailing arguments in square brackets to put them in an Array:

SomeBadFunction(next, data, [OtherFunction(), moreData]);

Anyway, I promised you an exhibit of why 'arguments' is bad – a bona-fide bug in ECMAScript, not just the rambling opinions of some Microsoft fuddy-duddy. Take this code:

function foo(x, y)
{
  print("x: " + x + ", y: " + y);
  Array.prototype.shift.call(arguments);
  print("x: " + x + ", y: " + y);
}

foo(42, 123);

What do you expect it to display? Don't ruin the fun by running it and posting the actual answer – just post what you think it is.

I say this is a bug in ECMAScript – rather than a bug in JScript – because section 10.1.8 of the ECMA 262 spec doesn't completely define how the arguments object interacts with the parameters of the function, and so you get some strange results. Note that both IE and Firefox give you the same result (replace 'print' with 'alert') but I doubt both companies independently came up with the same unexcpected result. I don't know if this is a case of IE being bug-compatible with the original Netscape Navigator code, or Firefox being bug-compatible with IE code, or a bit of both. I wasn't around when this was happening :-)

Fun Friends For Functions

ptorr — Wed, 30 Jan 2008 03:45:34 GMT

And now for a more techy post. In ECMAScript, functions are considered first-class objects. "What," you might ask, "differentiates a 'first-class' object from a merely 'economy-class' object?" Well, the main thing is that functions in ECMAScript are treated just like any other type of data (strings, numbers, arrays, etc.) in that they can be assigned to variables, they can be passed as function arguments, and they can have properties and methods of their own. They also have a prototype object, just like all other ECMAScript objects. The upshot of this is that you can do fun things with functions in ECMAScript that you can't do in other languages such as C or Java.

In this entry I will introduce four types of "helper" functions that are ideally suited to HDi development. They are:

Binding a function to a set of arguments and / or a 'this' object
Wrapping an existing function in an exception handler
Chaining function calls together one after the other
Calling a function after a specified period of time

In all cases, you write the actual function (the one that does the interesting work) just as you normally would, but when it comes time to call that function you do something funky with it to make some magic happen. Exactly why you might want to do this will become clearer soon, but it mostly comes down to how you write event handlers and callbacks.

A common theme in this post will be that you want to avoid closures as much as possible, and using these helpers should get you a long way to doing that (even though these helper functions rely on closures, they do so in a "safe" manner). And although there are various ad-hoc ways to accomplish the same tasks, generally these require writing specialised wrapper functions or following ugly coding practices (which I might blog about in the future) when what I would prefer to focus on is writing highly readable, highly re-usable code and utilising some clever, general-purpose helper functions to take care of all the tricky dirty work.

Binding to arguments

Many operations in HDi use a callback function to notify you when a specific task is completed (such as a File I/O operation or an event handler). Oftentimes you will want to associate more information with your callback than the HDi API actually allows for, and there are a couple of ways to accomplish this. But first, an example:

Say you have an application where you want to dynamically download some images from the network and display them inside a markup page. The basic code you want to write is probably like this (the code is simplified for clarity; obviously there is some additional code to convert the onStateChange callback into a single "the download is complete" call, etc.):

// function to download an image
function DownloadImage(src, dest)
{
  var httpClient = Network.createHttpClient(src, dest);
  httpClient.onStateChange = SetBackgroundImage;
  httpClient.send();
}

// function to set a backgroundImage
function SetBackgroundImage(markupId, url)
{
document[markupId].style.backgroundImage = url;
}

// call the download function... but what next?
DownloadImage("http://foo/bar.png", "file:///filecache/bar.png");

Note that both functions have been written to be "highly readable, highly re-usable" and do one thing in a very straight-forward manner. The problem, of course, is that this code won't work because there is no way for the callback to SetBackgroundImage to communicate what markupId or the final download url is supposed to be.

One common way of solving this problem is to use a nested function, like this:

// function to download an image and set it as the background URL
function DownloadBackgroundImage(src, dest, markupId)
{
  var httpClient = Network.createHttpClient(src, dest);
  httpClient.onStateChange = SetBackgroundImage;
  httpClient.send();

  // nested function to set a backgroundImage
  function SetBackgroundImage()
  {
    document[markupId].style.backgroundImage = dest;

    // Clean up the memory
    httpClient = null;
  }
}

// call the download function and update the element 'button1'
DownloadBackgroundImage("http://foo/bar.png", "file:///filecache/bar.png", "button1");

Note that even though we've done work to ensure the closure won't memory leak memory, there is still at big problem in that neither the download function nor the background setting function are generic any more. If you want to (eg) download an XML file, or if you want to set the backgroundImage to a file on persistent storage, you will need to either add some more hacks to this function (making it less readable and less maintainable) or you will need to duplicate the code into a similar-but-not-entirely-the-same function.

The solution I prefer is to use the fact that functions are first-class objects and solve this problem using "argument binding." In this case, the code would be re-written as follows:

// function to download a file and then call 'handler' when it is done
function DownloadFile(src, dest, handler)
{
  var httpClient = Network.createHttpClient(src, dest);
  httpClient.onStateChange = handler;
  httpClient.send();
}

// function to set a backgroundImage
function SetBackgroundImage(markupId, url)
{
document[markupId].style.backgroundImage = url;
}

// Call the download function, passing in the set-image function as the callback handler
var setBackgroundHandler = SetBackgroundImage.BindToArguments("button1", "file:///filecache/bar.png");
DownloadFile("http://foo/bar.png", "file:///filecache/bar.png", setBackgroundHandler);

Now both the download function and the set-image function are still written in a clear, simple, re-usable fashion, but we can easily chain them together to get the desired result. Even the call that makes this possible (the BindToArguments call) is straight-forward and understandable. All that is missing is the magic BindToArguments helper function... which looks like this:

// helper function to return a copy of the target function, but
// bound to the original arguments
function Function_BindToArguments()
{
  // store a reference to the target function and its arguments
  var storedFunction = this;
  var storedArguments = arguments;

// return the nested function
return boundFunction;

  // nested function simply invokes the stashed values
  function boundFunction()
  {
    storedFunction.apply(null, storedArguments);
  }
}

// make the helper available to all functions
Function.prototype.BindToArguments = Function_BindToArguments;

As noted in the comments, this helper function simply stores a reference to the "target" function and the arguments that are passed to the BindToArguments call. Then it returns a nested function, which when called will invoke the stored function (SetBackgroundImage) with the stored arguments ("button1" and "file:///filecache/bar.png"). Note that the apply function is a built-in function of ECMAScript Function objects that invokes a given function with the specified 'this' object and arguments array.

This is the most basic form of a "binding" helper, and the resultant function is one that has no 'this' object and that cannot take any arguments of its own. It also can't participate in a typical logging infrastructure because much of the required metadata (original function name, number of expected arguments, etc.) is missing. The good news is that all this can be added with the help of more code, and hopefully that's something I'll be posting soon.

As usual, this is longer (and took longer) to write, so I'll have to save the other "friends" for later. The depressing thing is that actually writing the code is the easy part (and it's all done already); it's wrapping it all up in a post and explaining how things work that take forever!

A Concert with PowerPoint Slides?

ptorr — Tue, 29 Jan 2008 20:36:56 GMT

As you may know, Howard Jones played an acoustic show in Seattle last night with Robin Boult on guitar. The performance was fantastic, and the venue (The Triple Door) was actually quite nice with a limited menu coming from the kitchens of Wild Ginger. (Although I enjoyed the food, I am not really a fan of having the waiters walk past every 5 minutes, or of all the clanging of silverware and glassware as people finish their dessert... but I digress).

Anyway, the reason for this blog post is that before many of the songs, Howard gave a little bit of background about the song – what it was about, how it was written, previous live performances, etc. – and at one point joked that he needed a PowerPoint deck to help tell his story. And this got me thinking about what we are doing with interactivity in HD DVD with HDi: although the movie (or TV show or concert performance) is central to any home viewing experience, it is really the "extra" stuff that sets HD DVD away from current DVD or broadcast TV.

A High Definition picture is nice, and lossless surround-sound is pretty sweet too, but as a filmmaker myself I know that there is always more to the story than you can tell in the movie (TV show; song; etc.) itself. And as a movie lover, I know there is often more that I want to know about the story than is available in the movie (...) itself. Standard DVD has the usual staple of vignettes (deleted scenes, making-of documentaries, trailers) but for various reasons they tend to get skipped by many viewers.

You might watch a 30 minute documentary in the hopes of learning more about (eg) the location a movie was shot, only to be disappointed at the end when no information is made available. And since DVD is "set in stone" once it goes to the replicator, there's no way for you to ask them to add that information later. Or perhaps a DVD extra provides a filmography or a gossip about the actors in the movie, but by the time the disc hits your DVD player it is woefully out of date. Problems such as these leave many people underwhelmed by DVD extras and lead to attitude that they're of little value... even though if you actually sit down and talk to those same people, many of them really do want more out of their disc-watching experiences, especially for content they care about (favourite movies, TV shows, or music artists) – and these are exactly the kind of discs they would buy!

With technologies such as HDi in HD DVD, the producers can add all manner of information to the disc and let "you control" which bits of information you to see, and when. No more sitting through hours of bonus material in the hope that maybe you'll get an out-dated snippet of what you're interested in. With true interactivity, you can go straight to the information you want and can be confident that it is up-to-date via connections to the internet. And if the information you want isn't there, why not send feedback directly to the producers that you want to see it added in the future?

On HD DVD, Howard Jones could have his PowerPoint slides (and then some!) and you – the viewer – could decide seamlessly and instantly whether you wanted to dive into the background or just enjoy the great music (or maybe do both at the same time). That's something you can't do on a standard DVD, and it's also something that doesn't require a gigantic TV, elaborate surround-sound system, or expensive disc player to appreciate.

Whenever people write off interactivity as a gimmick or something nobody needs, I think fondly back to the day when a University friend first showed me the World Wide Web. I just shrugged... why would anyone need that? I could get exactly the same information through Gopher, only faster!

How things change.

Good News

ptorr — Mon, 14 Jan 2008 23:02:10 GMT

Not one but two bits of good news this morning!

Firstly, TWICE reported on Toshiba's plans for HD DVD this year. Probably the biggest news was that the entry-level HD-A3 player will have a new recommended price of only $149 and that the higher-end models will get similar price cuts.

And in other news: Howard Jones is coming to Seattle!

What I’ve Learnt about Coffee Lids

ptorr — Mon, 17 Dec 2007 08:04:50 GMT

When I first moved to the US over eight years ago, I thought it was very odd that people put lids on their take-away coffee. I found it quite strange to drink through a hole in a lid rather than to just drink directly from the cup, and routinely discarded the lids in order to enjoy my beverage the old-fashioned way. I thought it odder still that people would walk and drink their coffee at the same time, but it didn't take long for me to realise that this was a supremely useful activity afforded by the afore-mentioned lids.

The annoying thing was that although the lids generally did a good job of keeping coffee in its place, every so often a small amount of coffee would escape from the cup and drip onto my clothes. And with Murphy's Law in full effect, it typically happened when I had on a clean white shirt. Before a meeting.

Initially I thought it was just random manufacturing defects in the lids, or that the lids weren't on well enough. But one day I investigated further into the drippage and uncovered the culprit: the seam in the cup where the cardboard is joined together. If the lid is placed in such a way that the mouthpiece is more-or-less directly over the cup's seam, chances are that the seal will not be completely air-tight, and when you tilt the cup to take a drink the coffee will drip out of the cup and onto your freshly-laundered shirt. The solution? Simply rotating the lid to ensure the mouthpiece is directly opposite the seam appears to solve the problem – at least it has never happened to me since I started being more vigilant about lid placement.

I've never really thought this was much of a revelation – it seems like common sense, really – but having explained it to two friends recently I decided maybe it's not that obvious after all, and hence this completely non-HD-DVD-related blog post :-)

HD DVD Emulator for Xbox 360!

ptorr — Fri, 14 Dec 2007 04:22:15 GMT

It's finally here! The Xbox 360 HD DVD Emulator went live today, enabling content authors to test their HD DVD content in a real-world consumer player before committing it to a shiny disc. This can save a lot of time and a lot of money for those folks building advanced HD DVD titles. It might help you save your hair, too :-)

"But," you might be saying, "doesn't Microsoft already ship an HD DVD Simulator for free? Why would I want this emulator?"

Good question. Here are just a few reasons why:

The simulator only plays Windows Media files, and poorly at that. The emulator plays real multiplexed EVOB files (with VC-1 and AVC and TrueHD etc.) that you will ship on the real disc.
The simulator doesn't support subtitles or multiple audio tracks. The emulator does, enabling you to actually test them.
The simulator shows you how your code performs on your desktop PC. The emulator shows you how it performs on real shipping hardware.
The simulator has some primitive logging functionality, but you need to do some work yourself. The emulator has very robust logging for all aspects of HDi playback (including markup timing!).

There are obviously more reasons to use the emulator, but that's a quick rundown of some of the key reasons why you might want to do it. Of course the simulator is still great for doing early title development – where access to real video and multiple audio tracks might not matter – because using the PC enables a great edit-run-debug workflow. But the emulator is absolutely critical for putting everything together, fine-tuning your code, verifying performance and compatibility, and running a through a thorough test pass prior to check-disc replication.

“Open Season” Hits #2; World Scratches Head

ptorr — Wed, 05 Dec 2007 05:56:28 GMT

If you follow the High-Definition format war at all, you're probably familiar with the weekly sales breakdown published in Home Media Magazine and other places. If you checked them last week, you would have seen something strange.

Something very strange.

At #1, you have Live Free or Die Hard on Blu-ray. This makes sense, since it is both a new release and the action genre fits the "PlayStation 3" demographic of many Blu-ray disc buyers, but then at #2 we see the animated comedy Open Season. Now if you took a double-take on reading that one, you're not alone. Open Season doesn't appear anywhere on last week's standard DVD sales or rentals charts, nor does it appear on the electronic sell-through and download charts. In fact, Open Season hasn't appeared on any of the Nielsen VideoScan charts for months – if ever. A quick jump over to Amazon shows why it isn't on any of the regular charts – it was released on January 30 2007, and with an estimated US box office of only $84m it is highly unlikely anyone would be rushing to buy the title almost a year after it was released. So what gives?

Well, look no further than this Best Buy advertisement, which bundled Open Season with the 40Gb PlayStation 3 for Thanksgiving. Everyone who bought a PlayStation 3 at Best Buy got a free copy of the movie, and those give-aways also got counted as "sales" towards the numbers reported each week in Home Media Magazine. This is a smart move for the BDA, which gets to give away a movie the customer wasn't going to buy anyway (hence no lost sales), plus they get to inflate the number of discs "sold" and put out a splashy "7 out of 10 discs were Blu-ray" ad.

This is one clear anomaly in the VideoScan data, but if you consider how many bundles, freebies, give-aways, BOGOs (Buy-One-Get-One Free), etc. have been offered by the Blu-ray Disc studios, you can start to draw your own conclusions.

Oh, and before you ask "What about the 5 free movies^[PDF] the HD DVD Promotion Group is offering?" the answer is easy – those don't count in the weekly sales data because they are mail-in rebates; only things physically scanned at the checkout get counted (irrespective of whether they are free or not). This means that, for example, the copy of King Kong bundled with the Xbox HD DVD add-on also doesn't get counted, nor does the copy of Spider-Man 3 bundled with the PlayStation 3. Only barcodes of discs scanned at registers get counted, even if the customer doesn't have to pay for them.

Maintaining Callback Semantics

ptorr — Wed, 24 Oct 2007 21:29:55 GMT

One of the key programming concepts in HDi is the use of asynchronous processing with callback functions. The HDi platform exposed to title authors is not multi-threaded (which helps reduce the complexity of writing applications), but many of the functions of the player are asynchronous in nature. For example, all the I/O in HDi (both local and network) is asynchronous so that the on-screen experience isn't degraded while the script waits for a long I/O call to complete. Instead, the script asks the player for a resource and then just keeps on plugging away. At some future point in time the resource will become available, the player will notify the script via a callback function, and the script can act on the resource without any further waiting.

As is common in any programming environment, many developers have built "helper functions" around some of the built-in HDi functions in order to centralise error handling, simplify high-level tasks, or for other various reasons. This is all well and good, but I have seen some developers use a programming pattern for these helpers that will lead to bugs down the road. I'm going to show an example of this bad pattern (and how to avoid it), but first here's an example of many people's first attempt at asynchronous programming:

// Global variables to hold state
var done = false;
var result = 0;

// Copy a file and return when it is done
function CopyFileHelper(src, dest)
{
// Copy the file, providing a simple callback function
FileIO.copy(src, dest, true, IOFinishedHandler);

  // Wait until the callback is called
  while (!done)
    ; // nothing

return result;
}

// Function to handle the callback and stop the loop
// inside CopyFileHelper
function IOFinishedHandler(errorInfo)
{
result = errorInfo;
done = true;
}

This code will never work, so luckily nobody has ever shipped a title like this (that I know of!). Nevertheless, it is interesting to learn why it doesn't work. The basic idea is to have a sentinel value (in this case, done) that is initially false. The FileIO operation is called and is provided a callback handler that will (i) stash the result of the I/O function for later use; and (ii) set the sentinel to true. The main function then performs a busy wait until the done variable is set to true by the callback, at which point the while loop ends and the function returns.

Although this kind of programming might work on a natively multi-threaded platform like .NET or Java, it won't work in HD DVD because of the way application time is scheduled. (Note it would also still be a bad idea on .NET or Java, but that's another story). In HD DVD, application code (including markup timing and script execution) is executed in a tick-based fashion. At the start of a tick, the necessary script code for that tick is executed (including event handlers, timers, and other callbacks) and then execution is passed to the markup layer. The markup layer performs timing and layout operations, and then the script is given another chance to run if it needs to handle any markup events. Finally, the markup page is generated and the system starts over again on a new tick. This all happens on a single thread. At the same time, other operations like A/V decoding and the FileIO APIs are happening on different threads.

What happens in this case is that the FileIO.copy function is called, a new thread is created to handle the copy operation, and then the script goes into the busy-wait loop. At some point in the future, the copy operation will finish and player will schedule the callback to happen on the next tick. But because the script code is busy waiting for done to become true, the current tick never ends and so your player's menu appears to hang. This is why nobody actually ships this code :-)

OK, so that's the impossibly-bad way of doing async IO. Here's the promised programming pattern that you should look out for and fix:

// Copy a file and do some common logging, etc
function CopyFileHelper(src, dest, clientCallbackHandler)
{
  // Perform some common, useful stuff...
  // ...
  // ...

  // Copy the file, providing the caller's callback handler
  // so they can continue when it is done
  try
  {
    FileIO.copy(src, dest, true, clientCallbackHandler);
  }
  catch (ex)
  {
    // Oops! Got an exception, so log it, convert to a FAILED result,
    // and let the caller know via their callback

    LogError("Exception while copying " + src + " to " + dest, ex);
    clientCallbackHandler(FileIO.FAILED);
  }
}

In this case, at least we're not trying to break the laws of physics. In most cases, this code will work fine. If the copy operation doesn't throw an exception, the client's callback is called and everyone is happy. If the copy operation does throw, we log the error and convert it into a failed result for their callback. This makes programming much easier for the client since they don't have to worry about exceptions anymore; they just treat all failures the same way... right?

Wrong.

Although it looks like you've made life simpler, you've just introduced a subtle bug that may not bite you until sometime in the future. The problem is that you've broken the semantics of the client's callback, which is that callbacks are always called asynchronously. You should never call a callback synchronously because the client code may not be expecting it.

Imagine the client has code like this:

// Perform a sequence of operations involving a file copy
function ClientCode()
{
DoFirstThing();

CopyFileHelper(src, dest, DoThirdThing);

DoSecondThing();
}

function DoFirstThing() { /* ... */ }
function DoSecondThing() { /* ... */ }
function DoThirdThing(result) { /* ... */ }

Under normal circumstances (no exceptions), the order of function calls will be DoFirstThing, DoSecondThing, and then at some future tick, DoThirdThing. Even if there is an error while copying, the code will progress in this sequence and the client can happily deal with any copy failures in DoThirdThing, secure in the knowledge that DoSecondThing has already completed. But what happens when there is an exception (which may not have happened during testing)? Now the order of execution is DoFirstThing, DoThirdThing, DoSecondThing, all in the same tick. You have reversed the order of the last two steps!

If DoSecondThing performs a critical task that is needed before DoThirdThing can succeed, the code is now broken and won't operate properly even though DoThirdThing may be coded robustly to deal with the copy error. The solution is to simply schedule the client's callback to happen at a later time, thus preserving the execution order of the client's code. This is easily done with a single-frame, non-repeating timer. Here's an example of how you might do it:

  // ... same code as before ...
  catch (ex)
  {
    // Oops! Got an exception, so log it, convert to a FAILED result,
    // and let the caller know via their callback

    LogError("Exception while copying " + src + " to " + dest, ex);
    ScheduleCallback(ExceptionCallback);
  }

  // Closure used to pass arguments to the callback
  function ExceptionCallback()
  {
    try
    {
      clientCallbackHandler(FileIO.FAILED);
    }
    catch (ex)
    {
      // Client callback failed; nowhere to go from here
      LogError("Fatal exception while calling client callback", ex);
    }
  }
}

// Call the function on the next tick. If you need to pass arguments to
// the function, then pass a closure (nested function) to do the work
function ScheduleCallback(callback)
{
  var t = createTimer("00:00:00:01", 1, callback);
  t.autoReset = false;
  t.enabled = true;
}

In this new code, if an exception occurs then rather than calling the callback directly, we schedule it to be called on the next tick via a couple of helper functions. We need the ExceptionCallback helper function in order to pass arguments to the callback (timers never pass arguments themselves), and we use the ScheduleCallback helper to avoid closures. This ensures that the client's code always executes in the order expected, and thus reduces your chances of having weird bugs in the future.

Sliced bread ain’t got nuthin’ on this!

ptorr — Tue, 02 Oct 2007 21:49:21 GMT

Forget being the best thing since sliced bread – this is the best thing ever. Well maybe not quite that good, but it's still darn cool!

Finally – FINALLY! – you can get decent IntelliSense for JScript inside Visual Studio!

Holy hotcakes, Batman! This is so awesome you have to download Visual Studio 2008 Beta 2 right now and try it out.

Back when I was on the script team, we couldn't get any features into Visual Studio (much to the chagrin of our customers) because all the hype was around ASP.NET. The rich client was dead. It was all about the server. Nobody needed client-side script any more. Everything was a web form post-back.

But now that AJAX and Silverlight and all these other buzzwords are suddenly important, people taking notice of the little script language that could.

Congrats and thanks to the Visual Web Developer and JScript teams – at last, JScript is getting some of the respect it has always deserved! :-)

Wikipedia editing

ptorr — Sun, 30 Sep 2007 21:43:59 GMT

Short post: In the interests of full disclosure, I've started editing Wikipedia on various topics related to HD DVD. Initially I made several "anonymous" edits, but decided to create an account for transparency and so I can "watch" pages as they change.

Basically I am just posting this here so nobody "discovers" my edits in the future and claims this is some "hidden agenda" by Microsoft to "lend credibility" by faking "grass roots" spin. Whatever.

Anyway, I'll still be doing the majority of my posting ("Haha," some of you might say, "not so much lately!") on this blog. The Wikipedia editing is mostly about keeping the pages interesting, up-to-date, and accurate.

Why Old Films Still Benefit from High-Def

ptorr — Thu, 27 Sep 2007 00:32:56 GMT

Every now and then, I hear people make comments like "Movies more than 5 years old won't look any better in HD because they were shot before HD was invented – the movie studios are lying to you about better quality!"

This is horribly, horribly, horribly wrong, and I'm not quite sure where this idea stems from.

A "Full HD" image (as the marketing people like to call it) is 1920 pixels wide by 1080 pixels high. Now that sounds like an awful lot of pixels (especially compared to Standard Definition) but it's small compared to the "resolution" of a good film stock.

Although film doesn't have well-defined "pixels" like digital video does – it is based on tiny particles inside the film's chemical structure – you can get an approximation to digital pixels and the current thinking is that it is around 6- to 8- thousand pixels horizontally (not vertically) – something the marketing folks would probably call "Ultimate HD 4320p" or some other silly name.

This brings up an interesting difference in how resolution is counted – in the consumer world, resolution is counted vertically – 480, 720, and 1080 all refer to the number of rows of pixels. But in the digital cinema world, it is counted horizontally – 2K and 4K refer to the number of columns of pixels. So your "1080p" TV is pretty close to a "2K" system in the digital cinema world (although not quite).

What this means is that pretty much any movie or TV show shot on a 35mm (or larger) film stock has far more resolution than even "Full HD" can show, and so old movies still get a big improvement by going to HD DVD (or Blu-ray) versus standard DVD. It all depends on the quality of the master and the time and effort taken in transferring it from film to digital and then compressing it for release.

You want proof? Look at the review of Casablanca (1942) for an example of how old films can look simply stunning in high definition.

[Update]

Another thing to consider is that despite the fact that film has more resolution than "Full HD," there are lots of problems with the way film is typically viewed that can dampen the experience. Prints get scratched. Projectors get out of focus. Screens get dirty. Theatres dim the light bulbs to save money. All of these things can lead to a poor-quality film presentation, so with a good HD transfer and a well-calibrated display you might even see more on an HD DVD disc than you would in the average supermall Cineplex.

Using Timers Effectively in HDi

ptorr — Thu, 30 Aug 2007 05:21:47 GMT

The use of timers in HDi applications is very common, and there are a few things you should keep in mind when using them to avoid performance issues when playing back your title.

I will categorise these as:

The autoReset problem
The closure problem
The re-use problem
The general-abuse problem

The AutoReset Problem

This problem stems from the fact that, by default, timers created in HDi are repeating timers, meaning that they will fire over and over and over again until you tell them to stop. Sometimes this is what the content author wants, but more often than not what they really need is a single notification after the timeout period has elapsed (what we call a "one-shot" timer). It's essentially the difference between setTimeout and setInterval in the browser DOM.

If you want a one-shot timer, the following pattern should be avoided:

// Create a timer; note it auto-repeats by default
var t = createTimer(TIME, CLOCK, callback);
t.enabled = true;

function callback()
{
// do stuff...

// Now disable the timer; sometimes people forget this step!
t.enabled = false;
}

Instead, consider the more straightforward (and less bug-prone) version:

// Create a timer and make sure it only fires once
var t = createTimer(TIME, CLOCK, callback);
t.autoReset = false;
t.enabled = true;

function callback()
{
// do stuff...
}

Sometimes, the problem is compounded because the author assumes the timer is a one-shot timer, and that in order to get repeating semantics they need to re-create the timer every time. That results in code like this:

// Create a timer; note it auto-repeats by default
var t = createTimer(TIME, CLOCK, callback);
t.enabled = true;

function callback()
{
// do stuff...

  // Now re-create the timer to ensure we get called again
  t = createTimer(TIME, CLOCK, callback);
  t.enabled = true;
}

Now what happens is that every time the timer fires, a new timer is created to fire on the same schedule! So after the first callback, there are two timers. After the second callback, there are four timers. After the third callback, there are eight timers, and so on. Eventually the old timers will get garbage collected by the script engine (and stop firing), but this could take a long time to happen depending on what else your script is doing. This is clearly a waste of resources and should be avoided; if you need a repeating timer, leave autoReset at its default value and don't re-create the timer later on (although I prefer explicitly setting autoReset to true to convey the intent that it really should repeat). Otherwise, set autoReset to false at the time you create the timer, and you will be happy.

The Closure Problem

In ECMAScript, a "closure" is a geeky name for a nested function. The reason we call them closures is to do with the way they affect the lifetime of other objects. I don't want to go into great detail now, but essentially a nested function maintains a reference to all the local variables of its containing function, thereby "closing them off" from being released by the garbage collector. Normally this isn't a problem for ECMAScript, because sooner or later the closure will die and thus the other objects it has been keeping references to will also die.

For example:

function MakeAClosure()
{
// Gobble up some memory
var bigObj = CreateECMAScriptObjectThatUsesLotsOfMemory();

// Deliberately set up a circular reference
bigObj.circularReference = Closure;

// Return the closure
return Closure;

  // Define the nested function (closure)
  function Closure()
  {
    // doesn't really matter what Closure does;
    // it still keeps a referenceto bigObj
  }
}

function foo()
{
  // Get the closure
  var closure = MakeAClosure();
  // maybe use it, maybe not...
}

// Call foo
foo();

In this example, we have an outer function (MakeAClosure), a nested function (Closure), another global function (foo), and some global code that calls foo. Here's what happens during execution:

foo gets called
foo calls MakeAClosure
MakeAClosure creates a large object that uses up some memory, and stashes it in bigObj
- The nested function Closure gets a reference to bigObj for reasons that I won't get in to here (check sections 10.2.3, 13, and 13.2 of ECMA-262 for technical details... or I can post more later :-))
MakeAClosure explicitly makes bigObj hold a reference to Closure so they both refer to each other – a classic circular reference problem
MakeAClosure returns the Closure Function object back to Foo
- Note that this includes the reference to bigObj
foo stores the Function object inside the closure variable
foo exits
Sooner or later, the garbage collector runs and sees that bigObj is used by closure, and closure is used by bigObj, but there is no way to access either of them (they are not "reachable") so it can safely kill both of them

Here everything is fine and dandy. But if bigObj where a non-ECMAScript object (eg, a Host object), the memory would never be reclaimed in Step 8 because the garbage collector cannot prove that bigObj is unreachable – maybe some other part of the system still relies on it – and so it has to assume that closure is still reachable, too. And since closure holds on to bigObj, the native object is never going to release itself, and the memory is leaked.

Perhaps it's clearer with an example.... using Timers, of course :-)

// Perform two animations, one after the other
function DoTwoPartAnimation()
{
// Perform some animation over 1 second
document.foo.animateProperty("y", "0px;100px", 1);

  // Setup a callback to do the other animation after the first is done
  var t = createTimer("00:00:01:00", 1, DoPartTwo);
  t.autoReset = false;
  t.enabled = true;

  function DoPartTwo()
  {
    // Perform second part of animation
    document.bar.animateProperty("x", "0px;100px", 1);
  }
}

This simple example illustrates a common pattern in HDi – chaining together two or more script-based animations by having a timer fire after the first animation is complete (this is unnecessary with markup animations, but they're not always practical).

In this case, DoTwoPartAnimation creates a local Timer variable, t, and gives it a reference to the nested function DoPartTwo as the callback function. As we learnt above, DoPartTwo maintains a reference to all of DoTwoPartAnimation's variables, including the timer t. So even after the timer has fired, and DoPartTwo has executed, we will leak the timer object (and the Function object for DoPartTwo) because the circular reference can't be broken.

The good news is that there are at least three ways to solve this problem:

Avoid closures altogether
Explicitly break the cycle
Avoid making the timer a local variable

The first is the simplest, and in the case above we can solve the problem by simply making DoPartTwo another global function instead of a nested function. Then although the timer t has a reference to DoPartTwo, DoPartTwo does not have a reference back to t and so the timer will be garbage collected sooner or later (which in turn will release DoPartTwo). The problem here is that often you need to use a nested function in order to correctly perform the callback operation – for example, the callback needs to know some of the parameters that were originally passed to the containing function in order to complete its task.

In that case, another simple solution is to explicitly break the cycle by manually releasing the reference to the timer. For example, we could change the DoPartTwo function slightly to look like this:

  function DoPartTwo()
  {
    document.bar.animateProperty("x", "0px;100px", 1);

    // Explicitly break the circular reference
    t = null;
  }

Now, as with the 1^st work-around, the script engine can eventually garbage collect t, which in turn will release DoPartTwo. Nevertheless, the problem with this solution – as with all solutions that require that the programmer religiously follow some strict rules all the time – is that you might forget to add this line to your callback, or you might have a return statement near the top of the function, or you might throw an exception, or... there are various ways your program might miss this opportunity to break the cycle.

So that leaves us with... my preferred approach, which is simply to make your own helper function – much like the web's setTimeout – that will create the timer for you. Here's a simple example:

// Call callback after timecode has elapsed
function SetTimeout(timecode, callback)
{
  var t = createTimer(timecode, 1, callback);
  t.autoReset = false;
  t.enabled = true;
}

Now you can simply replace all the pesky createTimer / autoReset / enabled code with a single call to SetTimeout, and avoid all the closure issues, too!

// Perform two animations, one after the other
function DoTwoPartAnimation()
{
// Perform some animation over 1 second
document.foo.animateProperty("y", "0px;100px", 1);

// Setup a callback to do the other animation after 1s
SetTimeout("00:00:01:00", DoPartTwo);

  function DoPartTwo()
  {
    // Perform second part of animation
    document.bar.animateProperty("x", "0px;100px", 1);
  }
}

The closure issues are avoided because DoPartTwo has zero knowledge of the Timer object t, which is a local variable of the SetTimeout function. Although the timer has a reference to DoPartTwo, the opposite is not true and so there is no circular reference.

Note about HTTPClient

Note that this "closure problem" also exists with HTTPClient objects and the onStateChange function. The same solutions exist for breaking the circular reference though – avoid closures, null out the object once it is finished (eg, STATE_COMPLETED, STATE_ERROR, or STATE_ABORT), or use a helper to create the object.

In general it doesn't affect other callback-based APIs like FileIO.openTextStream or XMLParser.parse because these are global objects, not local instances created by the enclosing function. The API will release the callback function as soon as it has been used (so it won't leak), and it doesn't really matter how many references there are to global objects; they will always exist.

The Re-Use Problem

Timer re-use is something that can be done correctly, but is often done incorrectly. Sometimes this is done by mistake (eg, forgetting to put var in front of a variable declaration, making it global), but other times it is deliberately done and leads to brittle code.

Here's a canonical example:

// Create timer; assume it is enabled
var timer = createTimer(TIME, CLOCK, CALLBACK);

function EventHandler()
{
// Clobber global timer; assume it is enabled
timer = createTimer(TIME2, CLOCK2, CALLBACK2);
}

function CancelTheTimer()
{
// Which timer are you cancelling?
timer.enabled = false;
}

The code is pretty much self explanatory:

A global timer is created
An event handler clobbers the first timer and creates a new one
A cancel function disables the timer

The problem is that depending on the order in which EventHandler and CancelTheTimer are called, different things may happen. If CancelTheTimer is called before EventHandler, it will cancel the global timer, but if the order is reversed then the global timer will fire and the event-handler timer will be disabled instead!

Even if you think you know the order in which events will occur, they may not always fire in exactly the same order – especially if they rely on user input or some asynchronous processes such as the progression of the title timeline. Better to always assume that the worst will happen, and program accordingly.

The simple solution here is to always use a unique name for your timers, or ensure they are local variables that won't conflict with other timers. (I would say to use the SetTimeout function from above, but that won't allow you to cancel the timer).

The General-Abuse Problem

The "general abuse" category basically covers any use of a timer that is unnecessary given the feature set of HDi. Sometimes authors abuse timers because they are not aware of HDi's built-in features (in which case it is generally easy to change from an abusive timer to use the right feature), or sometimes it is because the application's design has been built around timers rather than some more elegant solution (in which case it can be much harder to re-work the code to use fewer timers).

Two prime examples of the former include:

Performing many set operations on a repeating 1-frame timer, rather than using the <animate> element in markup or the animateProperty API
Constantly polling currentTitle.elapsedTime to check for a specific timecode, rather than using an <event> in markup or a PauseAt in the playlist

Examples of the latter are harder to come by, but (with no intention of bashing the other format) we have seen authors trying to replicate the behaviour of BD-J applications on HD DVD by an over-zealous use of timers, since they don't necessarily realise that the HDi system will do a lot of the heavy lifting for them.

Final Thought

An astute observer might ask the question "what happens if a timer is garbage-collected before it fires?" Good question. It is entirely possible that the local variable t used in these examples will be collected before the timer fires, especially if the timeout is quite long (say, several minutes). The answer is that the HD DVD specification requires timers to fire at least once, even if the script engine has released all references to them.

Simple ACA dumper

ptorr — Fri, 06 Jul 2007 00:36:00 GMT

Quick one today – if you have an ACA with some files in it, you can use the createaca tool (provided as part of the Jumpstart Kit) to list the files inside it. Trouble is that the output is not very helpful; it contains too much information.

So here is a simple JScript file you can run on the output of createaca -r to just dump the filenames (and it checks for duplicates, too!). Just copy to dumpaca.js (or the file of your choice) and run it from a command prompt. It expects a file named filelist.txt to exist in the same folder. Modifications to do whatever you want to do should be trivial...

// Open file named "filelist.txt"
‎var fso = new ActiveXObject("Scripting.FileSystemObject");
var stream = fso.OpenTextFile("filelist.txt");
var contents = stream.ReadAll();

// Regular expression to search for DATA_FNAME followed by a name
var re =/DATA_FNAME : ((?:\w|\.)+)/g;

// Lists of result data
var names = {};
var nameList = [];
var result;
var dupes = [];

// Search for every instance of the regular expression
while(null != (result = re.exec(contents)))
{
var name = result[1];

  // Record dupes
  if (names[name] != null)
  {
    dupes.push(name);
  }
  // else record new values
  else
  {
    names[name] = nameList.length;
    nameList.push(name);
  }
}

// Print results
print("---------- Complete file list ----------");

for (var i in nameList)
print(nameList[i]);

// Print dupes
if (dupes.length > 0)
{
print("---------- Duplicate file names ----------");

for (var j in dupes)
print(dupes[j]);
}

// Helper
function print(s) { WScript.Echo(String(s)); }