Julia Lerman has been experimenting with Ink annotations over video with Silverlight. In her post, she describes how to store StrokeCollection objects in XAML format (link to XML file), and some considerations with video annotation in Silverlight. Here's
