Annotating Video with Ink using Silverlight

swick — Thu, 15 Nov 2007 16:44:00 GMT

Silverlight makes delivering video content in the browser very easy. Silverlight also has ink support built-in. So why not combine these two features to enable some great new scenarios?

I have put together a little sample using Silverlight v1.0 to record and play back ink annotations, synchronized with the video playback. Below is the key piece of XAML markup for this application. We are using an InkPresenter that hosts the MediaElement with the video and takes care of the ink renderering. Also note the empty storyboard here as this is the magic sauce for the synchronized ink playback.

<InkPresenter Name='inkPresenter' Canvas.Left='10' Canvas.Top='10'

MouseLeftButtonDown='onInkPresenterDown'

MouseMove='onInkPresenterMove'

MouseLeftButtonUp='onInkPresenterUp'>

<InkPresenter.Resources>

<Storyboard Duration="0:0:0" Completed="onStrokePlaybackTimerTick"

x:Name="strokePlaybackTimer" />

</InkPresenter.Resources>

<MediaElement Name='mediaElement' Source='bear.wmv'

Width='720' Height='480'

AutoPlay='False' MediaEnded='onMediaEnded'/>

</InkPresenter>

The relevant pieces in the JavaScript code behind the XAML (in file events.js), are the following functions:

ConvertInkToString(strokes) - a function to serialize the point data for the ink strokes into a string for persistence purposes
ConvertStringToInk(serializedStylusPoints) - the reverse function that returns a StrokeCollection for a serialized ink string
onStrokePlaybackTimerTick() - during playback this function is called on every frame; when it is time to play back a stroke, we do so point by point

The full source code is attached to this post. I have also uploaded the sample to the Silverlight Streaming service, so I can pull it directly into my blog here. To check it out, just click the button below to load the sample. Once the video is loaded, you can click the 'Playback' button to replay my pre-recorded ink annotation - or you can click 'Start Recording' to record your own annotations, and then play them back afterwards.

Slide decks from my Oredev sessions

swick — Thu, 15 Nov 2007 16:43:00 GMT

Thanks to everyone who attended my sessions at the Oredev conference in Malmo, Sweden. Thank you also for all the great questions and feedback after the sessions - much appreciated! I have attached my slide decks to this post for those who are interested.

Some of the demos that I coded or showed during the sessions are already on this blog. I am working on getting the missing ones up here as well. Stay tuned ...

Also the video recordings from all Oredev sessions should become available soon on http://www.oredev.org.

Off to Oredev

swick — Mon, 12 Nov 2007 18:45:00 GMT

Just a quick note: I am on my way to Oredev conference. I'll be presenting the following two talks there:

Introduction to application development with WPF and Silverlight for TabletPCs and UMPCs

Advanced application development with WPF and Silverlight for TabletPCs and UMPCs

I'll post slide decks and demos here after I those presentations.

Stefan Wick's Weblog - Development with Silverlight, WPF and Tablet PC : oredev

Annotating Video with Ink using Silverlight

Slide decks from my Oredev sessions

Off to Oredev