Stefan Wick's Weblog - Development with Silverlight, WPF and Tablet PC : Ink

Custom Ink Rendering in Silverlight 2

swick — Tue, 02 Dec 2008 00:26:00 GMT

Drawing ink with the built-in stroke objects works great for handwriting and annotations. In some scenarios, however, you may want something more fancy that takes advantage of Silverlight's rich graphics capabilities.

Creating ink strokes with a customized appearance is fairly straight forward with Silverlight 2. You basically collect StylusPoints in mouse events as usual, but instead of adding them to stroke objects, you just draw your own geometries or shapes at the stylus/mouse location.

When collecting sample points during the MouseMove event, please be sure to call the StylusDevice.GetStylusPoints() method and process all points returned from that call. This will ensure that stylus users will get the full fidelity of input points, since their pointing device reports points more often than the MouseMove events fires. Also it is good practice to interpolate points between sample points, since they won't usually be equidistant.

I have a attached a sample project that demonstrates the concept end-to-end using Ellipse shapes with a LinearGradientBrush to give my ink strokes a custom look. Below is live version of the sample for you to play with.

Erasing Ink in Silverlight 2

swick — Sun, 30 Nov 2008 18:09:00 GMT

Another frequently asked question around inking in Silverlight is about erasing Ink strokes after they have been collected or loaded. Unlike WPF or the Tablet SDK for COM & Winforms, Silverlight does not come with controls that have built-in eraser modes in the box. However, with a little bit of code-behind you can implement those modes and adjust them to your specific needs. The key APIs you will need for this are StrokeCollection.HitTest() and Stroke.HitTest(). Both APIs take a collection of StylusPoints as input and they return a value indicating whether or not the ink stroke(s) have been hit. Based on that you can then modify your collection accordingly to represent the result of the erase operation. Popular inking apps (e.g. Windows Journal) offer two different modes of erasing:

Erase By Stroke:
Implementing this mode is fairly straight forward. Just pass the stylus points collected along the eraser path into the HitTest() method of your collection of ink strokes and the API will tell you which strokes to remove from your collection.

Erase By Point:
This one takes a little more effort to get right. We start like above, but then we don't remove the hit strokes. Instead, we figure out which segment of the stroke was hit (by using the Stroke.HitTest() API) and then we split the stroke by removing that segment.

I have attached a sample that implements both modes. You can also check out a live version of the sample below. The simplistic implementation of "Erase By Point" in the sample leaves some areas of improvements for the reader (or for a future blog post). Most notably, the size the chunk erased in this mode is not very consistent as it depends on several factors (distance points, location of the hit point, thickness of the stroke, etc.). A good improvement would be to add some logic that ensures the eraser always cuts a consistent chunk out of the ink strokes.

Saving Ink to IsolatedStorage with Silverlight 2

swick — Sat, 01 Nov 2008 21:38:00 GMT

One of the questions I have seen quite a few times in the Silverlight.net forums is how to persists (and de-persists) ink in Silverlight 2. Unlike WPF, Silverlight does not support a binary persistence format (ISF), but you can write some code to save and load ink to and from XAML. To demonstrate how to do this in a real world example, I am using another very cool feature in Silverlight 2: IsolatedStorage

IsolatedStorage is a per application, per computer, per user storage that offers a powerful, easy to use way for your web applications to store data locally on the user's and retrieve it later when the user visits your sites again on the same computer. You can use it like a file system, or as in the case of this example, simply use it to store application settings in key/value pairs. Learn more about how to do the latter here.

The goal of my application is to store the ink as XAML in IsolatedStorage when the user navigates away from the site. Upon return, the app will retrieve the persisted data from the storage, depersist it and present the ink to the user. Here are the relevant pieces of code (full code project is attached to this post).

Serializing and de-serializing ink strokes using XAML:

public string InkXAML

{

get

{

// persists ink strokes to XAML

StringBuilder builder = new StringBuilder();

XmlWriter writer = XmlWriter.Create(builder);

writer.WriteStartElement("StrokeCollection",

"http://schemas.microsoft.com/winfx/2006/xaml/presentation");

foreach (Stroke stroke in _presenter.Strokes)

{

writer.WriteStartElement("Stroke");

writer.WriteStartElement("Stroke.DrawingAttributes");

writer.WriteStartElement("DrawingAttributes");

writer.WriteAttributeString("Width",

stroke.DrawingAttributes.Width.ToString(invCult));

writer.WriteAttributeString("Height",

stroke.DrawingAttributes.Height.ToString(invCult));

writer.WriteAttributeString("Color",

string.Format(invCult, "#{0:X2}{1:X2}{2:X2}{3:X2}",

stroke.DrawingAttributes.Color.A,

stroke.DrawingAttributes.Color.R,

stroke.DrawingAttributes.Color.G,

stroke.DrawingAttributes.Color.B));

writer.WriteAttributeString("OutlineColor",

string.Format(invCult, "#{0:X2}{1:X2}{2:X2}{3:X2}",

stroke.DrawingAttributes.OutlineColor.A,

stroke.DrawingAttributes.OutlineColor.R,

stroke.DrawingAttributes.OutlineColor.G,

stroke.DrawingAttributes.OutlineColor.B));

writer.WriteEndElement();

writer.WriteStartElement("Stroke.StylusPoints");

writer.WriteStartElement("StylusPointCollection");

foreach (StylusPoint sp in stroke.StylusPoints)

{

writer.WriteStartElement("StylusPoint");

writer.WriteAttributeString("X", sp.X.ToString(invCult));

writer.WriteAttributeString("Y", sp.Y.ToString(invCult));

writer.WriteEndElement();

}

writer.WriteEndElement();

}

writer.WriteEndElement();

writer.Flush();

return builder.ToString();

}

set

{

// depersist ink strokes from XAML

try

{

_presenter.Strokes = XamlReader.Load(value) as StrokeCollection;

}

catch (Exception exc) { Debug.WriteLine(exc.ToString()); }

}

Persisting data on Application exit:

void Application_Exit(object sender, EventArgs e)

{

IsolatedStorageSettings settings = IsolatedStorageSettings.ApplicationSettings;

string inkXAML = _inkCollector.InkXAML;

if (settings.Contains("Ink"))

{

settings["Ink"] = inkXAML;

}

else

{

settings.Add("Ink", inkXAML);

}

Loading data on Application startup:

public Page()

{

InitializeComponent();

_inkCollector = new InkCollector(inkNote);

IsolatedStorageSettings settings =

IsolatedStorageSettings.ApplicationSettings;

string inkXAML = "";

if (settings.Contains("Ink"))

{

inkXAML = settings["Ink"] as string;

_inkCollector.InkXAML = inkXAML;

}

If you have Silverlight 2 installed (who hasn't? :-)), you can try it out live below. Draw some ink, navigate away, close the browser, clear the cache, etc. Then return to my blog and see that your ink is back, due to the power of IsolatedStorage.

TabletPC Development Gotchas Part 7: Detecting the input device in Silverlight 2

swick — Sat, 01 Nov 2008 20:07:00 GMT

When writing an application that supports multiple input devices (mouse, stylus, touch) it is often neccessary to detect which type of device is sending the input events.Silverlight 1.0 offered a DeviceType property on the StylusInfo object that gets reported in every mouse event.

In the Silverlight 2 managed API, you will not find an equivelnt object that provides this information. We omitted it on purpose in this release, because adding this to the OM could not be done in a way that would make the Silverlight API a compatible subset of the WPF API. In future versions of both platforms we will provide a consistent way to get to this information.

Omitting this for now is acceptable since we do provide a workaround to get to this information, via the HTML DOM bridge. Here is how you do it (full source attached):

Expose a scriptable method from managed code to handle the DeviceType information:

public Page()

{

InitializeComponent();

HtmlPage.RegisterScriptableObject("page", this);

}

[ScriptableMember]

public void SetDeviceType(string deviceType)

{

_deviceType = deviceType;

switch (_deviceType)

{

case "Mouse":

// add specific code for mouse

break;

case "Stylus":

// add specific code for stylus

break;

case "Touch":

// add specific code for touch

break;

}

Hook up to the MouseLeftButtonDown event in JavaScript and pass the StylusInfo.DeviceType information to managed code when the event fires:

function onSilverlightLoad(sender, args) {

// attach a JavaScript MouseDown event handler

var inkPresenter = sender._element.content.findName("inkPresenter");

inkPresenter.addEventListener("mouseLeftButtonDown", onMouseDown);

}

function onMouseDown(sender, args) {

// get the device type info and pass

// the information to the managed code

var stylusInfo = args.getStylusInfo();

var plugin = sender.getHost();

plugin.content.page.SetDeviceType(stylusInfo.deviceType.toString());

}

</script>

Now, as a proud owner of a Toshiba M700, I can run the solution and collect ink from different devices and the app automatically changes the pen attributes as I switch between input devices. Here is a screenshot showing ink strokes collected with the three different devices available on my Tablet PC:

Silverlight 2 Released - Write On!

swick — Tue, 14 Oct 2008 08:45:00 GMT

Now that Silverlight 2 is released, I finally got around to updating my blog after a long time of silence. For those of you interested in handwriting and Tablet related technologies, I want to start out with a canonical example of a Silverlight 2 Inking application. There have been a couple of OM changes since Beta1 around ink and stylus in Silverlight, mostly in order to make the Silverlight API more consistent with the WPF Ink/Stylus APIs.

Just like in Silverlight 1.0, the key element to inking is the InkPresenter which is a Canvas-like element with the ability to render ink strokes. To bring it to live and actually enable ink collection (and eraser) funtionality you will need to write some code-behind that responds to the user's input. In my example I have encapsulated the required code into a re-usable 'InkCollector' class (in InkCollector.cs), so you can just instantiate and assign it to your InkPresenter in order to get started with inking in your own projects.

Here is my example - source code attached - try it out by using your mouse or stylus to write and erase ink:

Snipping Pictures with Silverlight

swick — Tue, 18 Dec 2007 20:45:00 GMT

One of my favorite tools on the Tablet PC is the 'Snipping Tool'. I have been using it frequently since the early XP days (back then we shipped it seperately as a Power Toy). Now in Vista, it's part of the main OS and I find myself using it quite often event on my desktop PC.

When I was using the tool recently, I got this idea for a new Silverlight sample app: wouldn't it be cool to snip pieces from various images and put them together as a collage in Silverlight - and then annotate the result with text, drawings and/or handwriting?

Silverlight's imaging and ink APIs make this fairly easy. Here are my results of this effort using Silverlight v1.0 - the sample demonstrates these features:

Select an image from an animated collection of pictures
Snip a piece of the selected image (free-form, rectangle or entire picture) and arrange it on the canvas
Use 'Image Tools' to control zoom, rotation and opacity of individual snippets
Use 'Text Tools' to add/change text annotations
Use 'Ink Tools' to add/change drawings and handwritten annotations
Full screen and embedded mode

Couple of notes and caveats:

The pictures are currently hardcoded (appropriate for the holiday season :-) ), but you can easily change that in the attached source code to use your own pictures or to pull images from the web.
When you launch the app from this blog, it opens up in full screen mode (since this is the best mode to view and use it). You can press the 'Esc' key or use the 'Miscellaneous' menu to go back to embedded mode.
In full screen mode Silverlight disables keyboard input (for security reasons). So if you want to enter some text annotations, please switch to embedded mode.
The 'Create Bitmap' button doesn't do anything in this version. The idea is to invoke some server-side code here to generate a bitmap of your collage. If there is interest in the community, this might be a good topic for a seperate blog post. But then again: if you are on Windows Vista, you can just use the Snipping Tool in the OS, to grab your collage and save it as bitmap :-)

The sample's complete source code is attached to this post. A live version, hosted via Silverlight Streaming, is included below for you try out. Click the button to launch the app:

Ink Reflections - in Silverlight

swick — Sun, 02 Dec 2007 08:25:00 GMT

I have received several question from folks about my earlier post on Ink Reflections in WPF. People were curious how to accomplish the same or a similar effect in Silverlight, in the absence of the VisualBrush object.

Clearly, WPF's VisualBrush is the main object that enables scenarios like this and it definitely makes their implementation straightforward. However, in Silverlight you can also accomplish effects like this with little bit of extra coding.

First of all, Silverlight has an ImageBrush and a VideoBrush, so as long as the visul you want to mirror is an image or a video, you can apply the same technique as used in my WPF sample and you are done.

If you want to mirror a different type of visual, you will have to manage the mirror yourself, meaning you have to create a secondary, mirrored visual and update it whenever the original changes.

I have created a sample that demonstrates both techniques together. The yellow background with silver border is an image that gets mirrored using an ImageBrush. The ink you draw (and erase) gets mirrored programmatically using some JavaScript code-behind.

The sample's source code is attached. A live version, hosted via Silverlight Streaming, is included below for you try out.

Rendering ink and image to a bitmap using WPF

swick — Sun, 02 Dec 2007 08:21:00 GMT

To follow-up on my previous blog post and to complete the story about rendering ink onto pictures and saving the results as a bitmap file, I want to show how this is done in WPF.

In WPF all rendering uses the pipeline - pictures, videos, ink, text, all gets rendered via WPF's 'Media Integration Layer'. As a result, it's very easy and straightforward to combine rendering these different types, both on screen and on a bitmap.

I have attached a WPF sample app that demonstrates how to collect ink on top of an image and then save the combined visual to a bitmap file. In fact, I have used this sample to create the sample picture that is included in my previous post about doing this in Winforms. Here is a screenshot of the sample app (btw, the ink color picker is borrowed from the InkColorPicker SDK sample).

The key class to accomplish our job in WPF is the RenderTargetBitmap class. Here is the relevant code snippet from the sample app:

// render InkCanvas' visual tree to the RenderTargetBitmap

RenderTargetBitmap targetBitmap =

new RenderTargetBitmap((int)inkCanvas1.ActualWidth,

(int)inkCanvas1.ActualHeight,

96d, 96d,

PixelFormats.Default);

targetBitmap.Render(inkCanvas1);

// add the RenderTargetBitmap to a Bitmapencoder

BmpBitmapEncoder encoder = new BmpBitmapEncoder();

encoder.Frames.Add(BitmapFrame.Create(targetBitmap));

// save file to disk

FileStream fs = File.Open(fileName, FileMode.OpenOrCreate);

encoder.Save(fs);

Unlike the Winforms API, there is no behavioral difference between Windows Vista and Windows XP.

TabletPC Development Gotchas Part 5: Rendering ink and image to a bitmap using Winforms

swick — Sun, 02 Dec 2007 08:19:00 GMT

Annotating pictures with handwriting or drawings is a fun and useful scenario for digital ink. In many cases the application user wants to create a new image file with the ink annotation "burnt" into the picture, so they can use the result of their work in other contexts and with other applications (e.g. attach the annotated image to an e-mail or upload it to their personal web site or blog, like the image below).

So rendering ink on top of an image is a common scenario for TabletPC/Ink developers - and it should be real easy, right? Well, there is an important 'gotcha' you need to know about if you want to do this from Winforms. But let's start with native code first.

Using the COM Ink API in native code

If you are using the ActiveX version of InkPicture (or the COM InkOverlay component) then rendering its ink to a GDI bitmap is easy. Just get the hDC (handle to device context) of the bitmap and use the InkRenderer.Draw() method:

pInkRenderer->Draw(hDCtargetBitmap, pStrokesToRender);

Using the Winforms Ink API to target XP (Microsoft.Ink v1.7)

In managed Winforms code, the object that corresponds to the hDC is the System.Drawing.Graphics object. So the intuitive approach would be to get the Graphics object of the Winforms target bitmap and then use the Microsoft.Ink.Render.Draw() method to render the desired ink strokes on top of it. Something along the lines of this:

Graphics graphicsTarget = Graphics.FromImage(imageToSave);

inkPicture1.Renderer.Draw(graphicsTarget,

                            inkPicture1.Ink.Strokes);

imageToSave.Save(“InkAndImage.bmp”);

Unfortunately, the result of this operation does not look great: the quality of the ink on the bitmap is not the same as what you see on the screen. Especially if you set some advanced DrawingAttributes (like Transparency or RasterOperations) the resulting bitmap does resemble the image on the screen very well. What is the reason for this behavior? The short version is that the underlying Ink Renderer and the managed Winforms images use GDI in slightly different ways, so the rendering operation has to fall back to a lower quality mode.

Fortunately, there is a workaround: we can take both the Winforms image and our ink strokes and render them to a temporary device-independent-bitmap. Then we'll render the result of that to the actual target bitmap. In order to do this, however, we have to do some manged<->native code interop magic. All that magic is encapsulated in the DibGraphicsBuffer class (see highlighted line below). This helper class is included in the attached sample project. Here is the relevant code to apply the workaround using that helper class:

Graphics graphicsTarget = Graphics.FromImage(imageToSave);

Graphics graphicsSrc = inkPicture1.CreateGraphics();

DibGraphicsBuffer buffer = new DibGraphicsBuffer();

Graphics graphicsTemp = buffer.RequestBuffer(graphicsSrc,

                                               inkPicture1.Image.Width,

                                               inkPicture1.Image.Height);

graphicsTemp.DrawImage(inkPicture1.Image, new Point(0, 0));

inkPicture1.Renderer.Draw(graphicsTemp, inkPicture1.Ink.Strokes);

buffer.PaintBuffer(graphicsTarget, 0, 0);

Using the Winforms Ink API to target Vista (Microsoft.Ink v6.0)

In order to address the limitation of the XP API, the Vista version of Render.Draw() now has an additional overload that makes the above workaround obsolete (of course the workaround still works on Vista). You can now pass in a Bitmap object directly to the API, and your ink gets rendered onto the image in the highest quality with a one-liner:

inkPicture1.Renderer.Draw((Bitmap)imageToSave, inkPicture1.Ink.Strokes);

Attached Code projects:

I have attached two complete code projects to illustrate how to get the best results on both XP and Vista:

1) RenderInkOnBitmapXP - utilizes the DibGraphicsBuffer class to apply the workaround for Windows XP

2) RenderInkOnBitmapVista - uses the new overload to demonstrate the best practice on Windows Vista

Annotating Video with Ink using Silverlight

swick — Thu, 15 Nov 2007 16:44:00 GMT

Silverlight makes delivering video content in the browser very easy. Silverlight also has ink support built-in. So why not combine these two features to enable some great new scenarios?

I have put together a little sample using Silverlight v1.0 to record and play back ink annotations, synchronized with the video playback. Below is the key piece of XAML markup for this application. We are using an InkPresenter that hosts the MediaElement with the video and takes care of the ink renderering. Also note the empty storyboard here as this is the magic sauce for the synchronized ink playback.

<InkPresenter Name='inkPresenter' Canvas.Left='10' Canvas.Top='10'

MouseLeftButtonDown='onInkPresenterDown'

MouseMove='onInkPresenterMove'

MouseLeftButtonUp='onInkPresenterUp'>

<InkPresenter.Resources>

<Storyboard Duration="0:0:0" Completed="onStrokePlaybackTimerTick"

x:Name="strokePlaybackTimer" />

</InkPresenter.Resources>

<MediaElement Name='mediaElement' Source='bear.wmv'

Width='720' Height='480'

AutoPlay='False' MediaEnded='onMediaEnded'/>

</InkPresenter>

The relevant pieces in the JavaScript code behind the XAML (in file events.js), are the following functions:

ConvertInkToString(strokes) - a function to serialize the point data for the ink strokes into a string for persistence purposes
ConvertStringToInk(serializedStylusPoints) - the reverse function that returns a StrokeCollection for a serialized ink string
onStrokePlaybackTimerTick() - during playback this function is called on every frame; when it is time to play back a stroke, we do so point by point

The full source code is attached to this post. I have also uploaded the sample to the Silverlight Streaming service, so I can pull it directly into my blog here. To check it out, just click the button below to load the sample. Once the video is loaded, you can click the 'Playback' button to replay my pre-recorded ink annotation - or you can click 'Start Recording' to record your own annotations, and then play them back afterwards.

Ink blogging - using Windows Journal and Silverlight Streaming!

swick — Sun, 04 Nov 2007 21:56:00 GMT

Are you using Windows Journal to jot down your thoughts on your Tablet PC? Did you ever wish to publish your handwritten notes to your blog?

Here is one way to do this: The TabletPC SDK provides a Journal Reader API that provides access to the content of the .jnt file. Now you can read all content and convert it into a 'blog-friendly' format. Since Silverlight offers great ways to present content in an attractive, interactive way, I decided to write a Journal -> Silverlight converter. The result of this test project is now available as SDK sample on microsoft.com/downloads.

The converter is easy enough to use. Just drop down to a command box and run "JntToSilverlight.exe MyNotes.jnt". This will produce a 'MyNotes' folder that contains the resulting Silverlight application. You can test the results now by double-clicking the 'default.htm' file to load the app into your browser.

OK, now how do I get all this onto my blog? Using Silverlight Streaming, it's very easy: just zip it up, upload it to the service and then stick an IFrame into your blog's HTML. See my previous post on this subject.

To summarize the steps:

1) Create a note in Windows Journal

2) Convert note to Silverlight app

3) Upload to Silverlight Streaming service

4) Insert IFrame into blog

This is still a lot of steps and a bit tedious, but it's a proof of concept. Steps 2-4 all have an API they so are automatable and one could create a tool that does all that in one step.

Here is one example of a converted note on my blog (the original .jnt is attached to this post, if you want to compare). Besides flipping through the pages, you can add more ink, move selected content around and erase ink. Also don't miss the full-screen mode and the mouse-wheel zoom feature!

Fun with Ink & XAML Part4: WPF BitmapEffects applied to Ink

swick — Sun, 04 Nov 2007 21:47:00 GMT

Want to create some fancy looking handwritten text or drawing? Tweaking the standard DrawingAttributes on an ink stroke won't get you very far - and creating a custom ink renderer is a lot of work. Why not just apply some of the WPF BitmapEffects to your ink. All you need to do is add a few lines of markup to an <InkPresenter/> and you are done - see markup snippet below (the complete XAML file is attached to his post):

<InkPresenter.BitmapEffect>

<DropShadowBitmapEffect

Color="Black" Direction="20" Opacity="0.8" Noise="0.2" Softness="0.2"

ShadowDepth="{Binding ElementName=shadowSlider,Path=Value}"/>

<BlurBitmapEffect

KernelType='Gaussian'

Radius="{Binding ElementName=blurSlider,Path=Value}"/>

<OuterGlowBitmapEffect

GlowColor="Yellow" Noise="0.5" Opacity="0.5"

GlowSize="{Binding ElementName=glowSlider,Path=Value}"/>

<BevelBitmapEffect

BevelWidth="{Binding ElementName=bevelSlider,Path=Value}"/>

</BitmapEffectGroup>

</InkPresenter.BitmapEffect>

As you can tell from the above markup, I have bound some of the BitmapEffect properties to Slider controls, so you can change their values at runtime to see how it affects the look of your ink. To try it out on your computer, download the attached XAML file and open it in XamlPad (or use Visual Studio to create a WPF application from it).

Now what can you do with this? I think it's a great way to create images of fancy looking, personalized handwritten notes and drawings. You can export the resulting ink to a bitmap file and then use it on your website, in your application or on your blog. Here is some code-behind that takes a snapshot of the InkPresenter and writes it to a .png file:

RenderTargetBitmap target = new RenderTargetBitmap(

(int)inputCanvas.ActualWidth,

(int)inputCanvas.ActualHeight,

96d, 96d, PixelFormats.Default);

target.Render(effectsPresenter);

PngBitmapEncoder encoder = new PngBitmapEncoder();

encoder.Frames.Add(BitmapFrame.Create(target));

FileStream fs = File.Open(@"c:\stuff\InkWithEffects.png",

FileMode.OpenOrCreate);

encoder.Save(fs);

fs.Close();

General note about BitmapEffects: WPF bitmap effects are rendered in software mode. Any object that applies an effect will also be rendered in software. Performance is degraded the most when using Bitmap effects on large visuals or animating properties of a Bitmap effect. WPF bitmap effects do not support partial trust execution, so you cannot load my example code into IE. For more details on BitmapEffects visit this MSDN topic.

Previous post in this series: Fun with Ink & XAML Part3: Ink Data Binding

Silverlight Ink SDK Sample - hosted through Silverlight Streaming

swick — Sat, 03 Nov 2007 16:40:00 GMT

With the 'Silverlight Streaming' service you can now invoke a hosted Silverlight application using an IFrame - without any JavaScript! This allows you to embed Silverlight content on sites that do not allow adding JavaScript, as long as they support IFrames. For details refer to this MSDN topic.

To demonstrate this great service, I have taken the Silverlight Ink SDK Sample, uploaded it to Silverlight Streaming and then embedded it on this blog using an IFrame, like this:

The highligthed part is the key: it specifies your Silverlight Streaming application ID and name of your app. For more information about Microsoft Silverlight Streaming, visit http://silverlight.live.com/.

Here is the sample for you to try out:

Fun with Ink & XAML Part3: Ink Data Binding

swick — Sat, 03 Nov 2007 16:18:00 GMT

Data binding in WPF provides a great way for applications to present and interact with data. Elements can be bound to data from a variety of data sources - for example XML files. I have put together a sample that uses data binding to retrieve a collection of handwritten ink documents from a data store (XML file), present it to the user and let the user interact with it. As always in this series, the task is accomplished by markup only - no code behind required. The sample markup (and the data file) is attached to this post.

Let's take a look at the sample markup, specifically what's inside the <Grid.Resources> tag. I have highlighted the sections that are relevant for the data binding:

<Grid.Resources>

<TextBlock HorizontalAlignment="Center" Foreground="Red"

Text="{Binding XPath=@FileName}"/>

<InkPresenter HorizontalAlignment="Center"

LayoutTransform="0.2,0,0,0.2,0,0">

<InkPresenter.Strokes>

</InkPresenter.Strokes>

</InkPresenter>

</StackPanel>

</DataTemplate>

</Grid.Resources>

The first item here is the XmlDataProvider. This class facilitates the access to XML data for data binding. The other item, our DataTemplate, defines how the data is being presented: The 'FileName' value gets assigned to a <TextBlock/> and the 'Strokes' value gets assigned to the Strokes property of an <InkPresenter/>.

To see how these two resources are being used in the actual UI markup, let's inspect the ListBox element. The ListBox is being used to display a thumbnail view of all ink documents in the data store and to let the user choose which document they want to view or edit.

<ListBox Name="lbStrokes" Background="AntiqueWhite"

ItemsSource="{Binding Source={StaticResource InkData},

XPath=InkFile}"

ItemTemplate="{StaticResource InkDataTemplate}"/>

Now to view and interact with the selected item from our ListBox, we are adding a larger InkCanvas element that is data-bound to the ListBox. So when the user selects a new ink note from the list, that document gets loaded into the InkCanvas automatically. Here is the relevant markup to set up that binding:

<InkCanvas DataContext="{Binding ElementName=lbStrokes,Path=SelectedItem}"

Strokes="{Binding XPath=@Strokes}"

Background="Snow">

Below is a screenshot of the sample that is attached to this post. To run it, unpack the .zip file and load the .xaml file into XamlPad - or just double-click it to open it in IE. Note that the data file (InkData.xml) must reside in the same location as the XAML file.

Next post in this series: Fun with Ink & XAML Part4: WPF BitmapEffects applied to Ink

Previous post in this series: Fun with Ink & Xaml - Part2: Zoom and Scroll

TabletPC Development Gotchas Part3: Coerce to Factoid (XP vs. Vista)

swick — Sat, 03 Nov 2007 15:50:00 GMT

If you are writing (or maintaining) and application that uses handwriting recognition, please be aware of a subtle behavior difference between XP and Vista when coercing the recognition result to a Factoid (or an input scope or a word list). This may easily be missed in your application testing, but it will potentially result in unhandled exceptions from your application when consumers are starting to use it.

Here is the scoop: normally, when you assign strokes to a RecognizerContext for recognition, it will always be able to return some result - even for random scribbling. However, if you set the 'Coerce' flag to bias the recognizer towards a certain format (e.g. Date, URL, etc.) or word list (e.g. medical terms) then there is a (small) chance that it will not be able to compute a result. On Windows XP, the RecognizerContext returns a null RecognitionResult in this case. On Windows Vista, however, an InvalidOperationException is thrown.

So if you have written an application agains the XP behavior, please update your code to also catch InvalidOperationException if you expect your customers to use your app on Windows Vista. If you are writing a new app today on Vista, please be sure to also check the RecognitionResult for null if you plan to support your product on Windows XP Tablet PCs.

To clarify the issue in code, consider the following example: the goal here is to recognize a given strokes collection as URL. Now if the strokes are random enough, the recognizer won't be able to coerce it to a valid URL (with my handwriting, for example, writing "ooooo" does not yield a result).

private string Recognize(Strokes strokesToRecognize)

{

RecognitionStatus status;

RecognizerContext recoContext = new RecognizerContext();

recoContext.Factoid = "(!IS_URL)";

recoContext.RecognitionFlags = RecognitionModes.Coerce;

recoContext.Strokes = strokesToRecognize;

try

{

RecognitionResult result = recoContext.Recognize(out status);

if (result != null)

{

return result.TopString;

}

else

{

// Windows XP code path

return "Null means NO RESULT on XP";

}

catch (InvalidOperationException)

{

// Windows Vista code path

return "InvalidOperationException means NO RESULT on Vista";

}

So the bottom line is: if you are writing an application that uses handwriting recognition, you should harden your code to handle both a 'null' result as well as an 'InvalidOperationException' from calling RecognizerContext.Recognize().

Next post in this series: TabletPC Development Gotchas Part4: Semantics of GetSystemMetrics(SM_TABLETPC)

Previous post in this series: TabletPC Development Gotchas Part2: Deploying InkPicture applications to Windows XP desktops