Stefan Wick's Weblog - Development with Silverlight, WPF and Tablet PC : WPF Samples

TabletPC Development Gotchas Part 6: InkCanvas Element Selection/Move/Resize

swick — Tue, 18 Dec 2007 20:46:00 GMT

WPF's InkCanvas element provides a lot of built-in functionality for several common, ink-related tasks like stylus gesture recognition, point and stroke erase, as well as the selection, resizing and moving of ink strokes. The key to those features is the 'EditingMode' property - which is nicely demonstrated in the InkCanvas EditingModes SDK sample (btw, it also demonstrates an implementation of an Undo/Redo stack for ink operations).

One little known, yet very cool feature is not demonstrated in this sample, though: The selection, resizing and moving functioality (i.e. EditingMode="Select") can not only be applied to ink strokes, but also to child elements of the InkCanvas. This can be very handy if you want to build, for example, a note-taking application that can also host text, pictures and other content besides the handwritten ink. You can then use the 'Select" mode to let the user re-arrange and resize all their content.

The "gotcha" I wanted to point in this blog post is a limitation in this feature: Child elements in an InkCanvas may be positioned in a variety of different ways: for example you can position them by setting a "Margin" or by setting any combination of "Canvas.Left/Right/Top/Bottom" attached properties. InkCanvas's selection/moving/resizing feature, however, only works on elements that have absolute positioning set via "Canvas.Left/Top".

Let's look at the following example markup:

<Window x:Class="InkCanvasElementSelection.Window1"

xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"

xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"

Title="InkCanvasElementSelection"

ResizeMode="NoResize"

Height="450" Width="450">

<Window.Resources>

</Style>

</Window.Resources>

<Grid>

<InkCanvas Name="inkCanvas" EditingMode="Select"

Background="LightGoldenrodYellow"

SelectionChanging="OnSelectionChanging">

<Rectangle.RenderTransform>

</Rectangle.RenderTransform>

</Rectangle>

</InkCanvas>

</Grid>

</Window>

Note that the markup subscribes to the 'SelectionChanging' event on the InkCanvas. Let's leave the implementation of that event handler empty for now to demonstrate the limitation. Also note that the four rectangles use different ways to position themselves on the InkCanvas. Only the blue rectangle uses the "Canvas.Left/Top" attached properties for positioning.

Now you can use the lasso tool to select one or more elements, and then try to move and resize them. You will notice that it only works correctly for the blue rectangle (because that one has been positioned via "Canvas.Left/Top"). For the other rectangles you won't get the desired move or resize results.

Now what can I do if there are child elements that have not been positioned via "Canvas.Left/Top"? Here is a workaround to make this feature work even for those elements: In the 'SelectionChanging' event - which fires whenever you change what content is selected - you can walk the list of child elements and ensure they are positioned via "Canvas.Left/Top". Here is a piece you can add to the above sample to demonstrate this approach:

void OnSelectionChanging(object sender, InkCanvasSelectionChangingEventArgs e)

{

ReadOnlyCollection<UIElement> elements = e.GetSelectedElements();

foreach (UIElement element in elements)

{

// obtain actual location of element relative to InkCanvas

Point locationInInkCanvas = element.TranslatePoint(new Point(0, 0),

inkCanvas);

// set the location via Left/Top properties

element.SetValue(InkCanvas.LeftProperty, locationInInkCanvas.X);

element.SetValue(InkCanvas.TopProperty, locationInInkCanvas.Y);

// un-set right/bottom properties

element.SetValue(InkCanvas.RightProperty, double.NaN);

element.SetValue(InkCanvas.BottomProperty, double.NaN);

// re-translate any render transform

Matrix matRender = element.RenderTransform.Value;

matRender.Translate(-matRender.OffsetX, -matRender.OffsetY);

element.RenderTransform = new MatrixTransform(matRender);

// set margins to zero

if (element is FrameworkElement)

{

((FrameworkElement)element).Margin = new Thickness(0d);

}

Now if re-run the sample you will find all rectangles from the original markup can be selected, moved and resized as expected:

Full source code of this example is included in the attached Visual Studio project.

Rendering ink and image to a bitmap using WPF

swick — Sun, 02 Dec 2007 08:21:00 GMT

To follow-up on my previous blog post and to complete the story about rendering ink onto pictures and saving the results as a bitmap file, I want to show how this is done in WPF.

In WPF all rendering uses the pipeline - pictures, videos, ink, text, all gets rendered via WPF's 'Media Integration Layer'. As a result, it's very easy and straightforward to combine rendering these different types, both on screen and on a bitmap.

I have attached a WPF sample app that demonstrates how to collect ink on top of an image and then save the combined visual to a bitmap file. In fact, I have used this sample to create the sample picture that is included in my previous post about doing this in Winforms. Here is a screenshot of the sample app (btw, the ink color picker is borrowed from the InkColorPicker SDK sample).

The key class to accomplish our job in WPF is the RenderTargetBitmap class. Here is the relevant code snippet from the sample app:

// render InkCanvas' visual tree to the RenderTargetBitmap

RenderTargetBitmap targetBitmap =

new RenderTargetBitmap((int)inkCanvas1.ActualWidth,

(int)inkCanvas1.ActualHeight,

96d, 96d,

PixelFormats.Default);

targetBitmap.Render(inkCanvas1);

// add the RenderTargetBitmap to a Bitmapencoder

BmpBitmapEncoder encoder = new BmpBitmapEncoder();

encoder.Frames.Add(BitmapFrame.Create(targetBitmap));

// save file to disk

FileStream fs = File.Open(fileName, FileMode.OpenOrCreate);

encoder.Save(fs);

Unlike the Winforms API, there is no behavioral difference between Windows Vista and Windows XP.

Fun with Ink & XAML Part4: WPF BitmapEffects applied to Ink

swick — Sun, 04 Nov 2007 21:47:00 GMT

Want to create some fancy looking handwritten text or drawing? Tweaking the standard DrawingAttributes on an ink stroke won't get you very far - and creating a custom ink renderer is a lot of work. Why not just apply some of the WPF BitmapEffects to your ink. All you need to do is add a few lines of markup to an <InkPresenter/> and you are done - see markup snippet below (the complete XAML file is attached to his post):

<InkPresenter.BitmapEffect>

<DropShadowBitmapEffect

Color="Black" Direction="20" Opacity="0.8" Noise="0.2" Softness="0.2"

ShadowDepth="{Binding ElementName=shadowSlider,Path=Value}"/>

<BlurBitmapEffect

KernelType='Gaussian'

Radius="{Binding ElementName=blurSlider,Path=Value}"/>

<OuterGlowBitmapEffect

GlowColor="Yellow" Noise="0.5" Opacity="0.5"

GlowSize="{Binding ElementName=glowSlider,Path=Value}"/>

<BevelBitmapEffect

BevelWidth="{Binding ElementName=bevelSlider,Path=Value}"/>

</BitmapEffectGroup>

</InkPresenter.BitmapEffect>

As you can tell from the above markup, I have bound some of the BitmapEffect properties to Slider controls, so you can change their values at runtime to see how it affects the look of your ink. To try it out on your computer, download the attached XAML file and open it in XamlPad (or use Visual Studio to create a WPF application from it).

Now what can you do with this? I think it's a great way to create images of fancy looking, personalized handwritten notes and drawings. You can export the resulting ink to a bitmap file and then use it on your website, in your application or on your blog. Here is some code-behind that takes a snapshot of the InkPresenter and writes it to a .png file:

RenderTargetBitmap target = new RenderTargetBitmap(

(int)inputCanvas.ActualWidth,

(int)inputCanvas.ActualHeight,

96d, 96d, PixelFormats.Default);

target.Render(effectsPresenter);

PngBitmapEncoder encoder = new PngBitmapEncoder();

encoder.Frames.Add(BitmapFrame.Create(target));

FileStream fs = File.Open(@"c:\stuff\InkWithEffects.png",

FileMode.OpenOrCreate);

encoder.Save(fs);

fs.Close();

General note about BitmapEffects: WPF bitmap effects are rendered in software mode. Any object that applies an effect will also be rendered in software. Performance is degraded the most when using Bitmap effects on large visuals or animating properties of a Bitmap effect. WPF bitmap effects do not support partial trust execution, so you cannot load my example code into IE. For more details on BitmapEffects visit this MSDN topic.

Previous post in this series: Fun with Ink & XAML Part3: Ink Data Binding

Fun with Ink & XAML Part3: Ink Data Binding

swick — Sat, 03 Nov 2007 16:18:00 GMT

Data binding in WPF provides a great way for applications to present and interact with data. Elements can be bound to data from a variety of data sources - for example XML files. I have put together a sample that uses data binding to retrieve a collection of handwritten ink documents from a data store (XML file), present it to the user and let the user interact with it. As always in this series, the task is accomplished by markup only - no code behind required. The sample markup (and the data file) is attached to this post.

Let's take a look at the sample markup, specifically what's inside the <Grid.Resources> tag. I have highlighted the sections that are relevant for the data binding:

<Grid.Resources>

<TextBlock HorizontalAlignment="Center" Foreground="Red"

Text="{Binding XPath=@FileName}"/>

<InkPresenter HorizontalAlignment="Center"

LayoutTransform="0.2,0,0,0.2,0,0">

<InkPresenter.Strokes>

</InkPresenter.Strokes>

</InkPresenter>

</StackPanel>

</DataTemplate>

</Grid.Resources>

The first item here is the XmlDataProvider. This class facilitates the access to XML data for data binding. The other item, our DataTemplate, defines how the data is being presented: The 'FileName' value gets assigned to a <TextBlock/> and the 'Strokes' value gets assigned to the Strokes property of an <InkPresenter/>.

To see how these two resources are being used in the actual UI markup, let's inspect the ListBox element. The ListBox is being used to display a thumbnail view of all ink documents in the data store and to let the user choose which document they want to view or edit.

<ListBox Name="lbStrokes" Background="AntiqueWhite"

ItemsSource="{Binding Source={StaticResource InkData},

XPath=InkFile}"

ItemTemplate="{StaticResource InkDataTemplate}"/>

Now to view and interact with the selected item from our ListBox, we are adding a larger InkCanvas element that is data-bound to the ListBox. So when the user selects a new ink note from the list, that document gets loaded into the InkCanvas automatically. Here is the relevant markup to set up that binding:

<InkCanvas DataContext="{Binding ElementName=lbStrokes,Path=SelectedItem}"

Strokes="{Binding XPath=@Strokes}"

Background="Snow">

Below is a screenshot of the sample that is attached to this post. To run it, unpack the .zip file and load the .xaml file into XamlPad - or just double-click it to open it in IE. Note that the data file (InkData.xml) must reside in the same location as the XAML file.

Next post in this series: Fun with Ink & XAML Part4: WPF BitmapEffects applied to Ink

Previous post in this series: Fun with Ink & Xaml - Part2: Zoom and Scroll

East Asian Handwriting Recognition - the WPF way

swick — Fri, 02 Nov 2007 02:03:00 GMT

I have put together a sample that demonstrates the integration of East Asian boxed handwriting recognition with the inking and 3D features in WPF. The key, reusable component I have written here is the 'SingleLetterRecognizer" user control. It can, btw, also be used for western languages (by setting its 'Lcid' property accordingly), but i chose East Asian languages for this post because boxed, single character recognition is much more common in those languages.

As the user draws ink strokes into one of those 'SingleLetterRecognizer" boxes, the system's handwriting recognizer for the respective language starts analyzing the ink. When the analysis is complete and the recognition result is available, the control kicks off a 3D animation that flips the control around the x-axis to show the result as typed text. The user can double-click the control now to redo the input (if the result was not accurate).

The VS project for this sample is attached to this post. It requires Windows Vista Home Premium (or above). You can also run it on Windows XP if you have the TabletPC SDK and the Recognizer Pack installed.

Screenshot after ink input, prior to recognition complete (caveat: I don't actually write any of these languages, I just looked up some characters on the MSN homepage of the corresponding country and then wrote the simplest symbols I could find, stroke by stroke):

Screenshot after recognition complete, while animation is still going on:

Fun with Ink & Xaml - Part2: Zoom and Scroll

swick — Fri, 02 Nov 2007 02:01:00 GMT

Here is some more fun with ink and Xaml: Scrolling and zooming ink content - in WPF it is as easy as 1,2,3:

1) place an InkCanvas inside a ScrollViewer

2) assign a ScaleTransform to InkCanvas' LayoutTransform

3) data-bind to ScaleX/ScaleY properties in order to zoom in and out

All of this can be done in XAML - no code-behind required. Compare the below simple piece of markup with the code amount and complexity of the Winforms Ink Zoom & Scroll sample from the TabletPC SDK.

<Page xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation">

<DockPanel>

    <Slider Name="zoomSlider" DockPanel.Dock="Top"

            Minimum="0.1" Maximum="5" Value="1"/>

    <ScrollViewer HorizontalScrollBarVisibility="Auto">

      <InkCanvas Background="Yellow">

        <InkCanvas.LayoutTransform>

          <ScaleTransform ScaleX="{Binding ElementName=zoomSlider,

                                           Path=Value}"

                          ScaleY="{Binding ElementName=zoomSlider,

                                           Path=Value}"/>

        </InkCanvas.LayoutTransform>

      </InkCanvas>

    </ScrollViewer>

</DockPanel>

</Page>

The XAML file for this sample is attached to this post. You can load it directly into IE or play with it in XamlPad (see screenshot below). Use the slider on top of the Ink control to adjust the zoom level.

Next post in this series: Fun with Ink & XAML Part3: Ink Data Binding

Previous post in this series: Fun with Ink & Xaml - Part1: Ink Reflections

Shape Recognition with WPF and InkAnalysis

swick — Thu, 01 Nov 2007 19:45:00 GMT

Did you know that the Windows Vista SDK contains a shape recognition engine? It's part of the InkAnalysis framework that provides APIs for handwriting recognition, ink parsing and shape recognition. This framework enables some really cool scenarios, especially when combined with WPF. And it's very easy to program against it from a WPF app due to its strongly-typed, WPF-friendly wrapper assembly (iawinfx.dll). Today I want to focus on shapes, I'll work on a seperate post for handwriting recognition (for an overview of InkAnalysis take a look at this MSDN topic).

The sample that comes with this post in the attachment lets the user draw a shape (triangle, ellipse, rectangle, polygon) using the mouse (or stylus if available). The shape reco engine then kicks in and returns the corresponding WPF shape element. Once the shape elements are in the WPF tree, you can use the CheckBoxes to animate or move them around.

One of the key lines in the code is where we tell the InkAnalyzer that our ink stroke should be considered as 'Drawing' (as opposed to handwriting).

inkAnalyzer.SetStrokesType(e.Added, StrokeType.Drawing);

Then, when we get the result nodes from the InkAnalyzer, it will return the actual WPF shape objects, like this:

InkDrawingNode drawingNode = resultNode as InkDrawingNode;

if (drawingNode != null && drawingNode.GetShapeName() != "Other")

{

Shape shapeToAdd = drawingNode.GetShape();

...

}

Here is a screenshot of the sample just before completing the ink stroke:

After MouseUp (or StylusUp), shape recognition kicks in and the WPF shape element gets added to the Canvas.

Now that your handwritten stroke has been converted to a WPF shape, you can utilize the full power of WPF and do fancy things with them. For example: render them with semi-transparent gradients, animate smoothly or move them around - as demonstrated in my sample. The full sample project is attached to this post.

Fun with Ink & Xaml - Part1: Ink Reflections

swick — Thu, 01 Nov 2007 00:44:00 GMT

With the InkCanvas element in WPF you can create stunning inking experiences for TabletPC users (mouse users can play along, too!). Several very cool scenarios can be enabled just by writing markup - without any additional code behind.

Here is a first example: an ink input field (takes input from either stylus or mouse) that has a dynamic reflection effect. While the user is inking, the reflection gets updated dynamically, in real-time. Implemented entirely in markup.

The key feature (besides InkCanvas) here is the VisualBrush. The bottom part of the scene is a Rectangle that gets painted with a VisualBrush that is bound to the ink input control. As a result it gets updated dynamically as the user draws onto the control. Below is the relevant piece of markup. The full XAML file is attached to this post. You can load it into XamlPad to play with it, or just open in IE and ink away.

<Rectangle.Fill>
    <VisualBrush
      Visual="{Binding ElementName=inkBorder}">
      <VisualBrush.RelativeTransform>
        <TransformGroup>
          <ScaleTransform ScaleX="1" ScaleY="-1" />
          <TranslateTransform Y="1" />
        </TransformGroup>
      </VisualBrush.RelativeTransform>
    </VisualBrush>
</Rectangle.Fill>

More information about creating reflections using VisualBrush can be found in this MSDN How-To Topic. Additional How-To topics on Ink in WPF are also available on MSDN.

Next post in this series: Fun with Ink & Xaml - Part2: Zoom and Scroll

.NET 3.5: Viewport2DVisual3D and Ink

swick — Wed, 31 Oct 2007 22:59:00 GMT

WPF 3.5 includes an exciting new feature: Viewport2DVisual3D. It makes building interactive 3D experiences very easy and is nicely integrated with the rest of the WPF Platform. I have used this feature to write a new version of the "Ink Postcard" sample that I wrote a few years back for the Windows SDK. A VS project that contains the new and improved version is attached to this post (you need .NET 3.5 installed to run it).

The sample allows you to draw ink directly onto a 3D model (a Postcard in this case) - even when it is animated! It demonstrates how well this new feature is integrated with things like Ink, Animation, InkCanvas and input from mouse and stylus (when running on a TabletPC).

More information on Viewport2DVisual3D and other great samples around this can be found on Lester's WPF Blog and on Greg Schlechter's Blog.