Stefan Wick's Weblog - Development with Silverlight, WPF and Tablet PC : Winforms

TabletPC Development Gotchas Part 5: Rendering ink and image to a bitmap using Winforms

swick — Sun, 02 Dec 2007 08:19:00 GMT

Annotating pictures with handwriting or drawings is a fun and useful scenario for digital ink. In many cases the application user wants to create a new image file with the ink annotation "burnt" into the picture, so they can use the result of their work in other contexts and with other applications (e.g. attach the annotated image to an e-mail or upload it to their personal web site or blog, like the image below).

So rendering ink on top of an image is a common scenario for TabletPC/Ink developers - and it should be real easy, right? Well, there is an important 'gotcha' you need to know about if you want to do this from Winforms. But let's start with native code first.

Using the COM Ink API in native code

If you are using the ActiveX version of InkPicture (or the COM InkOverlay component) then rendering its ink to a GDI bitmap is easy. Just get the hDC (handle to device context) of the bitmap and use the InkRenderer.Draw() method:

pInkRenderer->Draw(hDCtargetBitmap, pStrokesToRender);

Using the Winforms Ink API to target XP (Microsoft.Ink v1.7)

In managed Winforms code, the object that corresponds to the hDC is the System.Drawing.Graphics object. So the intuitive approach would be to get the Graphics object of the Winforms target bitmap and then use the Microsoft.Ink.Render.Draw() method to render the desired ink strokes on top of it. Something along the lines of this:

Graphics graphicsTarget = Graphics.FromImage(imageToSave);

inkPicture1.Renderer.Draw(graphicsTarget,

                            inkPicture1.Ink.Strokes);

imageToSave.Save(“InkAndImage.bmp”);

Unfortunately, the result of this operation does not look great: the quality of the ink on the bitmap is not the same as what you see on the screen. Especially if you set some advanced DrawingAttributes (like Transparency or RasterOperations) the resulting bitmap does resemble the image on the screen very well. What is the reason for this behavior? The short version is that the underlying Ink Renderer and the managed Winforms images use GDI in slightly different ways, so the rendering operation has to fall back to a lower quality mode.

Fortunately, there is a workaround: we can take both the Winforms image and our ink strokes and render them to a temporary device-independent-bitmap. Then we'll render the result of that to the actual target bitmap. In order to do this, however, we have to do some manged<->native code interop magic. All that magic is encapsulated in the DibGraphicsBuffer class (see highlighted line below). This helper class is included in the attached sample project. Here is the relevant code to apply the workaround using that helper class:

Graphics graphicsTarget = Graphics.FromImage(imageToSave);

Graphics graphicsSrc = inkPicture1.CreateGraphics();

DibGraphicsBuffer buffer = new DibGraphicsBuffer();

Graphics graphicsTemp = buffer.RequestBuffer(graphicsSrc,

                                               inkPicture1.Image.Width,

                                               inkPicture1.Image.Height);

graphicsTemp.DrawImage(inkPicture1.Image, new Point(0, 0));

inkPicture1.Renderer.Draw(graphicsTemp, inkPicture1.Ink.Strokes);

buffer.PaintBuffer(graphicsTarget, 0, 0);

Using the Winforms Ink API to target Vista (Microsoft.Ink v6.0)

In order to address the limitation of the XP API, the Vista version of Render.Draw() now has an additional overload that makes the above workaround obsolete (of course the workaround still works on Vista). You can now pass in a Bitmap object directly to the API, and your ink gets rendered onto the image in the highest quality with a one-liner:

inkPicture1.Renderer.Draw((Bitmap)imageToSave, inkPicture1.Ink.Strokes);

Attached Code projects:

I have attached two complete code projects to illustrate how to get the best results on both XP and Vista:

1) RenderInkOnBitmapXP - utilizes the DibGraphicsBuffer class to apply the workaround for Windows XP

2) RenderInkOnBitmapVista - uses the new overload to demonstrate the best practice on Windows Vista

TabletPC Development Gotchas Part1: Handling exceptions in Winforms Ink events

swick — Thu, 01 Nov 2007 17:05:00 GMT

Over the past few years of developing apps using the TabletPC platform, I came across a couple of gotchas that are not exactly obvious to solve or workaround (at least they were not obvious to me). I figured I should blog about those and the respective solutions or workarounds to save others some head-scratching.

Here is the first one - it applies to the Microsoft.Ink assembly for developing inking apps using Winforms. Consider the following trivial app:

using System;

using System.Drawing;

using System.Windows.Forms;

using Microsoft.Ink;

class Program : Form

{

private InkOverlay _inkOverlay;

public Program()

{

_inkOverlay = new InkOverlay(this);

_inkOverlay.Enabled = true;

_inkOverlay.Stroke += new InkCollectorStrokeEventHandler(_inkOverlay_Stroke);

}

void _inkOverlay_Stroke(object sender, InkCollectorStrokeEventArgs e)

{

int foo = 2;

int bar = 5 / (2 - foo);

e.Stroke.DrawingAttributes.Color = Color.Red;

this.Invalidate();

}

static void Main(string[] args)

{

Application.Run(new Program());

}

The goal of the app is that each new stroke turns red once the stroke has been completed. However, when you run the app, this doesn't actually happen. The app works, but the ink remains in the default color (black). If you set a breakpoint on the line that sets the stroke's color to red, the breakpoint never gets hit. If you set a breakpoint at the top of the Stroke handler and step through the code, you will notice that the code execution returns to the app just before the color would be set.

Why is that? Well, in this simple example it's easy to see that there is a bug (division by zero). But why isn't the exception reported to us as an unhandled exception? The answer is not that obvious. It is due to the fact that the Microsoft.Ink assembly is really just a managed wrapper around a COM library (InkObj.dll). Take a look at the callstack when the event handler gets hit and you will see that your handler is being called from native code. So the unhandled, managed DivideByZero exception bubbles up the stack and eventually gets eaten by the native code that sits upstream from your event handler.

Now this behavior can result in some serious headscratching for the application developer. In most real-world cases, the bug is not as easy to spot as the one in my simplistic example. To avoid getting into these kind of issues, I recommend that you put your ink event handling code in try/catch blocks, so you can respond to exceptions thrown by your code, before they get eaten by the native/managed interop. Something along the lines of:

void _inkOverlay_Stroke(object sender, InkCollectorStrokeEventArgs e)

{

try

{

int foo = 2;

int bar = 5 / (2 - foo);

e.Stroke.DrawingAttributes.Color = Color.Red;

this.Invalidate();

}

catch (Exception exc)

{

MessageBox.Show(exc.ToString());

}

With this, the MessageBox will now tell you that there was a DivideByZero exception, and it will tell you where it happened.

Next post in this series: TabletPC Development Gotchas Part2: Deploying InkPicture applications to Windows XP desktops