Jaime Rodriguez
On Windows Store apps, Windows Phone, HTML and XAML

  • Jaime Rodriguez

    Introduction to WPF 4 Multitouch

    • 6 Comments

    This tutorial recaps the multitouch features in WPF 4, as of the Beta 2 release.
    I also included two basic samples to get you jumpstarted with working code:

    A Multitouch Backgrounder

    Multitouch is simply an abstraction from the OS (or a platform) that routes touch input to an application. 
    The OS exposes multitouch input with different levels of control and/or detail.  For example, Windows 7 exposes multitouch data in three modes:

    • Raw touch provides access to all touch messages. Aggregation or interpretation of the messages is left to the application. This level of access is useful for programs that require raw access to all the primitives possibly for some custom interpretation and handling of the messages. For example Corel Paint It Touch and Windows 7 Paint require this level of control to implement drawing. Other possible consumers for this level of detail are custom control or platform vendors.
    • Gestures is a very convenient abstraction from raw-touch.
      The platform interprets all the lower level events and translates them into pre-defined gestures, then notifies the application that a gesture has occurred.  The most common gestures are pan, zoom, rotate, and tap.
      Gestures offers a very easy programming model, but it has a limitation: gestures engines tend to handle only one gesture at a time ( for example, rotate or zoom, but not a rotate with zoom).  
    • Manipulation and inertia.   
      Manipulation is a superset of gestures; any thing you can do with gestures, you can do with manipulations, but you gain greater granularity and flexibility.  Of course, the trade-off is that manipulation is a pinch harder to program than gestures, but don’t worry, both are straight forward.
      Inertia is an extension to manipulation that adds physics support to make all of your manipulations smooth and realistic.

    If you are not familiar with multitouch, I recommend these articles on multitouch in Windows 7:

    Now that you are a touch expert,  we can simply focus on explaining WPF’s support.

    Multitouch in WPF 4

    WPF 4 includes support for raw touch and manipulation (with some inertia support).  This support extends throughout the platform; UIElement, UIElement3D, and ContentElement have all been tweaked to support raw-touch and manipulation.

    Post beta2, WPF 4 will also support touch in some of the controls (for example, ScrollViewer). The list of controls and level of support is not yet finalized, so don’t hold it against me; I will update this post as soon as I get details.

     

    Approach #1: Raw-touch in WPF 4

    Again, raw multitouch support begins at UIElement, UIElement3D and ContentElement.
    All of these types now support a TouchDown, TouchUp, TouchMove, TouchEnter and TouchLeave event.
    Each of these events have a corresponding routed event and a tunneling (Preview) event. 

    • public static readonly RoutedEvent TouchDownEvent;
    • public static readonly RoutedEvent TouchEnterEvent;
    • public static readonly RoutedEvent TouchLeaveEvent;
    • public static readonly RoutedEvent TouchMoveEvent;
    • public static readonly RoutedEvent TouchUpEvent;

     

    If you drill down through these events, you will find they all have a TouchEventArgs parameter that holds a TouchDevice member and can get you a TouchPoint.  The TouchPoint is the meaningful data since it tells you whether it was a Up,Down, or Move TouchAction, and it tells you the Position where the touch happened.

    I have included a class diagram below; the names are pretty descriptive.

    Touch

     

    Handling raw touch in WPF is really as simple as listening for these events and reacting to the points and the actions. 
    Unlike Manipulation where you do have to opt-in by setting the IsManipulationEnabled property to true,  event notifications for raw touch are available without an explicit opt-in


    A sample application for raw touch

    Of course, for raw touch I had to create the canonical helloMT drawing pad.   

    Disclaimer: I took the code written by Sela to demonstrate the .NET wrappers for Windows 7 multitouch and simply ported it to WPF 4.  Taking their apps and porting them to WPF 4 was about a 15 minute exercise.

    Download the source code.  

    When running the app, simply apply multiple fingers through the window to have the drawing pad draw strokes that follow your fingers’ movements.


     

    Approach #2: Manipulation in WPF 4

    Manipulation in WPF 4 is an opt-in behavior.  There is a simple process to handle manipulation events in any WPF element:

    1. Set IsManipulationEnabled=true on the element you are touch enabling. You can do this from XAML or from code. 

      <
      Image x:Name="image"Width="200"IsManipulationEnabled="True" Source="Windows7.png">
    2. [Optional] Subscribe to ManipulationStarting and set your ContainerElement. 
      The ContainerElement is the UI element to which all manipulation calculations and events are relative. If you do not set a ContainerElement, the UI element that is firing the event will be used. This works well for Zoom/Scale, but for all Translate or Rotate manipulations you should set a ContainerElement, or else the UI will flicker and be jumpy. This is not a UI glitch, it happens because a single manipulation will fire multiple deltas, so you are recording movements relative to the UI element that is being moved. Not cool!

      In ManipulationStarting, you can also set your ManipulationMode to control the manipulations you are allowing. You can select from All | None | Rotate | Translate | Scale | TranslateX | TranslateY. If you don’t override it, the default is All.

      Finally, if you want to do single hand rotations, you can set a Pivot that your UI element will rotate around.
       void image_ManipulationStarting(object sender, ManipulationStartingEventArgs e)
       {   
              //canvas is the parent of the image starting the manipulation;
              //Container does not have to be parent, but that is the most common scenario
               e.ManipulationContainer = canvas; 
              // you could set the mode here too 
              // e.Mode = ManipulationModes.All;             
       }
    3. Subscribe to the ManipulationDelta event in the UI Element (or an element higher in the Visual tree, since the event is routed).  ManipulationDelta is where all the action happens.  I have included a “default” implementation below, with good commenting.
      void image_ManipulationDelta(object sender, ManipulationDeltaEventArgs e)
      {
          //this just gets the source. 
          // I cast it to FE because I wanted to use ActualWidth for Center. You could try RenderSize as alternate
          var element = e.Source as FrameworkElement; 
          if ( element != null ) 
          { 
              //e.DeltaManipulation has the changes 
              // Scale is a delta multiplier; 1.0 is last size,  (so 1.1 == scale 10%, 0.8 = shrink 20%) 
              // Rotate = Rotation, in degrees
              // Pan = Translation, == Translate offset, in Device Independent Pixels 
               
              var deltaManipulation = e.DeltaManipulation; 
              var matrix  = ((MatrixTransform)element.RenderTransform).Matrix;            
              // find the old center; arguaby this could be cached 
              Point center =  new Point ( element.ActualWidth/2, element.ActualHeight/2) ;
              // transform it to take into account transforms from previous manipulations 
              center = matrix.Transform(center); 
              //this will be a Zoom. 
              matrix.ScaleAt(deltaManipulation.Scale.X, deltaManipulation.Scale.Y, center.X, center.Y); 
              // Rotation 
              matrix.RotateAt(e.DeltaManipulation.Rotation, center.X, center.Y);             
              //Translation (pan) 
              matrix.Translate(e.DeltaManipulation.Translation.X, e.DeltaManipulation.Translation.Y);
      
              ((MatrixTransform)element.RenderTransform).Matrix = matrix; 
      
              e.Handled = true;
          }
      }


    That is how simple manipulation is. All the raw-touch data, translated into these simple Delta Matrixes!  

    Enhancing Manipulation with Inertia

    Inertia adds physics to a manipulation to make it feel more natural.  As expected, it works on all UI elements that support manipulation. The way to think of inertia is that it carries through the physical momentum of a manipulation. For example, if you are implementing a translation manipulation that is moving an image across the X-axis, inertia will continue the manipulation a bit longer than the actual manipulation contact and it would decelerate at a speed you define, simulating the momentum and the friction to stop the translation.

    To add support for inertia, we simply update our old code and listen to a new event and then add code to handle inertia on our manipulation Delta.

    1. Subscribe to ManipulationInertiaStarting. 
      This event is similar to ManipulationStarting, it gets called at the beginning of each individual manipulation. In the event handler we append parameters to the Manipulation. For inertia, the interesting properties include:
      • ExpansionBehavior – decelerates at DIPs per squared millisecond . 
      • TranslationBehavior  - decelerates at DIPs per millisecond.
      • RotationBehavior - decelerates at degrees per millisecond
      • InitialVelocities is read-only; it gives you the velocities calculated from the previous stage of the manipulation. You can use these values to calculate your own behaviors. 

        image 

        Here is the code to add our desired behaviors for inertia:

        void canvas_ManipulationInertiaStarting(object sender, ManipulationInertiaStartingEventArgs e)
        {                
                // Decrease the velocity of the Rectangle's movement by 
                // 10 inches per second every second.
                // (10 inches * 96 DIPS per inch / 1000ms^2)
                e.TranslationBehavior = new InertiaTranslationBehavior()
                {
                    InitialVelocity = e.InitialVelocities.LinearVelocity,
                    DesiredDeceleration = 10.0 * 96.0 / (1000.0 * 1000.0)
                };
        
                // Decrease the velocity of the Rectangle's resizing by 
                // 0.1 inches per second every second.
                // (0.1 inches * 96 DIPS per inch / (1000ms^2)
                e.ExpansionBehavior = new InertiaExpansionBehavior()
                {
                    InitialVelocity = e.InitialVelocities.ExpansionVelocity,
                    DesiredDeceleration = 0.1 * 96 / 1000.0 * 1000.0
                };
        
                // Decrease the velocity of the Rectangle's rotation rate by 
                // 2 rotations per second every second.
                // (2 * 360 degrees / (1000ms^2)
                e.RotationBehavior = new InertiaRotationBehavior()
                {
                    InitialVelocity = e.InitialVelocities.AngularVelocity,
                    DesiredDeceleration = 720 / (1000.0 * 1000.0)
                };
                e.Handled = true;                  
        }
      • You may notice I did not override the ManipulationContainer. This is not required; it will reuse the ManipulationContainer we set during the ManipulationStarting event.

    2. [Optional] Now, we could add code at our ManipulationDelta event handler for inertia. 
      This is optional, if you run the code at this point, inertia is already working, but you will notice there is no boundaries (the images fly off the screen).  So, just as an example, I will add code to handle the boundaries and stop the inertia when we reach the boundaries.
      void image_ManipulationDelta(object sender, ManipulationDeltaEventArgs e)
      {
          // …. this is the same code as above, in our manipulation delta.. 
      ((MatrixTransform)element.RenderTransform).Matrix = matrix; e.Handled = true; // Here is the new code.
      // We are only checking boundaries during inertia in real world, we would check all the time
      if (e.IsInertial) { Rect containingRect = new Rect(((FrameworkElement)e.ManipulationContainer).RenderSize); Rect shapeBounds = element.RenderTransform.TransformBounds(new Rect(element.RenderSize)); // Check if the element is completely in the window. // If it is not and intertia is occurring, stop the manipulation. if (e.IsInertial && !containingRect.Contains(shapeBounds)) { //Report that we have gone over our boundary e.ReportBoundaryFeedback(e.DeltaManipulation); // comment out this line to see the Window 'shake' or 'bounce' // similar to Win32 Windows when they reach a boundary; this comes for free in .NET 4 e.Complete(); } } } }


      That is it. Our image viewer now has inertia support, and we have full control on the deceleration, rotation ratios, etc.
    A sample application for Manipulation and Inertia:

    Image Viewer. Manipulation
    This sample uses the code above to manipulate the images on the canvas. 

    Download the source code

    The viewer supports scaling, translating, and rotating the images, using multitouch gestures. There is also inertia support as you execute any manipulation.

     

     

     

     

    Mixing and matching approaches:

    In WPF 4 raw-touch and manipulation are not mutually exclusive –this is different from Win32. 
    You can enable both raw touch and manipulation at the same time on any UI Element.  

    The table below explains how logic is handled for scenarios with different options enabled.

    Manipulations Enabled

    TouchDown is Handled

    GotTouchCapture is Handled

    User Logic

    WPF Logic

    None

    No

    No

    None

    Promoted to Mouse

    None

    Yes

    No

    Handled as Touch by user

    None

    None

    Yes

    Yes

    Handled as Touch by user

    None

    Enabled

    No

    No

    None

    1. Handled by Manipulation logic, TouchDevice is Captured,

    2. Manipulation logic will handle GotTouchCapture event and manipulation events will be reported

    Enabled

    Yes

    No

    Handled as Touch by user. User has to explicitly capture the touch device.

    Manipulation logic will handle GotTouchCapture event and manipulation events will be reported

    Enabled

    Yes

    Yes

    1. Handled as Touch by User.

    2. User has to explicity capture the touch device.

    3. GotCapture handled by user, user has to explicitly AddManipulator to invoke manipulation

    None

     

    Summary

    This tutorial provided a basic introduction to multitouch in WPF. As you have seen, WPF supports both raw-touch and manipulation (with inertia) for all WPF UI elements. 

    Using WPF’s new touch support, you can accomplish quite a bit with just a few lines of code. The support compliments and integrates quite well with the platform.

  • Jaime Rodriguez

    WPF discussions, 091101

    • 0 Comments

    As usual, raw, unedited, useful info from the Microsoft’s internal WPF discussions lists.


    Subject: All keys don't work in a WPF window when it is calling from a WinForm project.
    We had a big project developed in WinForm. Now I'm adding a new window to the project using WPF. The WPF window is now part of the project, i.e. it is not a separate project or dll. What happened now is any control that is supposed to accept keyboard inputs, such as textbox, does not respond to my keyboard input. The window only responds to mouse.

    Answer:
    You must use call the
    EnableModelessKeyboardInterop method for keyboard to work.


    Subject: Precision on WPF animations
    This question was from a partner. they have some synchronization issue between their own animation and WPF animation. The difference of ~0.2 can happen because WPF animation precision depends on rendering timing, doesn’t it ?

    Answer:
    That's right - we do a number of things to sync with rendering. 
    Time only "changes" at the start of a render pass, which is scheduled differently based on a number of factors (Desktop Window Manager present and enabled? Monitor refresh rate, desired framerate for animations, etc), and then too the time chosen is actually "in the future" a bit because we're trying to produce a set of changes that will be correct when they hit the screen.  To do this, we estimate the future presentation time for a given UI thread's render pass.  My guess is that this is what they're seeing in this case


    Subject:  Using Blend Behavior in Style?
    Why am I getting an error when setting a Behavior in a style. Isn’t Blend Interaction an AttachedProperty?

    Answer:
    Blend’s behaviors are an attached property, but publicly they are not exposed as a DP- you can do attached properties that are not DP’s by having just the static GetProperty/SetProperty.
    We use this syntax to keep the behavior syntax smaller- by not using a real DP here we were able to default the collection to having a value and remove 2 lines of XAML. If this were a real DP then you’d have to add the collection to the XAML as well:

    <Button>

      <i:Interaction.Behaviors>

    <i:BehaviorCollection>

          <s:SimpleBehavior/>

    </i:BehaviorCollection>

      </i:Interaction.Behaviors>

    </Button>

    In addition behaviors cannot be used inside of styles; the core issue is that we intentionally made behaviors not sharable- you can’t apply the same behavior to multiple elements. The reason is that if you use the WPF animation API as an example that it adds a ton of complexity to make the types sharable and it really detracts from the level of simplicity that we were looking for in Behaviors.

    In WPF, everything applied through a style is shared  across each element that it’s applied to. To get around this in early prototypes I used a trick with Freezables and the CoerceValueCallback to clone the behaviors every time they’re applied to an element and are already applied to something else, but none of this is present in SL and it can lead to some unexpected runtime behavior.


    Subject: Best Match for ICO in WPF?
    Is there a built-in way in the WPF 4.0 imaging classes to get a “best match” for a multi-frame ICO?

    Answer:
    We don’t have any automatic selection logic. You can iterate over all of the frames and find the one you want based upon each frame’s properties. You should be able to get them all from a frame by doing frame.Decoder.Frames. Alternatively, you can just do BitmapDecoder.Create and read the frames that way rather than indirectly through BitmapFrame.Create.
    Note: pre-Win7 WIC does not support Vista’s PNG icon frames. If you hit a PNG frame you will get an exception on Vista and XP.


    Subject: Binding and Custom Type Descriptors
    In WPF we’re binding an object that has a custom type descriptor and are wondering if there is a way to get around a couple issues:

    1. Is there a way to get WPF to listen to these changes given that WPF does not appear to subscribe to change events via PropertyDescriptor.AddValueChanged?
    It appears that WPF will bind to the properties offered through the type descriptor, but WPF does not monitor that type descriptor for value changes.    It does appear to listen to INotifyPropertyChanged events, but this is not that useful if we are extending and object with custom properties.

    2. Is there a way to have WPF bypass ICustomTypeDescriptor.GetProperties() when setting up a binding? In this case, custom types may hide the thing we actually want to bind to, so it would be useful to force WPF not to use the custom descriptor during binding.

    Answer:
    1. WPF will listen to ValueChanged if the object doesn’t implement INotifyPropertyChanged.  If both are available we only listen to INPC, to avoid duplicate notifications.  There are objects that expose both – chiefly ADO.Net’s DataRowView – so it’s a real issue.   If you have appropriate access, you can get the object to raise the PropertyChanged event with a property name for your custom property.  But if it’s not your object (and its OnPropertyChanged method is private), you’re out of luck.

    2. No.  We actually call TypeDescriptor.GetProperties(item), which in turn calls ICTD.GetProperties, so it’s out of our hands.  Usually people want the custom descriptor to override the native one;  we don’t have any way to ask for the other way around.  (I’m curious what your scenario is, though.  This is the first time someone’s asked for this.)


    Subject: Binding to dictionary with multiple indexers
    If I add a second indexer to a collection class, it will no longer bind to WPF FrameworkElement such that its index path can be referenced.
    The following works fine on regular collection, But I add a second indexer, or if the collection is keyed, the binding no longer works:

      Path=[0].

    Answer:
    Your two indexers have different signatures, probably something like

                    public object this[int index] { …}

                    public object this[string s] {…}

    The property path is declared in XAML, where everything is a string.  So when you say “Path=[0]”, WPF has to decide whether you mean the first indexer with argument (int)0, or the second indexer with argument (string)”0”.   There’s nothing in the XAML to indicate which one you mean, so I think we choose the line of least resistance and pick the second indexer – it requires no type conversion.

    At any rate, you can provide the missing guidance by saying

                    Path=[(sys:Int32)0]

    assuming you’ve previously declared

                    xmlns:sys=”clr-namespace:System;assembly=mscorlib”

    This says what you think it does:  “use the indexer that takes an int argument”.


    Subject: SketchFlow transitions between screens?
    Is it possible to create transitions between screens in SketchFlow?

    Answer:
    It should be doing this by default. The default transition is a fade, if you right-click a navigation connection in the map, there are a few more to choose from “Transition Styles”.


    Subject: RE: How to improve the text rendering of your .NET 4.0 WPF Applications

    Answer:
    Important caveat: Do not use TextFormattingMode=”Display” on text that is going to be scaled by a RenderTransform (or scaled in any way other than by changing the font size). It will end up blurry.

    Some more recommendations: http://blogs.msdn.com/text/archive/2009/08/24/wpf-4-0-text-stack-improvements.aspx


    Subject: RE: Partner's question about WPF-Image convert & Flash support

    Answer:

    There are external EMF->XAML converters:
    http://www.wpf-graphics.com/Paste2Xaml.aspx
    http://www.verydoc.com/emf2vector/emf-to-xps.html

    Indeed, WPF does not provide any native flash rendering.  I doubt we ever will.  Some options:
    1) Host a web browser.  Yes this is HWND interop code, so there are some compromises: the infamous airspace issues being the most prominent.

    2) For display only, you might look at a DirectShow filter, like: http://www.medialooks.com/products/directshow_filters/flash_source.html


    Subject: Property increment value i nBlend
    If you expose a numeric property on a type and it is available in the  property window you can change the value by dragging the cursor up, down, left and right.  Is there a way to tell Blend what the incremental value should be?

    Answer:

    Yes, you need to supply a design-time assembly that uses the NumberIncrementsAttribute.

    public sealed class NumberIncrementsAttribute : Attribute, IIndexableAttribute

    Name:

    Microsoft.Windows.Design.PropertyEditing.NumberIncrementsAttribute

    Assembly:

    Microsoft.Windows.Design.Interaction, Version=4.0.0.0

    http://blogs.msdn.com/unnir/archive/2009/03/22/writing-a-design-time-experience-for-a-silverlight-control.aspx


    Subject: pack: registration?

    I forget, how do you open a pack: URI?
    I tried using WebRequest.Create, but that gave me an error saying that pack: wasn’t registered… Then I tried using PackUriHelper to cause it’s static cctor to run, to try and get the prefix registered… PackWebRequest isn’t constructable (from what I can see)…

    Answer:

    [Multiple interesting data points]

    #1 System.Windows.Application has a static ctor where ResourceContainer package is added to PreloadedPackages so that downstream PackWebRequestFactory can find it. So you need to be running in the context of an Avalon application to get this.

    #2  If you use PackUriHelper class, the “pack:” prefix gets registered with the System.Uri class and this helps in performing the correct parsing and construction of System.Uri objects for pack Uris.

    The other registration is of the pack: scheme with WebRequest so that you can use WebRequest.Create method to return the PackWebRequest object. This can be done in your code. PackWebRequestFactory does not register it. You could use PackWebRequestFactory directly to get the PackWebRequest too –

    PackWebRequest request = (PackWebRequest)((IWebRequestCreate)new PackWebRequestFactory()).Create(packUri);


    Subject: RichTextBox viewable area

    Is there any way to get the viewable area in a RichTextBox?

    Answer:

    TextPointer upperLeftCorner = rtb.GetPositionFromPoint(new Point(0, 0), true /* snapToText */); TextPointer lowerRightCorner = rtb.GetPositionFromPoint(new Point(rtb.ActualWidth, rtb.ActualHeight), true /* snapToText */);

    You could refine this to get a tighter fit by looking at RichTextBox.ViewportWidth/ViewportHeight, to omit space for surrounding chrome like the possibly visible ScollViewer or Border. But for optimizing a property set on the viewable area first, including a small amount of extra content around the viewport is probably fine.


    Subject: ValidatesOnDataErrors not working on bindings inside an ItemsControl.

    I noticed that ValidatesOnDataErrors is not working on bindings inside an ItemsControl.ItemTemplate. I confirmed this problem has been fixed in Framework 4.0 but our target is 3.5.

    Answer:

    the workaround is to use the “long” form of ValidatesOnDataErrors:

    <Binding.ValidationRules>

                    <DataErrorValidationRule/>

    </Binding.ValidationRules>

     


    Happy coding!!

Page 1 of 1 (2 items)