September, 2009 - Microsoft PixelSense Blog - Site Home - MSDN Blogs
Blog Entries
  • Microsoft PixelSense Blog

    Events in London featuring Microsoft Surface

    • 0 Comments

    Community I’m sharing this list of events featuring Microsoft Surface that I received from the Microsoft Technology Center team in London per an inquiry on Twitter.

    If you’re in London or any other region and have an event coming up featuring Surface, reply to this post and let everyone know the where and when. Tweet it out with the hash tag #surface. Microsoft Surface is often brought to events by customers, partners and not just Microsoft. We’re not always aware of where Surface will appear next. We rely on community members like you to spread the word.

    Cheers.
    Eric

  • Microsoft PixelSense Blog

    Hard Rock’s new Vegas cafe features Microsoft Surface

    • 0 Comments

    Here we go with more Microsoft Surface in Vegas! Hard Rock just announced the grand opening of a new Hard Rock Cafe Las Vegas – a 950 seat restaurant and live concert venue. Everything about this new cafe is cool – and the tech is no exception.  Visitors can touch and view Hard Rock’s memorabilia with a huge 18’ wide multi-touch interactive Rock Wall(tm) and in-booth touch screens running Microsoft Windows employing Deep Zoom technology. Here on the Surface blog, I’m most excited to share that Microsoft Surface will be right there with an amazing multi-user way to browse Hard Rock properties and memorabilia around the world. The Hard Rock reports that we can expect many other custom applications on Microsoft Surface too.

    The software running on Surface represents a collaboration between the Hard Rock International, Duncan/Channon and Microsoft Surface partner Vertigo. You can see a video created by Duncan/Channon below and more video here and photos here.

  • Microsoft PixelSense Blog

    Surface Developer Tip: The Core Interaction Framework

    • 0 Comments

    The Surface platform supports two UI frameworks: WPF & XNA.  When our the Microsoft Surface SP1 platform was launched earlier this year we mainly talked about the improvements made for WPF developers.  Today I’d like to talk about a big addition we made to the Surface SDK for XNA developers: The “Core Interaction Framework” (or CIF). 

    The Core Interaction Framework aims to speed up common UI implementation tasks and provide users with a more consistent experience.  Before describing the CIF though, I need to explain what lead us to creating it.

    The WPF programming model consists primarily of developers laying out a set of controls to define their UI structure, setting properties to customize the look and feel of those controls, and then writing code to handle specific events that the controls raise.

    XNA on the other hand is a much lower level framework where the programming model is made up of two primitive stages: 1) apps are asked to “update” their internal state – this is usually when they’ll ask each input device (mouse, keyboard, gamepad, or Surface) for their state and then do a bunch of processing to interpret and decide how to respond to that input.  2) apps are asked to “draw” onto the screen based on their state.  These two things happen really really fast over and over again to enable high quality UI where the developer has a ton of flexibility.

    (Yes, this is a gross oversimplification of both frameworks but it’s enough background for the topic of this post :)

    For Surface v1, we decided to enable the development of Surface apps with XNA but really put most of our investments into the WPF side of things.  We went this way because the control-based model of WPF enabled us to work on figuring out good ways to handle common interaction design problems (through research/experimentation/iteration) and then wrap our “best practices” up into reusable controls that apps could then build upon and customize.  For a simple example, if a user puts 2 fingers down on a button and only lifts 1 finger up, should the button “click” or wait until the other finger also lifts up?  We came to a good solution to this problem (it’s more nuanced than just yes or no so I’ll save that for a later post) and then built those best practices into our Button, RepeatButton, ToggleButton, RadioButton, and CheckBox controls.  So now whenever you use Surface WPF applications, their buttons all have the same behaviors and nuances.

    Meanwhile, XNA developers were simply empowered to leverage Surface input in their Update & Draw loops.  Even without a controls framework, several of our early partners decided to go with XNA instead of WPF.  Some because their developers had more familiarity with the XNA programming model (coming from a DirectX background, for example) and some because they needed the powerful graphics capabilities only provided by XNA.  The apps these partners built were quite compelling & well received.  However, back in Redmond we noticed the inconsistencies in how our WPF controls behaved and how the equivalents our partners had to create from scratch in XNA behaved.  Seeing these subtle differences made us realize that there was an opportunity for us to improve the Surface experience for both developers and end users: 1) our partners were having to invest time in solving these basic problems rather than working on the unique scenarios of their apps and 2) users had to discover/remember behavioral differences for common interactions when switching between apps.

    For the SP1 release of the Microsoft Surface platform we decided to take on these problems and mitigate them with what we call the “Core Interaction Framework” or CIF.  Many XNA developers want low level control of things like how their UI is rendered and how hit testing is performed – but very few want to get into the nitty gritty of how buttons and scrollbars actually behave.  So what we did with the CIF is create a bunch of reusable infrastructure classes that handle just the nitty gritty of how common controls behave – without making any assumptions or requirements regarding how the app does rendering or hit testing.  These classes are essentially configurable “state machines” – you set them up, delegate certain touch inputs to them, and then have your UI render based on the state they report.  Some controls have relatively simple state: button consists of “am I currently being pressed” and “was I just clicked”.  Other controls are a little more complicated: scrollbar reports back what part of it (track vs. thumb) is currently being touched, where along the track the thumb should be rendered and how big it should be, and what the currently selected ‘value’ is. 

    The basic steps to leveraging CIF are pretty simple:

    1. Construct a UIController.  This is the object that will orchestrate passing input between your app and the various control state machines.
    2. Construct state machine objects to represent each of your controls (ButtonStateMachine, ListBoxStateMachine, …)
    3. Create a hit testing callback method and provide it to the UIController.  This delegate will be passed info about each touch input.  Your job is to call Set*HitTestDetails on each of those inputs to tell the controller which state machine the input corresponds to.
    4. In your Update method, call UIController.Update().  This will trigger the controller to do some internal processing and invoke your hit testing callback with the latest Surface input data. 
    5. Also in your Update method, check your state machines objects for things you need to react to for example:
          if (myButtonStateMachine.GotClicked) { // ... }
    6. In your Draw method, render each of your controls according to their state.  For example, you’d use different visuals to represent a button state machine when it’s IsPressed==true.

    And that’s it!  You can get really creative on how these controls look on the screen (a curved scrollbar, for example) while being confident that the interaction model for those things is consistent with other Surface apps and leveraging all of the research done by the Surface team.

    The Surface SDK also has sample code which shows how to use these state machines in XNA with visuals that align with our WPF controls, but that’s a totally optional add-on.

    Everything you need to get started with this is in the Samples directory of the Surface SDK:

    .\Core\Framework -  The code for the CIF itself.  You can compile and use this as-is or make any changes/additions to meet your app’s needs.

    .\Core\Framework\CoreFramework.chm – This help file contains API reference documentation for all public classes in the CIF.  You can refer to this as a quick reference to what the CIF contains and how to use it.

    .\Core\Cloth – A new sample we created for the SP1 release which demonstrates nearly every feature of the CIF.  It’s fun to play with too :)

    .\Core\XnaScatter – An updated version of the sample from Surface v1 which now uses a CIF-based implementation of ScatterView for much simpler code

    If you use these things, we’d love to get your feedback.  We’d also love feedback if you’re a Surface XNA developer and decide not to use the CIF for new projects.

    -Robert

Page 3 of 4 (11 items) 1234

September, 2009