January, 2008 - Microsoft PixelSense Blog - Site Home - MSDN Blogs
Blog Entries
  • Microsoft PixelSense Blog

    Surface Grand Piano


    Rob and Thomas think one cool use of Surface would a piano app.  I agree :)  Believe it or not, this concept is actually one of the first tech samples we shipped to partners in early drops of the Surface SDK.  We chose this because 1) it's a good example of leveraging multi-input, 2) from an development perspective, it's a good example of the benefits Surface gets from leveraging WPF, and 3) most of the work was already done for us.

    It turns out that the Expression Blend team shipped a Grand Piano sample with their tool to highlight the power of having developers & designers work closely together using tools built for their specific needs.  In that sample, each key on the Piano is simply a WPF Button control which is given a custom style which makes it look like a 3D piano key.  If you don't have Expression Blend installed, there's a Silverlight version of this that you can play with online.

    Grand Piano 

    So how did we Surface-enable this?  3 easy steps:

    • Replace <Window> in the XAML with <SurfaceWindow>
      • SurfaceWindow is a subclass of the WPF Window class which automatically does some initialization stuff that we require of all Surface apps (like making the window full-screen on the right display)
    • Replace <Button> in the XAML with <SurfaceButton>
      • SurfaceButton is a subclass of the WPF Button class which knows how to correctly respond to Surface input so you can press multiple buttons at once and use multiple fingers on each button.
    • Ask Celso on the Blend team to provide updated images for the piano keys
      • The ones shipped in the original version of this app have an optimization that assumes only one key will be pressed at a time.

    I don't have a video handy to show this in action (it's pretty much exactly what you'd expect), but will create one if people express interest in the comments on this post.

    Disclaimer: This was created purely as an SDK tech sample to help show partners how to leverage Blend and WPF with our SDK.  It's not something we consider to be a "complete" app (there are tons of ways it could/should be expanded - including ideas Thomas mentioned on his blog) and it's not something our Demo crew uses at events to tell the Surface story (they prefer things that better fit our v1 business model and leverage all the multi-user & object recognition aspects of Surface).

  • Microsoft PixelSense Blog

    How creating Surface apps is different, part 2


    How creating Surface apps is different, part 2

    Continuing the list from my previous post

    Multiple simultaneous users

    The Surface is all about multiple simultaneous users interacting with each other and the Surface.  The typical PC is designed for a single user.


    There are a lot of implications of having multiple people using your application at once.  Global states should be avoided.  You wouldn’t want one user to be able to delete all the photos from a photos application while another user is correcting redeye.


    You can use per-user UI to give a user their own section of the screen (like where an individual player would keep their cards and chips in a poker game.)  This can take a lot of screen space.  It also assumes you know how many users are actually using the app.


    Sometimes you can depend on social interaction to mitigate multi-user problems (like people agreeing on a color change in the paint app.)  Other times you can’t.  It’s probably a good thing to reduce the amount of negotiations between your users.  As a parent I’ve had this made very clear whenever I ask my kids to share a single toy.  There has been some interesting research on this.

    Multiple user contexts

    There may be trouble with multiple users using, say, an instant messaging application.  The client side of the IM service may only allow a single logon per machine.  The server may only allow one logon per IP address.  If your application uses local OS security features, your application may have difficulty getting at restricted resources for different users simultaneously.


    There aren’t any magic fixes for this other than to architect your system to support this end to end.

    Multiple simultaneous drag-drop operations

    On a desktop OS, drag and drop is a system modal operation.  The APIs for this type of operation really only expect one drag and drop operation active at a time.  On the Surface you can have many users dragging many different things from many “drag sources” to many “drop targets.”


    System-wide we do not have any great solutions for this yet.  Within a single application there are ways to accomplish this in a fairly clean manner (the Music demo has to deal with this.)  The SDK team is producing samples to demonstrate one way to do this.

    Traditional idea of keyboard focus and insertion carets will not work

    On a desktop OS, only one edit control on the system will receive keyboard input at any time.  There is also only one edit control that will have an insertion caret.


    On the Surface, this is not such a big deal since there is no hardware keyboard anyway.  However if you want to enter text in your app using an on-screen keyboard, you will have difficulties using the standard Windows/WPF controls.


    For the best Surface user experience you want to design applications to not require text input.  For the times where text input is absolutely required we are producing a soft keyboard and related components that will make it easy for application developers to do text input.

  • Microsoft PixelSense Blog

    Surface Debuts at CES


    Surface had its CES coming out party last night, as Bill Gates showed off one of our new demo applications (a retail product customization scenario) in his final keynote to the industry. History-making stuff and I'm glad Surface could be a part of it. Technically, we were here last year with an NDA event with some select media as we previewed what was ahead (in advance of the May 2007 announcement) but it doesn't really count as this is the first chance CES attendees could touch Surface. Surface and other natural user interfaces were one of the new trends Bill predicted would mark this next "digital decade."

    Today, the Surface section of the Microsoft booth was a madhouse...don't get me wrong...we love the throngs of Surface enthusiasts that crowded the booth, some even pushing and shoving to get a better view and hands-on time with the product. It was just packed and packed with people! I'm told that at one point the temperature in our area reached 106 degrees (not sure who discovered that, but better yet, why do we have a thermometer in the booth)...anyways, it actually felt hotter! They are apparently reworking the booth flow in our area this evening to accomodate the crowds better.

    Here's Vanessa, our partner marketing manager, surrounded (and being filmed)...


    I'm demoing the product customization scenario in the Microsoft Theater twice a day...stop on by for the show...

    Kyle on stage at CES

    In the booth, the team is demoing the previous announcement demos that many of you have likely seen. In addition, we are showcasing two new experiences...1) a retail product customization scenario (in a sporting goods store where you can customize your own snowboard), and 2) a winebar scenario. In the next few posts, I'll highlight the demo applications themselves, show you some screenshots and talk about why we selected those particular experiences.

    Check back tomorrow for more updates from the floor.


    K Robert Warnick


Page 1 of 2 (6 items) 12

January, 2008