March, 2010 - Microsoft PixelSense Blog - Site Home - MSDN Blogs
Blog Entries
  • Microsoft PixelSense Blog

    Raise of Hands: Do you use the Surface Simulator? Do you use the Surface Simulator APIs?


    Dear Readers,

    Hello, I am Luis Cabrera, the Program Manager responsible for the Surface SDK. I would like to get a better idea for the usage of some of the functionality we have exposed in our Surface SDK in the past.

    For those of you who are developing using the Microsoft Surface SDK, I would like you to respond to the following questions:

    1. Do you know about the Surface Simulator? (y/n)
    2. Do you use the Surface Simulator? (y/n)
    3. Do you know about the Surface Simulator APIs? (y/n)
    4. Do you use the Surface Simulator APIs? (y/n)

    Any additional comments regarding how you test your applications: ________________

     You can respond to this post, or if your prefer to respond privately, please shoot me an email at luisca at




  • Microsoft PixelSense Blog

    Using the RawImage in a WPF application.


    One common question I hear is:

    How can you consume the raw image provided by the core APIs in a WPF application? How do you display it on the screen?


    I just extracted it from the code I used to “process the image” created by the cup on the monster demo.

    Here is a simple answer to that question, the code is “AS IS” – use at your own risk.


    Follow these steps:


    1.        Add the Microsoft.Surface.Core assembly  to your project so you can reference  it.

    2.        After your SurfaceWindow is initialized, do the necessary setup so that you can start receiving Raw Image events and data.


    // Copyright © Microsoft Corporation.  All Rights Reserved.

    // This code released under the terms of the

    // Microsoft Public License (MS-PL,

            /// <summary>

            /// Default constructor.

            /// </summary>

            public SurfaceWindow1()




                SourceInitialized += new EventHandler(InitializeCore);


                // Add handlers for Application activation events





            void InitializeCore(object sender, EventArgs e)


                // Create a target for surface input, and start

                // receiving normalized raw images.

                contactTarget = new ContactTarget(
                                new WindowInteropHelper(this).Handle,

                // See step 3 for definition of OnContactTargetFrameReceived.

                contactTarget.FrameReceived += OnContactTargetFrameReceived;  





    3.        Now that you are receiving the raw image, do something with it. I my case I write it to a WriteableBitmap so that I can use it as a source to an Image in my visual tree. I write to the bitmap every 100 milliseconds. 

    // Copyright © Microsoft Corporation.  All Rights Reserved.

    // This code released under the terms of the

    // Microsoft Public License (MS-PL,


            // Assuming the following members of the class

            private byte[] normalizedImage;

            private ImageMetrics normalizedImageMetrics;


            private static WriteableBitmap writeableBitmap;


            // remembers the last time we showed the raw image.

            private long oldTimeStamp;


            /// <summary>

            /// Handler for the FrameReceived event. Here we get the

            /// rawimage data from FrameReceivedEventArgs object.

            /// </summary>

            /// <param name="sender">ContactTarget that received

            ///  the frame</param>

            /// <param name="e">Object containing information about

            ///  the current frame</param>

            private void OnContactTargetFrameReceived(

                object sender,

                FrameReceivedEventArgs e)


                long now = DateTime.Now.Ticks;


                lock (this)


                    long ticksDelta = Math.Abs(now - oldTimeStamp);


                    // Update image every 100 milliseconds.

                    if (ticksDelta > 100 * 10000)


                        int paddingLeft, paddingRight;


                        Rect imageBound = new Rect();

                        imageBound.X = 0;

                        imageBound.Y = 0;

                        imageBound.Width = 1024;

                        imageBound.Height = 768;


                        if (e.TryGetRawImage(






                            out normalizedImage,

                            out normalizedImageMetrics,

                            out paddingLeft,

                            out paddingRight))



                            imageToShow.Source = writeableBitmap;



                        oldTimeStamp = now;





            /// <summary>

            /// The WriteToImage method updates the

            /// WriteableBitmap by using unsafe code to write

            /// a pixel into the back buffer.

            /// </summary>

            /// <param name="normalizedImage">The raw image</param>

            /// <param name="metrics">The size of the raw image</param>

            static void WriteToImage(byte[] image, ImageMetrics metrics)


                int maxRow = metrics.Height;

                int maxCol = metrics.Width;


                if (writeableBitmap == null)


                    writeableBitmap =

                        new WriteableBitmap(





                            PixelFormats.Bgr32, null);







                    for (int y = 0; y < maxRow; y++)


                        for (int x = 0; x < maxCol; x++)


                            int color_data = 0;

                            int pBackBuffer = (int)writeableBitmap.BackBuffer;


                            pBackBuffer += y * writeableBitmap.BackBufferStride;

                            pBackBuffer += x * 4;


                            // Set Red, Green, Blue respectively

                            color_data = image[x + y * metrics.Stride] * 2 << 16;

                            color_data |= image[x + y * metrics.Stride] * 2 << 8;

                            color_data |= image[x + y * metrics.Stride] * 2 << 0;


                            *((int*)pBackBuffer) = color_data;







                    writeableBitmap.AddDirtyRect(new Int32Rect(0, 0, maxCol, maxRow));


                catch (Exception ex)


                    string s = ex.ToString();










    That was it! I hope this is helpful. Let me know if you have questions.


    Luis Cabrera
    Platform Program Manager

    Microsoft Surface

  • Microsoft PixelSense Blog

    Surface and Carnegie Mellon at PAX with D&D


    For the uninitiated to the Penny Arcade Expo, I can best describe it as a gaming conference by the fans, for the fans. This conference is normally held in Seattle to sell out and capacity crowds. This year Gabe and Tycho gave the fans (which includes themselves) even more by expanding to Boston with PAX East. This has made it accessible to even more fans who couldn't make the trek to Seattle. Intrigued? There's a brilliant blog post that puts PAX in perspective.

    I’ve been talking a lot about the Carnegie Mellon University grad student project that brought the Dungeons & Dragons proof-of-concept to Microsoft Surface. You can learn more about WHY (and read the disclosures that it’s an academic project and not a shipping application) in my previous blog post on the topic. It will help put this into perspective as to why it is relevant to Surface.

    This is the second time Surface has been at PAX. The reception is just as warm here in Boston. We’re located in the tabletop gaming area, room 111. (Tabletop HQ & Library) Fans of Surface or D&D can sit down and play with the Carnegie Mellon team. If you’re at PAX, you really should stop by – even if D&D isn’t your cup of tea – you’ll find out just how accessible Surface can make this game for the uninitiated.

    I’ll be reporting in throughout the weekend, so make sure to check back in.

    - Eric (follow Surface on Twitter and Facebook)

Page 1 of 4 (10 items) 1234

March, 2010