About this time each year, I reach out to my colleagues at Microsoft Research to see what cool projects they are working on that might have some direct or indirect application to health and healthcare. A few years back we looked at technology they called “play anywhere”. It was later renamed Surface Computing and today several of our partners are building health solutions that utilize the Surface user interface.
With that in mind, I’m pleased to show you my latest visit to Microsoft Research. Once again, I was blown away by the projects I saw and how these software innovations might one day be applied to improve health and clinical computing around the world.
My first stop was to visit with Dr. Eric Horvitz, area manager for Microsoft Research. I usually start with Eric because he is a fellow physician, and also because he is often at the center of some of the most thought provoking and cool research projects at MSR. I wasn’t disappointed. Eric showed us the next evolution of something we saw last year called situated interaction.
Eric and fellow researcher, Dan Bohus, have now applied the technology to an automated, robotic-like assistant that keeps track of Eric’s every move. An on-screen avatar interacts with visitors outside Eric’s office just like a real-life receptionist might do. Only this assistant has the brain of a computer and knows just about everything that is going on in Eric’s life based on years of real-time data. She (it) even knows how likely it is that he’ll show up for a scheduled meeting.
My next stop was to see something called “What’s New on this Page?”. Researcher Susan Dumais is working on technology that helps us pinpoint what data is new on a web page every time anything is updated. You’d be surprised how many of the web pages you visit are updated every few seconds with new information. Susan’s project shows potential for helping users who must navigate through large volumes of data separate what is new and important from what they’ve seen before. I instantly thought of how such technology might be applied to laboratory and other clinical systems that constantly feed data into a patient record. Such technology would show “What’s New on the Page”.
Our third stop was to visit with Tim Paek. Tim is especially interested in mobile devices and applications. He showed us how speech input on a mobile device can be used to search for the information we need. As I was watching Tim find things on his mobile phone, I couldn’t help but think of how this might apply to busy clinicians looking for new lab results or other patient information while on the fly.
The last stop on this year’s visit to Microsoft Research is sure to be a crowd pleaser. Researchers Hrvoje Benko and Jonathan Fay (of Worldwide Telescope fame) demonstrated what is known as the Omniprojector. It’s a little like Surface Computing in 3 dimensions. Whether applied to astronomy, graphics, or perhaps human anatomy I think you’ll immediately grasp how this technology might play out in clinical settings (radiology, pathology, etc.) and medical education.
I hope you enjoy this year’s visit to Microsoft Research. You can watch each of the four segments individually by clicking on any of the screen shots of that segment. You can watch the entire program (about 25 minutes) by visiting my House Calls for Healthcare Professionals site on Microsoft.com.
Bill Crounse, MD Senior Director, Worldwide Health Microsoft
Dr. Bill, the robot administrative assistant reminded me of Aria in the movie Eagle Eye. It sort of creeped me out...but exciting stuff anyway!!!
Debbie. Thanks for your comment. Although the avatar is somewhat "robotic" today, the technology is advancing so fast that over the next decade or so, I believe you won't be able to tell what's human from what is animated. MSR is doing some terrific work in artificial intelligence and robotics. Some of their most recent work incorporates such human traits as emotion and empathy. So stay tuned for increasingly less "creepy" interactions with your robotic assistant.
Bill Crounse, MD
Great Research Development for Helthcare field. Surface computing 3 dimensions can be useful in Telemedicine branch
mainly Teleradiology and Pathology for different viewing before final diagnosis. Good Luck