Download Research Tools
Robotics technology plays an increasingly important role in search-and-rescue missions. Robots are used to explore areas that are deemed too dangerous or difficult for human teams to access. They can, for example, be used to investigate a hazardous material spill or search for disaster survivors. In the case of a disaster, a robot may save the life of not only the victims but also the rescue workers who might otherwise place themselves in harm’s way to search for survivors. Because of the life-saving potential of search-and-rescue robots in emergency situations, researchers are investigating better ways to control the robots in stressful and challenging environments.
Robots that are used for search and rescue are essentially an extension of the human rescue team. Cameras, microphones, and other sensors that are attached to the robot transmit critical information to the rescue team, who typically controls the robot’s movement remotely. Until recently, rescuers who managed a search-and-rescue robot normally had to manipulate complicated joysticks, dials, and switches on a very elaborate controller with multiple electro-mechanical parts. As described in our blog entry last year, the robotics lab at the University of Massachusetts, Lowell (UML), has developed a natural user interface (NUI) controller that promises greater finesse and control of robots during search-and-rescue operations.
Today we are pleased to present a new short video that highlights the accomplishment of this work and gives you an update on its status.
Building the DREAM Controller
The Lowell robotics lab takes a NUI approach for the Dynamically Resizing Ergonomic and Multi-Touch (DREAM) Controller, which has been in development since 2008. Two Microsoft technologies underlie the DREAM Controller: the Microsoft Robotics Developer Studio (used for simulation) and Microsoft Surface (the user interface).
The Microsoft Surface is a coffee-table-sized device with a computer inside and a touch-sensitive interface on top. The Surface allows multiple users to interact with the computer simultaneously by using whole-hand or multiple-finger gestures. These gestures enable rescue teams to control robots with greater dexterity than they could with traditional robotics controllers—and precise control of the robots is critical for search-and-rescue efforts. In addition, the Surface permits more than one robot to be controlled simultaneously—previously not possible with a single controller.
To use the DREAM Controller implemented on the Surface, users simply place their hands on the interface. The DREAM Controller identifies the user’s fingers and thumbs and displays a virtual “joystick” beneath their hand. The user then uses their thumb to manipulate the virtual joystick. There are up to four degrees of freedom (two on each thumb, that is, X-Y on each), enabling the control of four different dimensions.
The Lowell team (Holly Yanco and Mark Micire) is also developing a series of pre-programmed gestures with guidance from expert search-and-rescue volunteers. The goal is to develop code that enables the DREAM Controller to recognize specific gestures that rescue workers make naturally during a search-and-rescue operation, thereby facilitating and accelerating rescue efforts.
The novel NUI approach to robotics that was employed by the Lowell robotics lab in this socially significant application helped the DREAM Controller project win one of eight grants that Microsoft Research offered under our Social Human Robot Interaction Request for Proposals (RFP). The grant award included financial support, a donated Microsoft Surface, and access to the Microsoft Research team.
I think the DREAM Controller project truly shows what a better first response system—using NUI technology—could look like in the very near future. Check out the video!
—Stewart Tansley, Senior Research Program Manager, Microsoft Research Connections
HIV infection may not be the death sentence it once was, but it remains an undeniably serious condition that requires aggressive, life-long treatment and entails the ever-present threat of severe immunological impairment. Consequently, medical researchers continue to investigate the mechanisms by which HIV infection evades detection by the body’s normal immune responses. In the August 3, 2011, issue of Nature, investigators from the Ragon Institute of MGH, MIT, and Harvard; Imperial College London; the National Cancer Institute; and Microsoft Research have shed light on the interactions of HIV and the immune system’s natural killer (NK) cells. Our paper is the first to show that NK cells play a direct role in fighting HIV. This knowledge opens a new path of research into ways to beat the virus.
Scientists have long known that NK cells play an important role in the control of viral infections, mounting short-lived but highly toxic assaults on infected cells. NK cells bind to virus-infected cells, releasing proteins that destroy the target cells. To regulate this cytotoxic potential, the membranes of NK cells are studded with activating receptors, which unleash the cell-killing response, and inhibitory receptors, which keep it in check.
It’s logical to expect that NK cells would play a role in the control of HIV infections, and, in fact, various in-vitro and epidemiological studies suggest that NK cells do just that. For example, research has shown that the population of NK cells increases during the earliest phase of HIV infection and that NK cells can suppress HIV replication in cultured tissues. Moreover, epidemiological evidence indicates that infected individuals who have particular versions of the genes that code for a class of NK cell receptors called KIRs (killer immunoglobulin-like receptors) are better able to control HIV levels. However, it remained unknown whether NK cells directly mediate anti-HIV immune pressure inside the human body.
We wanted to test the hypothesis that mutations in the HIV proteins that are recognized by KIRs could allow the virus to escape NK cell activity. Proving this hypothesis would support a role for NK cells in HIV control. After analyzing the sequences of both HIV proteins and the genes encoding KIR molecules from 91 infected individuals, we found that particular variants in viral proteins were associated with specific KIR genes. This finding suggested that the virus mutates in response to NK cell activity. In particular, we found individuals whose NK cells included an inhibitory receptor called KIR2DL2 were more likely to have variant forms of HIV that enhance viral interaction with that receptor. Those results suggest that the HIV mutates into a form that interacts with the inhibitory receptor, thereby preventing NK cells from attacking HIV-infected cells.
Microsoft Research was intensely involved in this study. The first tell-tale signs that NK cells were affecting HIV were found by using a sophisticated software tool that was developed at Microsoft Research. The tool used almost a CPU-year of computation to sift through millions of possible clues as to how our immune system interacts with this deadly virus.
Our study provides hope that a greater appreciation of the NK-cell-mediated immune responses to HIV can lead to therapies that interrupt the virus’s evasive processes, thereby giving physicians another weapon in their long-running battle with HIV and AIDS.
—David Heckerman, Distinguished Scientist, Microsoft Research
Interest in the Kinect for Windows Software Development Kit (SDK) beta, released on June 16, 2011, has been strong, and we’re delighted to learn that so many developers and innovators who are experimenting with natural user interface (NUI) applications have taken advantage of the SDK to explore the potential of the Kinect sensor.
In support of our commitment to encourage researchers and enthusiasts in their exploration of the exciting possibilities of the Kinect sensor, we have now released a refreshed version of the SDK. The community has provided us with a lot of good feedback, and this release addresses some of the top items you’ve told us about.
Before summarizing the updates, let’s quickly recap the key features of the Kinect for Windows SDK beta. This non-commercial SDK beta enables human motion tracking, voice recognition, and depth sensing on PCs, enabling developers to create innovative natural user interface applications. The SDK includes drivers and rich APIs for raw sensor streams and natural user interfaces, as well as installation documents and resource materials.
So, what’s in the refresh?
The refresh also includes many improvements to the documentation, including clarifications and the deletion of information pertaining to non-functional components. Also, the SDK samples have been enhanced.
If you’re an academic researcher or an enthusiast who wants to take advantage of the latest developments in natural user interface experimentation, we encourage you to learn more about and download the Kinect for Windows SDK beta refresh. We plan on releasing the next refresh of the Kinect for Windows SDK beta later this year (still with a non-commercial license).
Let us know what you think—as this refresh demonstrates, we’re committed to using your feedback to make the best possible SDK!
—Tony Hey, Corporate Vice President, Microsoft Research Connections