Download Research Tools
Time to celebrate: we are releasing the Hawaii OCR (optical character recognition) service this week! This OCR service is the next step in the evolution of Project Hawaii, the Microsoft Research project that is exploring how to take full advantage of the cloud to enhance the use of smartphones. With Hawaii OCR, you can use your smartphone's camera to take a picture of an object that contains text (in Roman characters), send the image to the cloud, and in return receive a Unicode string of the text. This text string can be used in a number of interesting scenarios, such as translation of street signs or restaurant menus.
Adding OCR to the computational, mapping, and identification services in Project Hawaii is another step in our journey to create a set of cloud-enabled mobile applications and support services. Our current platform consists of a Windows Phone 7 smartphone and several cloud services, including Relay, Rendezvous, Speech to Text, and Windows Azure for computation and data storage.
Team Dynovader visits the Microsoft Research, Redmond lab. Pictured left to right: Evie Gillie (teaching assistant), Vignan Pattamatta, Mike Ortiz, Arjmand Samuel (Microsoft Research), Naran Bayanbat, Forrest Lin, Lu Li
In related news, we welcomed a group of Project Hawaii collaborators from Stanford University to our Redmond, Washington, lab in late March. Our guests were students in Jay Borenstein's Computer Science 210 (CS210) course, which provides students the opportunity to collaborate on a real-world project with a corporate partner. This semester, Microsoft Research is sponsoring a CS210 cadre on Project Hawaii. The student group, named Team Dynovader, is working on a citizen science project called myScience, enabling scientists to crowdsource data collection for their research projects at the click of a button. No coding is required by the scientists. It allows Windows Phone 7 users to contribute data to various citizen science projects that use the same mobile app.
Here's how it works: scientists will go to the myScience website and launch a citizen science project. It is then automatically deployed to users who download myScience from the Windows Phone Marketplace. Users can browse through a catalog of projects and contribute to those they find interesting. The data is then stored in the cloud, and made available to scientists via our website.
Team Dynovader believes that myScience will transform the way observational research is conducted in the future. Imagine a network of thousands of mobile phones—each with a camera, microphone, GPS, and accelerometer—conducting observations and pushing the data to a central repository.
To quote Scott R. Loarie, a scientist in the Department of Global Ecology at Carnegie Institution, Stanford University, "Mobile phones coordinated through citizen-science projects are emerging as a powerful new tool for data collection. They rival distributed sensors, such as satellites, in their ability to scale and complement these systems, because boots on the ground brandishing cellphones can detect many things that fixed sensors cannot."
Team Dynovader presents myScience project to Microsoft Researchers
We are looking forward to the release of myScience and its adoption by scientists. Good luck Team Dynovader!
—Arjmand Samuel, Research Program Manager, Microsoft Research Connections
These days, much is made of applications that run in the metaphorical cloud. Well, here's an example of hardware and software that soared through the clouds, both real and metaphorical. On March 4, the ASTRA 7, a stratospheric gas balloon carrying a mobile phone running the Windows Phone 7 operating system, was launched from the Cotswolds in west-central England. The hardy phone made its way through the real clouds and into the stratosphere, recording and sending location data that was processed through the virtual cloud of Windows Azure. Part of the University of Southampton's ASTRA (Atmospheric Science Through Robotic Aircraft) initiative, the launch was designed to test the capabilities of the Windows 7 mobile computing platform in capturing, analyzing, and transmitting location data from unmanned vehicles in the upper atmosphere.
The phone's logger application included a "hunter mode," which allowed ASTRA staff on the ground to track the payload during its flight, thus enabling its recovery. The application uses Bing Maps to display the location of the balloon payload, the hunter's phone, the locations of the other hunters, as well as the predicted landing location, which was constantly re-computed in the cloud by Windows Azure as new location reports beamed down from the on-board phone.
The ASTRA 7 reached a maximum altitude of 18,237 meters during a flight of 1 hour 16 minutes, soaring deep into the stratosphere, where the ambient pressure was less than 10 percent of its sea level value and the temperature dropped to -58 C. The maximum speed reached by ASTRA 7 was approximately 145 kilometers per hour, logged at an altitude of 10.1 kilometers as the balloon traversed the jet stream. ASTRA 7 landed about 75 kilometers downrange—very close to the pre-flight prediction based on the ASTRA balloon flight simulation model. ASTRA 7 also took more than 1,200 photos during its flight, a small selection of which are included in this blog.
The phone and the rest of the equipment were protected by a high-grade cell-foam enclosure to ensure the reliable operation of the on-board electronics in the extreme environmental conditions of the upper atmosphere. The enclosure was manufactured by using a computer-controlled laser cutter at the university's Engineering Design and Manufacturing Centre. As part of the payload bay's development process, the ASTRA team tested the foam enclosure in a vacuum chamber to ensure that its mechanical properties would be satisfactory in the extremely low-pressure environment of the stratosphere.
On March 8, ASTRA launched a longer flight to see how the technology would cope with more prolonged exposure to stratospheric conditions. The payload, consisting of a Windows Phone 7, battery, and camera, remained airborne for approximately 2 hours 40 minutes, covering about 110 kilometers in the process.
ASTRA scientists are extremely pleased with the performance of the Windows 7 package, which fits perfectly with the initiative's goal of developing and testing platforms capable of delivering scientific instruments via unmanned vehicles to altitudes ranging from the planetary boundary layer to the upper stratosphere. Dr. András Sóbester, leader of the ASTRA initiative, summed it up nicely: "We are excited that this constitutes a unique opportunity to collect important data that will give new insight into how the upper atmosphere affects Earth's climate and environment, using affordable technology."
—Geoff Hughes, Academic Strategy Advisor, Microsoft UK Developer Platform Evangelism
There is a saying dating back to the days of punched cards that "the software is in the holes"—and therefore invisible. At the recent Microsoft Research Software Summit in Paris, software was anything but invisible. It was all around us and manifest in the smartphones, gadgets, and light tables, and on the huge screens that circled the beautiful foyer of Microsoft France's Le Campus conference facility.
The first change that struck us was the availability of the conference schedule on our Windows 7 smartphones, courtesy of Thomas Zimmermann and Christian Bird from Microsoft's Research in Software Engineering group, RiSE. With the touch of the phone screen, attendees could see what was next, in what room, as well as all the abstracts, bios, and pictures of the speakers. While not the first conference app developed for smartphones, it certainly had a touch of class and showed off the speed and ease of use of the Windows Phone 7 user interface.
Nikolaj Bjorner, Chris Bird, and Thomas Zimmermann show off the Windows Phone 7.
While looking at their phones, attendees were drawn to the other apps that were ready for download, as featured on the post-summit website. One of these, TouchStudio, was featured in a keynote address by Wolfram Schulte. In a slick demo, Nikolai Tillmann showed how his team had overcome “the tyranny of the fingertip” by shifting the process of programming on a smartphone from typing programs letter-for-letter to choosing from options by tapping on the screen—once again using the layout and design of Windows Phone 7 to the fullest. On Friday afternoon, there was a workshop on mobile computing where we were all encouraged to write programs in TouchStudio. Because it was so easy and short, I am taking the unusual step of including my first TouchStudio program in this blog.
TouchStudio code example
Taking a photo is actually a very complex operation and would take several pages of C# code to get right. Here we have it in one line. The endless possibilities for scientists, hobbyists, students, and children to access the power of the phone through TouchStudio were not lost on us seasoned academics in the audience.
Another very visible piece of software was on display in .NET Gadgeteer, a unique mix of programming and a kit of hardware modules. Gadgeteer enables users to quickly assemble useful, fun gadgets that have the ability to display images, play back sounds, take pictures, sense the environment, and communicate with other devices. It was great fun to watch computer scientists, designers, and even psychologists furiously and enthusiastically building gadgets.
Of course, a summit is not all about coding and listening to talks. The 230 academics, industrialists, and researchers thoroughly enjoyed the long breaks over delicious French food, discussing the demos on display and the sessions they had attended.
One of the sessions that attracted a standing-room-only crowd focused on Verified Software. There, Cambridge lab researchers, including Tony Hoare, teamed up with Thomas Santen from the European Microsoft Innovation Center (EMIC) and industry representative from nearby Europe and far-away Australia. Together, they presented the latest results in software verification for the all-important embedded software industry. We got a rare glimpse into what goes on behind the scenes before software is put into devices in cars, trains, and planes.
There was so much more, but fortunately, we have the websites to go back to and can review the program and download the software. All the talks and slides will also soon be posted, and what a feast that will be for those who attended and those who could not.
—Judith Bishop, Director of Computer Science, Microsoft Research Connections