Microsoft Robotics Blog

Call for Robotics Projects!

Loke Uei — Fri, 29 Jun 2012 18:40:37 GMT

Hi all,

My name is Loke, I’m the Sr. Product Manager here at the Robotics team in Microsoft. We are currently in the process of redesigning our website and one of the new areas we’re thinking of creating is a Showcase section where we’ll highlight some of the cool projects that our community is working on with regards to Robotics on Microsoft Technologies, including RDS.

If you are currently working on a project, or using RDS or any other Microsoft technologies for building Robots (both software and hardware), we’d appreciate it if you could send us a short description of what you’re doing, how you’re using the technologies, and even some feedback about the process. We’d love to hear from you.

Please send the details to roboinfo@microsoft.com. Looking forward to hearing about your awesome projects.

Loke

Microsoft Robotics team will be at the Seattle Mini-Maker Faire!

Loke Uei — Tue, 29 May 2012 18:35:52 GMT

The Seattle Mini-Maker faire will be at the Seattle Center on June 2nd and 3rd over the weekend. If you're in town during that time, please drop by to see really cool projects made by hobbyists in the community. Our team will be there en-force to show off Robotics related technologies, such as Robotics Developer Studio, Kinect for Windows, Mayhem Project, .NET Gadgeteer and Kinect Fusion.

Announcing the Winners of the Microsoft Robotics @ Home Competition

Habib Heydarian [MSFT] — Tue, 22 May 2012 22:18:00 GMT

The wait is finally over! We announced the winners of the Robotics @ Home competition at Maker Faire Bay Area 2012 which was held on the weekend. The event was a huge success with over 100,000 people in attendance.

All three winners spent two days with us in the Microsoft Robotics booth at Maker Faire, and it was a blast. Here is a photo of us trying to frantically setup our booth!

And the Winners Are...

Second Prize goes to Dan Sionov and the Plant Sitter team, who won $3,000 for their plant- watering robot. We loved the idea of having an autonomous robot take care of plants. Here’s Dan (on the right) accepting his well-deserved prize from Ira Snyder, the General Manager for Microsoft Robotics. And here’s a video of the Plant Sitter in action.

The first prize winner is the KEMODA team with the elderly assistance robot, which was built by Todd Christell. Healthcare is a huge growth opportunity for robotics, and Todd did a great job demonstrating the possibilities as seen in this video. Here is Todd (on the right) accepting his $5,000 check. Well-deserved Todd!

And the grand prize goes to … the SmartTripod robot, designed and built by Arthur Wait. His SmartTripod robot really captured our imagination and demonstrated the type of scenarios that can be solved using robotics as demonstrated by this video. Arthur’s use of the Kinect to implement “Cue Zones” is both unique and very innovative. Here is Arthur (on the left) accepting his grand prize of $10,000. Congratulations to Arthur!

We’d like to thank everyone who participated in the Robotics @ Home competition. It was great to see how the participants used the EDDIE platform and Microsoft Robotics Developer Studio to bring a robot to life, performing useful, and in some instances important tasks.

From all of us on the Microsoft Robotics team -- congratulations!!

Habib Heydarian
Microsoft Robotics

Robotics @ Home Competition Update

Habib Heydarian [MSFT] — Tue, 15 May 2012 16:19:00 GMT

The Robotics @ Home competition officially closed on April 30th and we now have the final submissions! Congratulations to the finalists who successfully completed the project. We are planning to unveil the results of the competition and announce the winners at the Maker Faire Bay Area on Saturday May 19th at 10:00 AM.

If you are at Maker Faire Bay Area this weekend, come join us at the Microsoft booth and see the robots in person. We will also be sharing all the details about the winners here soon. Stay tuned for more details!

Habib Heydarian

Microsoft Robotics

Welcome to Microsoft Robotics Developer Studio 4

Habib Heydarian [MSFT] — Thu, 08 Mar 2012 14:00:00 GMT

Today, we’re delighted to announce the general availability of Microsoft Robotics Developer Studio 4 (RDS 4) which can be downloaded for free from the Microsoft Robotics website. It was just over five months ago that we announced the availability of RDS 4 Beta and since then, the Microsoft Robotics team has been hard at work putting the final touches on RDS 4 to give developers access to the software they need to build robotics applications.

Since the Beta, our number one focus has been on improving the overall quality and stability of RDS 4. With your feedback, we’ve fixed a number of important issues in the product as well as made improvements to existing samples. Additionally, we updated RDS 4 to work with the release version of the Kinect for Windows SDK.

We are already seeing RDS 4 being utilized to develop some interesting applications. Speaking of which, last fall, we announced the Robotics @ Home competition, and are thrilled with the submissions we’ve received. Today, we’re excited to reveal the competition’s 10 finalists. Visit the Robotics @ Home Finalists page to read more about the finalists, including how they think robotics will change the world and what they would do with the prize winnings. We look forward to announcing the grand prize winner in May of this year.

Also, our own team has been using RDS 4 for a while now and we’ve come up with a few cool and unique applications. Check out the video of the Kinect Follow Me robot which was created by our team.

(Please visit the site to view this video)
From all of us on the Microsoft Robotics team, we’d like to thank you for your continued support and your feedback on RDS 4. As always, you can use the forums on the Microsoft Robotics website to send us any additional questions and suggestions.

Habib Heydarian

Microsoft Robotics

Pet Companion Robot using RDS

Trevor Taylor — Fri, 10 Feb 2012 17:40:44 GMT

Building a Pet Companion

When my wife and I both worked fulltime, we had to leave our dog Darwin home alone for 7-8 hours every day, so I decided to build a pet companion to keep him company. Primarily, I wanted to build something that would entertain Darwin, but I also thought it would be fun to create something that would allow me to check on him from work.

My vision for a pet companion was actually less of a robot and more of a remotely operated vehicle. In fact, the only autonomous behavior in the robot is its ability to find and pick up a ball and reload the ball launcher. Most importantly, I wanted to be able to interact with the remote environment and have fun doing it. The “fun” aspect was the most important and had a lot of impact on my design decisions.

I started with the Microsoft Robotics reference platform and the Microsoft Robotics Developer Studio software. My process for designing and building the robot was basically trial and error. I didn’t have the skills to use 3D modeling programs well enough to be effective, so I would embrace an idea and give it a try. I built early prototypes using “instant gratification” processes and materials. In fact, the first prototype was built almost entirely of cardboard and hot glue. Once I had a design that seemed to fit and function, I upgraded to chemically welded styrene and aluminum parts. In some cases, it took several major iterations before I found a design that actually worked. In fact, while the current treat dispenser seems pretty simple today, it is actually the third major iteration that I built to completion because the first two had jamming issues.

Building a robot is a challenging endeavor, especially for someone like me who has little electronics experience, no mechanical engineering knowledge, and a limited budget. I would categorize my most common struggles into two areas, first and probably foremost was budget.

Early on in the project, I was actually focusing on getting the robot to work using a BasicX microcontroller and an IP-enabled web cam (because I happened to already have those things lying around). I spent many hours trying to build a pan/tilt for the big bulky web cam and trying to program my BasicX to be a decent servo controller (let alone network server etc.). Much of this work turned out to be a waste of time compared to the relatively meager cost of good servo controllers and cheap netbook PCs. One key lesson learned is I should not shy away from spending a few bucks to achieve my vision.

The second major frustration that kept rearing its head was part fabrication. In many cases, I thought up a design for a part or component, only to be frustrated by the fact that I couldn’t make it work! I spent many hours at the local home improvement, toy and hobby stores trying to find an available product that had what I needed. This issue definitely killed many of my ideas. But, learning from my first challenge, I recently ordered a RepRap Mendel 3D printer and am hopeful this will be less of an issue in the future.

Overall, designing and building a pet companion was a fantastic learning experience and was a lot of fun. Now if I can only get the pet companion to walk my dog for me…

This article was written by Jordan Correa, a Test Developer on the Robotics Team. Jordan’s robot has evolved through several iterations with a lot of hand-built hardware and custom software. You can view a video about Jordan’s robot at http://www.youtube.com/watch?v=afvdZ343-vE

Behind the Scenes: Motion Tracking Robot Controller at Maker Faire 2011

Gershon Parent — Thu, 08 Dec 2011 08:00:00 GMT

On November 15th Elliott posted a blog on the The Motion Tracking Robot Controller. I was fortunate enough to work on this project and host an interactive demo of it at Maker Faire 2011 in New York this last Fall. You can see what the installation looked like in the second half of this video:

Microsoft at Maker Faire: Gadgeteer and Kinect

It was a great experience and I would like to share some “Behind the Scenes” info about what it was like being there.

Maker Fair 2011

This was my first time at Maker Faire, and it was amazing. If you haven’t been, I highly recommend it. We had the Robo-Razzi robot roaming taking pictures to the delight of the crowds (Smile for the camera!). I spent nearly all my time in the Microsoft tent running the Kinect demo for scores of people each day. My arms were sore at the end of each day but it was worth it… we shared some cool technology with thousands of people and got a blue ribbon award for an “excellent exhibit”.

The ‘Driving Eddie with Kinect’ demo was a big hit. There were lines to try it much of the day both days. I had to make some last minute adjustments to the code when we first set it up and fine tune it for one of the bots, but other than that, it ran flawlessly for hours and people were just amazed. The only down time was to swap laptop batteries every few hours or to switch to a charged robot toward the end of the day and I got those down-times to under 3 minutes.

I can’t begin to tell you how many giggling kids and fascinated adults absolutely loved it. I got to see people’s faces light up all day long. I was constantly saying “OK… we have a line… times up… who’s next?” I repeatedly heard… “That is just amazing”… “Incredible!”. So many people wanted to know more and loved hearing all about it and many wanted to try it over and over again. I saw Parents take extra time to explain to their very young kids what was going on and make sure they realized how amazing it was. You could tell they saw this as something important that should be understood. No one was disappointed, everyone left with a smile and many took time to learn more about the EDDIE and the software. It was a great lead-in to talking about RDS4, the contest and Microsoft’s vision of advanced software for robots.

Everyone tall enough to see over the monitor into the robot pen could get the hang of how to drive it quickly and it was fascinating to watch them learn to drive and develop their own driving style. The best thing was when people would come and watch the robot moving around in the pen for a while without knowing how it was being controlled… then they would see someone to the side with her hands out rolling back and forth like an airplane and they would put one-and-one together and I could see them just light up with excitement and get in line. Then there were people who got in line without knowing what was going on. They saw people waving in front of a big screen and wanted to try. When they stepped up and I told them they would be controlling that robot over there with their body they couldn’t believe it and when it responded to them the smile was priceless. It pleased men, women, boys, and girls of all ages.

I purposely had some of the demo code showing behind the client app and lots of people asked about it… “Is that the code”, “did you write that”, “what language is that?”, “was it hard to write?”, “how many lines of code?”. I would take time to answer them or direct them to another team member so I could keep the line moving.

There was one young woman who got my usual fifteen second instruction and then immediately was driving it around like a pro… I could tell by the way she stood and the fluid, graceful motion of her arms, and how her wrists and fingers perfectly matched her arm movements that she was a dancer… probably ballet. The robot was gracefully swimming around the pen better than I had ever seen before. I asked her if she was a dancer and she smiled and said she was. I told her that the robot was moving more fluidly and beautifully than it had for anyone before her and she smiled and continued to make it dance and swirl for a few minutes. I was so impressed. I felt like I was in a PBS special about computer human interaction.

Another young man wanted to know if it would track someone standing on their head. So did I, and then next thing I know he is driving the robot around with his legs much to everyone’s delight. In that orientation, the Kinect software interpreted his legs as arms and away he went.

Several people asked “How is this useful?” or “What are you hoping to accomplish”. I would explain that this is a technology demonstration intended to spark the imagination and invite the user to experience first hand becoming embodied in a simple robot. While the specific setup we had there might not have a practical application besides entertainment I think we accomplished that goal. As robots become more common in our world, exploring new ways of interacting with them using powerful user interface devices like Kinect is both fun and exciting… and I’m pretty sure we sparked an imagination or two.

What would you do with an Eddie robot in your home? Enter for a chance to win $10,000!

Announcing Robotics Developer Studio 4 Beta 2

Trevor Taylor — Wed, 07 Dec 2011 20:18:30 GMT

A new release of Robotics Developer Studio 4 Beta 2 is now available that is an update to the Beta released in September at Maker Faire in New York. This new Beta 2 version includes the following:

Support for the Kinect for Windows SDK Beta 2
DSS Log Analyzer
Structured Logging
Improved Obstacle Avoidance Service with a Visualization UI
CCR for Silverlight 4.0, including new Extension Methods
Additional documentation

This version of RDS requires Windows 7 and Visual Studio 2010 (C# Express or higher). You can get a copy of RDS 4 Beta 2 free of charge from the Download page.

Trevor Taylor
Senior Program Manager
Microsoft Robotics Group

What is a Household Robot?

Greg Shirakyan — Thu, 01 Dec 2011 15:33:00 GMT

When presented with that question, most people don’t ask for clarification. They intuitively know what the question means, even children do. And then, after naming one or two examples that invariably include a robot tidying up a house one way or another - people often stumble. Children, with their less constrained imagination, dream up “Transformers inspired” big brother robots, or ones that would do their homework and play games. My kids were interested in a robot with a magic wand, but let’s not discuss synthetic Harry Potter for now. We’ll come back to him, though.

“If I had asked people what they wanted, they would have said faster horses.”
― Henry Ford
In virtually all cases, people look at their daily lives, their familiar environment and then try imagining how a robot would fit in. No wonder the robot comes up to be either an unrealistically advanced humanoid or a floor-roving disk about a foot across.

Homework first (no pun intended)

To talk about a “household robot”, let’s first define the term. For the lack of a better definition – we’ll go with “a device that can intelligently move parts or all of itself around a home and do useful stuff”.

The goal of this definition is not to be academically acceptable, but to help us zoom in on a specific set of qualities without implying implementation. The term “intelligently” in this context merely means “not being continuously controlled by a person, and not getting stuck”. Why do we want it to move at all? Because it’s got to have at least some degree of freedom, reach places and interact with the physical world – otherwise it would not be called a robot. And the more mobile it is – the more ‘robotic’ it feels. Somehow, we humans view autonomously moving objects as ‘alive’ regardless of how intelligent they are. For instance, most Roombas are given a name. But I do not know a single PC owner who bothers to name their desktop, even though a computer is much ‘smarter’ than the randomly roaming vacuum cleaner.

If we visited a supermarket today and looked for products that fit the above description, we’d end up with devices that can be categorized in 2 main groups:

Toys designed to appear intelligent, though usually quite simple inside.

Floor cleaners (and maybe Mowers) - again simple, but useful single-purpose devices.

That’s pretty much it. Get it? Robots have been doing a lot of ‘useful stuff’ on other planets for decades, in manufacturing, versatile Industrial robots have been a norm long before the PC was born, and the military embraced robots earlier then you might think, so why are they so rare in our homes?

Let’s explore some of the reasons, and maybe that will help us get a glimpse on what to expect going forward.

Reason #1 – Discriminating Environment. Our homes, appliances and tools are optimized to be used by people (hardly a surprise) and people only, i.e. if you had a cat with a human IQ of 120, what ‘useful stuff’ could the poor puss do in an unmodified modern apartment? We can’t expect it to be able to whip up an omelet or do the dishes for example. The best we can hope for – is a friend and companion that we must take care of.

Forget cats. Our homes are tricky places for our own children. How would you like eating with a 2 foot long fork from a dinner table that’s taller than you are? Welcome to the world of a 3 year old. But we love our kids and pets and therefore enhance our living space to make them happy.

Any robot looking to inhabit a human dwelling and be ‘useful’ in the utilitarian sense of that word - faces similar challenges. It would have to resemble at least some of the physical and mental characteristics of a grown person in order to perform even mildly complicated tasks. Just imagine the level of dexterity and contextual awareness a robot would need in order to make a pizza in your kitchen. It would have to find, evaluate and pre-process raw ingredients using tools and appliances designed for human hands, eyes, ears, noses, and even human psyche. Furniture, lids, peelers, wraps, labels, the sink, dishes, pans, stove … Makes me smile. Not going to happen. What’s ironic about this example is that a robot is destined to be a far better cook than the vast majority of humans.

Let’s drill down on this one. Think about it - taking raw ingredients and turning them into a descent stew is not rocket science, after all, even a complicated recipe is just an algorithm with a few dozen standard steps. A machine can do this. We just have to design an appropriate environment. A robot-friendly kitchen must have different tools, and different infrastructure for accessing water, fridge, stove, pantry, drain, trash, etc. It would not take a humanoid to operate such a kitchen, in fact it would be awkward. An ‘integrated’ robot would make more sense, perhaps resembling some of the semi-stationary industrial machines used today in manufacturing with manipulators capable of reaching any item in ‘robot-accessible’ storage, a variety of ‘end of arm’ tools that one minute can be whisking eggs, another – flipping a pancake.

Me: “Hi Kitchy, we are having six guests today. We’ll need a nice 3 course meal. Indian. Not very spicy, ready to be served by 6:15. A salad too. Make sure it’s not very heavy and at least one dish is gluten free.”

Robot: “Any non-vegetarian dishes?”

Me: “Sure, add one.”

Robot: “I am going to make … (lists the menu)”

Me: “Sounds good, and squeeze some fresh orange juice. Bye.”

Can’t wait for the day I’ll have this phone conversation.

“From a new born to a 5 year old – there is a world of distance. From a 5 year old to me – only a step.”

― Leo Tolstoy

Reason #2 – Social Un-Intelligence. I am no expert in the rich and fragmented field of AI, but it is pretty clear that we have yet to overcome some serious hurdles in the human-machine interaction domain. There is a tremendous amount of context to human communication, and we are just beginning to make baby steps in modeling the mechanics of it. But why are we talking about intelligence? Do robots have to be intelligent at all? Aren't socially unaware robots useful?

Sure they are. We can call those – Blue Collar Robots. Cleaning, cooking, maintenance, security, etc. They take on jobs that people must do, but hate doing, don’t want to do, or simply can’t. It’s safe to bet on a steady increase in the number and variety of such machines. Low hanging fruit that include floor cleaning and mowing are already being tackled by a variety of vendors. A step up in complexity – security and surveillance may come next.

And then there is a White Collar Robot – one that helps us do things we love doing. I am most fascinated by this breed. Satisfying curiosity, being creative, socializing, studying, having fun – those sorts of activities are strictly human and therefore only human-centric robots can actively participate and enhance our experiences. We may change our environment to help blue collar bots excel at their jobs, but we can’t modify our own nature, so a white collar robot has a difficult task of grasping the intricacies of human world. And that’s where the cutting edge of science is still not sharp enough. Cute toys and more toys seem to be the only available game in town. There are, of course many research projects, but widespread commercialization of technology has not started yet.

Note, a White Collar robot alone is neither necessary nor sufficient to bring about a household robot boom. But it could greatly accelerate the process because Reason #1 does not apply here quite as strongly. We would readily forgive a few clumsy moves and lack of utility if those were traits of our delightful companion. We would also be open to modify the environment a bit to make our new family member happy, and that’s exactly what the robot race needs to occupy our homes for good. So, an intelligent human-centric robot is quite important. This brings us to Reason #3.

Reason #3 – A Reference Design. Or rather – an apparent lack thereof. We take the basic concept of a personal computer (screen + keyboard + ‘magic box with wires’) for granted now, but it was a great discovery. It has proven to be incredibly versatile, and through custom software let us engage in fundamentally different activities: have fun, do work, learn and communicate. All this using conceptually the same hardware configuration and open design, which made it easier for new industries (i.e. PC software, peripheral devices) to appear and grow quickly. There seems to be no similarly versatile household robot design yet. As mentioned before, it is more than likely that we’ll see more of specialist bots in the near future, but it raises a question: who is going to manage them? Eventually it is us, of course, but constantly reprogramming a legion of independent machines to fit our volatile calendar – does not scale. We will need a ‘central command center’ of some sort, or … a white collar concierge robot to act as a liaison between its less social hard working brethren and people. What physical, sensorial and communicational characteristics would such a device need in order to do ‘all things robotic’ the same way a PC can do “all things digital”? How intelligent will it have to be? Will it need a screen? Manipulators? Will it even be a physical entity or a virtual one living on the home network, or somewhere in the cloud? And will we even call it a robot at that point?

No one knows for sure. The only way to find out is to take a few educated guesses and give it a try. That’s exactly what we did at Microsoft with the MARK design (Mobile Autonomous Robot using Kinect). We believe it’s a great platform for imagining up household robots. Here is one example. Microsoft Robotics Developer Studio 4 supports MARK in the simulator, and RDS services will be available when our partners start selling the hardware. You won’t have to do soldering, debug firmware or write low level control code. All built in devices, including Kinect are supported by RDS, and can be accessed from high level, C# code. As my friend Gershon put it – “a robot should be something you program, not something you solder”.

“One man's magic is another man's engineering.”

― Robert A. Heinlein

When my 8 year old daughter wished for a robot with a magic wand, and my 5 y/o empathically agreed - I initially dismissed their silliness. Then I thought a bit more, defined a magic wand as “a portable device that takes commands from its holder and changes the state of objects” and realized it was me who was being silly. We all have magic wands, of course. The fact that remote controls are relatively single-purpose is not a function of technological limits or even cost, but of practicality. Sure there are ‘universals’ that can operate just about anything that supports a remote, and can even be pprogrammed for new devices, or ‘learn’, but let’s face it: universal remotes are Swiss Army knives – jacks of all trades, and masters of none. They may be cumbersome to set up and they work reasonably well only if all devices are close to each other. As soon as you have more than one room with remotely controllable electronics – a single controller becomes impractical. A robot does not have a problem with misplacing its ‘remote’ or forgetting ‘which button does what’. It can easily interact with 100s of devices ranging from light switches to … other robots in different locations over different protocols and interfaces. Infrared, radio, network, you get the picture. Robots managing robots on your behalf. How do you like that? And it’s not even far-fetched.
“Magic Wand Robot Edition ©” would make most real wizards envious. For it is a 2-way wand. Not only can it ‘command’ other objects, but ‘hear’ them too. RFID technology is so cheap, that it’s not hard to imagine that most items in your house can be tagged. “Daddy, where is my blue doll?” “Don’t know honey, ask Robbie”.

“From the one who has been entrusted with much, much more will be asked”

― Luke 12:48 NIV

aka “With great power comes great responsibility”

― From Spiderman

Security and Privacy ... Imagine a legion of robots that collectively know everything about your home. They are mobile, interact with your family members, mix your drinks, have many eyes, ears, sensors, actuators, the “magic wand” … They are ‘plugged in’ to the very heart of your personal life. Now picture those robots compromised and remotely controlled by a hijacker. The devastating effects this might have far supersede the worst that can happen if a virus turned your PC into a zombie. We may not know much about how domestic robots of the future will be designed, but one thing is certain – security, safety and privacy will not be thrown in as afterthoughts.

“I never predict anything, and I never will.”
― Paul Gascoigne

When will house robots start playing a significant role in our daily lives? It’s a tricky question. First, technology needs to be released. Then, if it survives present day, we adapt to use it, and it adapts back to our new usage patterns, which changes how we interact with it further, etc. In the midst of this infinitely recursive stochastic process, our views on what technology ‘should’ look like is constantly redefined and quickly taken for granted in ways that neither consumers, nor technologists or sci-fi writers can predict.

Sorry to disappoint, the household robot invasion is not happening tomorrow. Not only because the technology is not ready, but because our mentality and our surroundings have not been modified yet to welcome robots en masse. If that sounds like a catch-22 – it’s because it is. But do not despair, the initial wave of robots will be designed (or is already designed) to survive the ‘stone age’ environment of the modern man, and that’s where it starts. Remember, the first successful electrical appliances were powered through a light bulb screw adapter ...

So, what is a household robot after all? It’s a class of machines without which we can’t imagine our daily lives. The majority of them are common, cheap and easily recognized, many are hidden, some are specialized, others - cool and highly coveted. They handle dirty work, inspire our imagination and do everything in between. Our homes are designed to accommodate them. Indeed, you can’t tell where the home ends and robots begin. We depend on them, partner with them, and in return they empower us. As of Nov 2011 - they are just beginning to be imagined. Too bad I can’t patent this description.

Kinect for Robotics

Trevor Taylor — Wed, 30 Nov 2011 05:40:33 GMT

The Kinect sensor is one of the best devices, for its price, to become available for robotics in the last decade. For $150 you get 3D range (distance) data and RGB color (webcam video) data. There is a microphone array thrown in as well. But wait! There’s more. The Kinect can detect people and generate skeleton data and you don’t have to write a single line of code to do the processing – the Kinect for Windows SDK does all the hard work for you.

How it Works

The Kinect uses an infrared (IR) laser to spray out a pseudo-random beam pattern. An IR camera captures an image of the dots that are reflected off objects (as in the picture below) and the electronics inside the Kinect figures out how much the dot pattern has been distorted. The distortion is a measure of distance from the camera. This approach is commonly called Structured Light. All this happens at 30 frames per second – not bad for a cheap consumer device.

As a programmer, the Kinect gives you an array of depth values that correspond to the pixels of the RGB image. This unusual coordinate space, consisting of x and y as pixel coordinates and z as a distance in millimeters, is called the Depth Image Space. All of the distances are measured from a virtual plane passing through the Kinect camera. You can convert the data into conventional (x, y z) in meters but this requires additional processing overhead.

Skeleton Space on the other hand uses conventional (x, y, z) coordinates. A skeleton consists of a set of 20 joints, each with its own 3D coordinates. Detecting gestures is relatively easy. For example, to detect a person waving their arm over their head you just need to compare the height of their wrist to see if it is above their head, and then see if it is moving from side to side.

Comparison to Laser Range Finders

Laser Range Finders, or LIDAR (Light Detection and Ranging) devices, have been on the market for a long time. A LRF works by sending out pulses of infrared laser light and timing the return signal. Therefore they are known as Time of Flight devices.

The German SICK brand of LRFs have long been the workhorses of the research community, but these cost thousands of dollars. More recently, Hokuyo in Japan has been selling a cheaper range of LRFs that are approaching the $1,000 barrier. However, for this amount of money you can buy six Kinect sensors, although you would need enough USB ports to plug them all in and the necessary processing power to make use of all the 3D data.

The primary differences between a LRF and a Kinect are the Field of View (FOV), Maximum Range and Resolution. A conventional LRF has a FOV of 180 degrees, and some are up to 270 degrees. In contrast, the Kinect only has a 57 degree FOV.

LRFs are available in a variety of ranges from as little as 2 meters out to well over 100 meters. The Kinect, because it was designed for use in a living room, has a maximum range of 4 meters.

Even with a very large maximum range, the resolution of a LRF is from millimeters to centimeters (depending on how expensive it is) and the accuracy is constant across its entire range. Distance data from the Kinect varies in accuracy from sub-centimeter up close to as much as a 5cm error at its maximum range. This is a consequence of how it measures distance.

One downside of a LRF is that it operates in 2D, effectively taking a horizontal slice through the environment around the robot. In order to capture 3D data, the LRF must be mounted on a tilt mechanism and tilted up and down or mounted sideways and panned from side to side. This makes capturing 3D data relatively slow.

The Future for Kinect

Currently the Kinect has a limited range of 80cm to 4 meters. This means that the Kinect cannot see objects that are right in front of the robot, so you still need traditional obstacle sensors such as sonar (which is why they are included on the RDS Reference Platform). Next year, when the Kinect for Windows Hardware is released, there will be a new “near mode” that will range from 50cm to 3 meters. Although this helps with detecting nearby obstacles, a robot should still have other sensors for redundancy.

Going beyond next year, Microsoft will continue researching even better Kinect hardware. This means that 3D depth data is now here to stay, so sharpen up your 3D geometry skills and get cracking on applications that take full advantage of these new devices.

The Motion Tracking Robot Controller

elemenager — Tue, 15 Nov 2011 17:51:00 GMT

The romote control has come along way from the first remote patent by Nikola Tesla in 1898 and the first wireless TV remote “The Flashmatic” that used photon cells as shown in the picture below.

Fast forward 100 years and remote controls are breaking new ground for Human-Robotic Interaction (HRI) by leveraging Natural User Interface (NUI) allowing users to carry out relatively natural motions, movements or gestures which in return control computer applications, manipulate on-screen content or in this case control a robot.

How it works

EDDIE, the reference platform used for this demo comes with an 8-core Propeller microcontroller to directly control two 12v motors. These motors can be control remotely or Eddie can roam autonomously by leveraging several sensors around the robot and see in 3D using Microsoft Kinect. Gershon Parent , a developer with the Microsoft Robotics group, has added a new twist on how EDDIE can be wirelessly controlled which he’s dubbed the “Motion Tracking Robot Controller”. By leveraging skeletal tracking through a Kinect sensor Gershon can control the two 12v motors through arm gestures navigating EDDIE through his environment.

(Please visit the site to view this video)

When standing in front of the Kinect sensor, Gershon’s right hand controls the right motor and his left hand controls the left motor; it’s kind of like a tank driver. When he raises both arms simultaneously the robot will move forward in a straight line and the higher he raises his hands the faster the robot will go. To put the robot in reverse he simply lowers both of his arms at the same time. To turn the robot all he has to do is tilt his hands one over the other from side to side to give the robot the desired degree of turn.

What’s next?

Gershon’s demo is a great example on how NUI can be used to control a robot, but it’s just the tip of the iceberg. There are many different types of controls that can be built through a Kinect interface – a steering wheel or joystick type of control, for example. Also, as shown in the video the robot has a Kinect sensor on it where it could be sensing its own environment, detecting obstacles and relaying this information back to the user. EDDIE’s Control Board provides additional I/O allowing a wide variety of sensors and accessories like cameras as seen before in Roborazzi.

Get involved

Do you have an idea like the Motion Tracking Robot Controller? We would love to hear from you and you can submit an entry in our Robotics @Home Contest. And if you’re already using RDS4, we hope you’ll join our developer community for any technical assistance you might need. And finally, we are always looking to hear from our community; reach out to us anytime on Facebook and Twitter if you want to learn more about what’s going on at Microsoft Robotics or to Geek out on robots with us.

Smile for the camera!

elemenager — Thu, 27 Oct 2011 17:49:00 GMT

It’s estimated that 3.5 trillion photos have been taken since the first pinhole camera. Even if you account for population growth the progression of photography is mind boggling. Did you know that we take 4 times as many photos compared to 10 years ago and Facebook currently hosts over 140 billion of them, with an additional 70 billion more expected this year? (1000memories.com)

Technology has made it possible to capture images of every party, birthday, summer BBQ and family gathering in rich detail while sharing them with friends around the world. So what does all of this have to do with Robotics? Greg Shirakyan, a developer on the Microsoft Robotics team, used Microsoft Robotics Developer Studio 4 (RDS4) and the first implementation of the RDS4 Reference Platform specifications from Parallax (EDDIE) to make it even easier to capture these memories.

Party Photographer aka Roborazzi

Greg’s party photographer is designed to wander autonomously, engaging people by detecting them using Microsoft Kinect’s skeletal tracking capabilities and once they’re in frame EDDIE snaps a pic. Once the image has been captured you can have the picture uploaded to Flickr. Roborazzi alleviates the need for individuals to capture the special memories at an event and allows them to focus their time on creating them. Watch the video below to see Roborazzi in action and a technical debrief of the project from Greg.

(Please visit the site to view this video)

What’s Next

Greg’s Roborazzi is a great solution that plays to relevant trends in human behavior however, it’s not the end all be all solution. What if Roborazzi merged all the photos of an event in a three-dimensional experience like “The Moment” , breaking the mold of presenting images in a timeline? Images are not the only media that EDDIE can capture and parties are not the only scenario where innovative robotic solutions can be created. Your imagination is the limit.

If you have an idea for an at home robot show us what you got in our Robotics @Home Contest. If you want to learn more what’s going on with the Microsoft Robotics team join us on Facebook and Twitter. For technical assistance on building a robot with RDS4 join our developer community and keep an eye out for more @home solutions created by the Microsoft Robotics team.

Updated Samples for Professional MRDS textbook

Trevor Taylor — Mon, 10 Oct 2011 05:33:12 GMT

The samples from the book "Professional Microsoft Robotics Developer Studio" have been updated to work with RDS 4 Beta and are available for download from CodePlex:
http://promrds.codeplex.com

There were very few changes required, but new users will find it more convenient to download the updated samples. Note that the projects in the updated samples are for Visual Studio 2010 because RDS no longer supports Visual Studio 2008. Also you must be on Windows 7 to use a Kinect sensor because the Kinect for Windows SDK requires Windows 7. The RDS 4 Beta currently requires that you install Kinect for Windows even if you only want to use the simulated Kinect. This requirement will be removed in the final release.

The samples will be updated again for the final release of RDS 4.

Announcing Microsoft Robotics Developer Studio 4 Beta

Stathis Papaefstathiou — Sat, 17 Sep 2011 10:30:00 GMT

In my July blog post, we announced the Kinect Services for Microsoft Robotics Developer Studio (RDS) and I promised that we would have some exciting announcements this fall. It is my pleasure to announce the release of RDS 4 Beta available as a free of charge download at Microsoft.com/robotics.

It's inspiring to see the tremendous excitement with Microsoft Kinect within the robotics space signaling the energy and potential opportunities that exist in essentially transforming robots to low-cost mainstream consumer devices. This beta release is one of our early steps towards realizing our long term strategy of accelerating the consumer robotics industry. There are many new capabilities that we are bringing to the latest release of RDS, but I will focus on three major additions I’m particularly excited about.

Unlike computers that primarily implement a single computational model, which means that when I write a sorting algorithm that runs on one computer, the same algorithm will work on another computer even if it uses a different processor technology, has different component configuration, and runs a different software stack. This is not the case with robots that come in many shapes and configurations. For example a navigation algorithm is typically dependent on the sensor and other hardware configurations. This makes it difficult for people to share solutions.

For the first time we define a Kinect-based hardware reference platform that can help in the delivery of an affordable and capable robot that will be able to support technologies and scenarios for the consumer audience. Most everyone can build a robot using this specification, and we are working with robotic hardware vendors that will make kits and preassembled units available. The first such kit is called EDDIE and it is available from Parallax for pre-order now. EDDIE is intended to be a turnkey solution for using RDS with this platform and our aspiration is that there will be development and sharing of technology for this hardware platform. In addition, we made sure that there is an accurate representation of a robot that is based on the hardware platform in our simulation environment. You can start developing your applications in the simulator before you decide to buy and run them on a physical robot.

The second additional feature in RDS 4 is beta the availability of Kinect services that support the Kinect for Windows SDK capabilities shipped by Microsoft earlier this summer. You will be able to use the CCR/DSS programming model to access all of the Kinect for Windows SDK functionality. This unlocks a whole new world around Human Robot Interaction (HRI) with the use of skeleton tracking and speech. But we are also making available the raw sensor stream for your use in building additional capabilities such as navigation algorithms. As a matter of fact, we are shipping an obstacle avoidance service that fuses the Kinect input with the other proximity sensors to demonstrate how the sensor array can be used to intelligently perform directional based navigation.

In RDS 4 we make available for the first time the CCR programming model in Silverlight. Why is this important? Although this new capability might seem esoteric, it is manifesting a baby step towards engaging a broader community of developers who might not have previously been involved with Robotic scenarios. We feel that this is very important because we see that many of the experience and scenario innovations used today in the mobile device space can translate to new unique and exciting applications for Robotics.

In an effort to help spur continued creativity and new consumer robotics scenario within the community, we are also excited to announce our first Robots @ Home Contest. We are inviting developers of all stripes to innovate and create new software that can lead to more personal consumer robotics scenarios. You can submit idea proposals into one of the following categories:

Consumer usage scenarios;
Human Robot Interaction; and
Autonomous Navigation.

Please visit our Microsoft Robotics Developer Studio Web site for more details.

It’s been exciting to see how Microsoft’s Kinect sensor has already captured the imaginations of researchers, academics, and robotics enthusiasts around the world. Our motivation in releasing these tools is to extend and democratize access to robotics development, bringing value to the space through ease-of-use, accessibility, and a robust existing developer community.

As you can imagine, these are hard problems that we are trying to solve. And given the constraints and expectations of the market within the consumer robotics space, we believe that if we combine both Microsoft’s unique technology portfolio with the creativity and enthusiasm of the community, we can develop new experiences and scenarios that can revolutionize peoples’ relationship with Robotics.

--Stathis Papaefstathiou, General Manager, Microsoft Robotics

Get the latest info about RDS at Maker Faire New York

Trevor Taylor — Sat, 27 Aug 2011 23:14:14 GMT

If you are in New York on the weekend of the 17th and 18th of September, come along to Maker Faire and chat with members of the Robotics Team who will be showing off a pre-release of the next version of Robotics Developer Studio. Microsoft Research will also be presenting the Kinect for Windows SDK and the .NET Gadgeteer. Look for the Microsoft tent near the main Maker Pavillion in Zone C at the northern end of the Faire in the grounds of the New York Hall of Science.

CCR Tips and Tricks

Trevor Taylor — Wed, 17 Aug 2011 19:52:08 GMT

One of the Robotics Team developers, Emil Gustafsson, has turned his own learning experiences with the Concurrency and Coordination Runtime (CCR) into a series of blog posts. His set of Tips and Tricks explains in detail how various aspects of CCR work. As well as additional useful documentation, the posts are sprinkled with code snippets. Visit the CCR Tips and Tricks blog page for more info.

New book about RDS using VPL and LEGO NXT

Trevor Taylor — Mon, 15 Aug 2011 19:47:14 GMT

A new book titled Robot Development Using Microsoft Robotics Developer Studio has been released by CRC Press. (It is available from many booksellers including Amazon). This book provides a step-by-step introduction to using the Visual Programming Language (VPL) with a LEGO NXT robot. It was written by academic staff at the National Taiwan University, where it is used to teach an introductory robotics course. All of the sample code as well as Powerpoint slides are available from the companion website.

Microsoft Surface + Microsoft Robotics Developer Studio = DREAM!

Loke Uei — Wed, 10 Aug 2011 20:16:12 GMT

Watch how the Lowell Team from the University of Massachusetts use Surface and RDS to build a very natural user interface for controlling robots for search and rescue. Read Stewart's blog post here for more detail : http://blogs.msdn.com/b/msr_er/archive/2011/08/08/learn-how-robots-can-help-when-disaster-strikes.aspx

Announcing Kinect Services for RDS 2008 R3

Stathis Papaefstathiou — Wed, 13 Jul 2011 16:00:00 GMT

These are exciting times for Robotics. It is clear that the convergence of technologies and secular trends will make Robotics relevant to our day to day lives. The availability of Kinect and the work that the Robotics community has been doing the last few months is an example of the energy and the potential that exists in this space. You may have already seen in the news a couple of weeks ago that Microsoft released our Kinect for Windows SDK Beta (Kinect SDK) —creating opportunities in exciting ways by making advanced technologies, such as skeleton tracking, available to everyone.

Today, I’m pleased to announce that we are releasing Kinect Services for RDS, available as a free download from our website. Kinect Services for RDS is built on top of the Kinect for Windows SDK. We have wrapped in the core Kinect SDK capabilities so you can use them with the CCR/DSS programming model. However, this is only the beginning. Keep your eyes peeled for updates in the fall. We know that there are many practical issues for applying the Kinect technologies to robotics capabilities that we will be addressing in upcoming RDS releases.

A few years ago, we recognized that the dawn of robotics in the consumer market was coming, and asked ourselves how we could drive the technologies that will make robotics relevant to the consumer market. There is very interesting and exciting work that is taking place in academia, research, community, and among many start-ups. We didn’t want to replicate these efforts, so we decided to focus on leveraging Microsoft’s vast portfolio of technologies, and the brainpower of Microsoft Research to develop capabilities, scenarios, and experiences that are necessary to push Robotics over the last “hump” to become part of our everyday lives.

As you can imagine, this has been a challenging task given the constraints and expectations of consumers. Consumers expect that their products will always work, that the experience will be seamless, and that there is sufficient value to justify a new product category in their homes. At the same time, unlike research robots, the constraints of a consumer device given the price point are limited in terms of sensors and other capabilities (e.g. manipulators).

Democratizing robotics technologies to enable the consumer robotics market to flourish is our passion and number one priority. At the same time, this new strategy has forced us to be less visible with our Robotics Developer Studio work. During the last few years, we have continued to have an active user community and I thank you for your patience and your support. I would like to reinforce that all of the technologies that we include in the RDS releases are the underpinning of our new focus in consumer robotics.

We look forward to working with the developer community to create a common platform – using Kinect – to co-create consumer robotics projects and scenarios.

--Stathis Papaefstathiou, General Manager, Microsoft Robotics

Lego Hexapod powered by Microsoft Robotics Developer Studio

Loke Uei — Wed, 29 Jun 2011 17:53:44 GMT

This is a cool hexapod robot that's built using the Lego Mindstorm kit with logic written using the Microsoft Robotics Developer Studio platform. Gokulnath Murugesan who hails from Chennai India put this together and recorded a video of it in action. Also, in this video, you'll see him control the robot using an Android Phone.

http://blogs.gokulnath.com//2011/06/27/LegoHexapodControlledUsingMicrosoftRoboticsDeveloperStudioMRDS.aspx

(Please visit the site to view this video)

Software Engineering course material for Robotics

Loke Uei — Fri, 18 Mar 2011 19:55:00 GMT

Microsoft together with their partner, Georgia Tech, developed a comprehensive course for teaching the basics of software engineering as it’s applied to robotics. This course utilizes the Microsoft Robotic Developer Studio R3 for software implementation and examples. This lecture series covers the following :

Basic software engineering
Software engineering tools
Introduction to Microsoft Robotics Developer Studio
Development using simulation
Process/System Models
Communication and Synchronization
Standard System Architectures
Hardware interfacing

The course also includes a number of exercises to enable students to consider how design decisions such as data flow, synchronization, and how it impacts the overall performance of the system.

You can download the complete course content here : http://swrobotics.codeplex.com/ or read Georgia Tech' Course Syllabus here : http://www.cc.gatech.edu/~hic/8803-SER-10/Syllabus.html

Feel free to contact us via our Twitter account or on the Microsoft Robotics Forums if you have any questions or feedback.

Using RDS in Cars and for Home Automation

Trevor Taylor — Wed, 13 Oct 2010 18:19:00 GMT

Stacy Harris and John Mullinax were the key people behind the University of Michigan project to use RDS for social networking in cars. (See a previous blog entry). In this short interview with John O'Donnell they talk about how the project came about and Stacy also gives some insight into his experiments with home automation using RDS:

http://channel9.msdn.com/Blogs/jodonnell/Microsoft-Robotics-Developer-Studio-Cars-and-Home-Automation-with-Stacy-Harris-and-John-Mullinax

Microsoft Surface meets Microsoft Robotics Developer Studio at U. Mass Lowell

Trevor Taylor — Wed, 25 Aug 2010 19:28:01 GMT

Researchers at the University of Massachusetts Lowell Robotics Lab have used a Microsoft Surface as a way to command multiple robots to move around in arbitrary paths. You can watch the video to see how smoothly it works. Visit the Robotics Lab web site for more information on work in progress.

Add-ons for the RDS Simulator

Trevor Taylor — Mon, 09 Aug 2010 19:36:00 GMT

The Robotics Developer Studio (RDS) Simulator is a key feature of the package that allows you to get started without buying expensive robots. It is a great tool for use in education. The add-ons outlined below help you to create your own simulation environments and get started on learning about robotics.

Using SimplyCube from VPL

SimplyCube is a development environment for simulation created by SimplySim that is closely integrated with RDS. In their latest blog post, SimplySim shows how to use VPL (Visual Programming Language) to manipulate entities in a simulated environment.

Creative IT Education

The HelloApps web site has a new addition. The Creative IT Education page includes a showcase for the 5-day Creative IT Camp course (held Aug 2 to Aug 6, 2010) and you can see the high school student's various creative solutions for the two given missions.

40 high school students (IT Gifted Youth) attended this camp and they showed wonderful creative solutions using RDS simulation during the camp.

The combination of RDS simulation and SPL are very suitable tools for enhancing creativity and IT skills simultaneously. This showcase could be valuable reference for the Technical Faculty audience. The web page includes all lecture slides on Creative IT program (from 3-hour course to a 5-day course).

Artificial Robotics Editing Environment

The AR package from Artificial Robotics combines the Google SketchUp tool with the RDS Simulator to create a rapid prototyping environment for building simulations. The AR Express package is available for free.

Partner News from Studica, Robosoft

Trevor Taylor — Tue, 29 Jun 2010 18:06:00 GMT

This partner update includes news from the RDS partners Studica and Robosoft.

Studica News

Studica is a new Robotics partner. They are the North American distributor for Fischertechnik and they are very active in the education community.

Recently Studica ran their Second Annual Virtual Robotics Competition at the Ontario Technical Skills Competitions at RIM Park in Waterloo Ontario. There were 9 teams that participated from all over Ontario, varying in experience from just being introduced to VPL two days earlier, to some teams that delved deep into C# for the past several months. One team in particular wrote their own Sumo Player by stripping out all the ‘state’ implementation of the main control code os the base SumoPlayer and implemented their own, procedural-style code. Another team eliminated the floor sensors code and used only the ImageProcessing to control the Boundary avoidance, while another team used extensive ImageProcessing for the Target object tracking.

As a surprise, the Sumo Ring Object was slightly to reflect a Gray arena rather than a Black one, so some teams needed to make modifications of thresholds to their code. Despite some technical glitches, the Round-Robin matches were completed successfully. A couple of photos are included below from the event.

Robosoft News

The robuBOX-Kompai is an open source package (licensed under LGPL) based on Microsoft Robotic Developer Studio R3. It provides support for Robosoft’s Kompaï R&D robot.
See Kompaï in action at http://www.youtube.com/watch?v=GciSisi1cMg

The package contains:

a simulated Kompaï for MRDS simulation engine,
full hardware support (speed control, laser, ultrasonic, infrared, odometry)
path following
mapping, localization and path generation, based on Karto, from SRI
speech interaction
a dashboard to monitor and use these functionalities

The robuBOX-Kompai package can be downloaded from the Kompaï Home Page. The user manual is also available online.