Kinect for Windows Blog

Kinect for Windows: SDK and Runtime version 1.5 Released

Kinect for Windows Team — Mon, 21 May 2012 13:05:00 GMT

I am pleased to announce that today we have released version 1.5 of the Kinect for Windows runtime and SDK. Additionally, Kinect for Windows hardware is now available in Hong Kong, South Korea, Singapore, and Taiwan. Starting next month, Kinect for Windows hardware will be available in 15 additional countries: Austria, Belgium, Brazil, Denmark, Finland, India, the Netherlands, Norway, Portugal, Russia, Saudi Arabia, South Africa, Sweden, Switzerland and the United Arab Emirates. When this wave of expansion is complete, Kinect for Windows will be available in 31 countries around the world. Go to our Kinect for Windows website to find a reseller in your region.

We have added more capabilities to help developers build amazing applications, including:

Kinect Studio, our new tool which allows developers to record and play back Kinect data, dramatically shortening and simplifying the development lifecycle of a Kinect application. Now a developer writing a Kinect for Windows application can record clips of users in the application’s target environment and then replay those clips at a later time for testing and further development.
A set of Human Interface Guidelines (HIG) to guide developers on best practices for the creation of Natural User Interfaces using Kinect.
The Face Tracking SDK, which provides a real-time 3D mesh of facial features—tracking the head position, location of eyebrows, shape of the mouth, etc.
Significant sample code additions and improvements. There are many new samples in both C++ and C#, plus a “Basics” series of samples with language coverage in C++, C#, and Visual Basic.
SDK documentati on improvements, including new resources as well as migration of documentation to MSDN for easier discoverability and real-time updates.

We have continued to expand and improve our skeletal tracking capabilities in this release:

Seated Skeletal Tracking is now available. This tracks a 10-joint head/shoulders/arms skeleton, ignoring the leg and hip joints. It is not restricted to seated positions; it also tracks head/shoulders/arms when a person is standing. This makes it possible to create applications that are optimized for seated scenarios (such as office work with productivity software or interacting with 3D data) or standing scenarios in which the lower body isn’t visible to the sensor (such as interacting with a kiosk or when navigating through MRI data in an operating room).
Skeletal Tracking is supported in Near Mode, including both Default and Seated tracking modes. This allows businesses and developers to create applications that track skeletal movement at closer proximity, like when the end user is sitting at a desk or needs to stand close to an interactive display.

We have made performance and data quality enhancements, which improve the experience of all Kinect for Windows applications using the RGB camera or needing RGB and depth data to be mapped together (“green screen” applications are a common example):

Performance for the mapping of a depth frame to a color frame has been significantly improved, with an average speed increase of 5x.
Depth and color frames will now be kept in sync with each other. The Kinect for Windows runtime continuously monitors the depth and color streams and corrects any drift.
RGB Image quality has been improved in the RGB 640x480 @30fps and YUV 640x480 @15fps video modes. The image quality is now sharper and more color-accurate in high and low lighting conditions.

New capabilities to enable avatar animation scenarios, which makes it easier for developers to build applications that control a 3D avatar, such as Kinect Sports.

Kinect for Windows runtime provides Joint Orientation information for the skeletons tracked by the Skeletal Tracking pipeline.
The Joint Orientation is provided in two forms: A Hierarchical Rotation based on a bone relationship defined on the Skeletal Tracking joint structure, and an Absolute Orientation in Kinect camera coordinates.

Finally, as I mentioned in my Sneak Peek Blog post, we released four new languages for speech recognition – French, Spanish, Italian, and Japanese. In addition, we released new language packs which enable speech recognition for the way a language is spoken in different regions: English/Great Britain, English/Ireland, English/Australia, English/New Zealand, English/Canada, French/France, French/Canada, Italian/Italy, Japanese/Japan, Spanish/Spain, and Spanish/Mexico.

As we have worked with customers large and small over the past months, we’ve seen the value in having a fully integrated approach: the Kinect software and hardware are designed together; audio, video, and depth are all fully supported and integrated; our sensor, drivers, and software work together to provide world class echo cancellation; our approach to human tracking, which is designed in conjunction with the Kinect sensor, works across a broad range of people of all shapes, sizes, clothes, and hairstyles, etc. And because we design the hardware and software together, we are able to make changes that open up exciting new areas for innovation, like Near Mode.

Furthermore, because Kinect for Windows is from Microsoft, our support, distribution, and partner network are all at a global scale. For example, the Kinect for Windows hardware and software are tested together and supported as a unit in every country we are in (31 countries by June!), and we will continue to add countries over time. Microsoft’s developer tools are world class, and our SDK is built to fully integrate with Visual Studio. Especially important for our global business customers is Microsoft’s ability to connect them to partners and experts who can help them use Kinect for Windows to re-imagine their brands, their products, and their processes.

It is exciting for us to have built and shipped such a significantly enhanced version of the Kinect for Windows SDK less than 16 weeks after launch. But we are even more excited about our plans for the future – both in country expansion for the sensor, and in enhanced capabilities of our runtime and SDK. We believe the best is yet to come, and we can’t wait to see what developers will build with this!

Craig Eisler
General Manager, Kinect for Windows

Kinect for Windows Wins Innovation of the Year

Kinect for Windows Team — Wed, 09 May 2012 02:46:00 GMT

Last week, winners of GeekWire’s fourth annual Seattle 2.0 Startup Awards were announced. Seattle has a vibrant startup community, and this is a very popular event within that community. I attended the awards ceremony, and it was amazing to see how much energy and excitement was in the room – and the EMP (Experience Music Project) where the event was held was packed.

There were entrepreneurs of all types and levels of experience (I even ran into Ray Ozzie!), venture capitalists, CEOs, and so much IQ and passion in one place it was a rush – the same rush I feel every Monday when I spend time at the Kinect Accelerator in Microsoft’s Westlake office.

We were thrilled when Kinect for Windows was named “Innovation of the Year” – not because we won (which is great!), but because it was the popular vote of the startup community. This is the same community that is continuing to deliver so many amazingly different ideas and products that have Kinect for Windows at the center. In so many ways, Kinect for Windows is “Innovation of the Year” because of the innovators who are using it: Thank you.

Learn more about all of the winners of the Seattle 2.0 Startup Awards. And thanks to all of you who voted for Kinect for Windows!

Craig Eisler
General Manager, Kinect for Windows

Help Kinect for Windows Become Innovation of the Year

Kinect for Windows Team — Fri, 20 Apr 2012 17:11:00 GMT

Earlier this month, Geekwire announced the nominees for the 2012 Seattle 2.0 Startup Awards. We're honored Kinect for Windows was selected in the “Innovation of the Year” category, which looks at technologies, which are setting a course for ”where the world is going and the way of the future.”

Other nominees in this category are Symform, ExtraHop, LaserMotive, and Vioguard.

If you’re developing with the Kinect for Windows SDK and sensor, or simply a fan of the technology, cast your vote and help us become Seattle’s startup innovation of the year.

Voting ends Monday, April 23^rd. Winners will be announced at the Seattle 2.0 Startup Awards bash on May 3 at the Experience Music Project (EMP) in Seattle.

Vote today!

Kinect for Windows Team

Kinect for Windows Technologies: Boxing Robots to Limitless Possibilities

Kinect for Windows Team — Tue, 17 Apr 2012 22:40:00 GMT

Most developers, including myself, are natural tinkerers. We hear of a new technology and want to try it out, exploring what it can do, dream up interesting uses, and pushing the limits of what’s possible. Most recently, the Channel 9 team incorporated Kinect for Windows into two projects: BoxingBots, and Project Detroit.

The life-sized BoxingBots made their debut in early March at SXSW in Austin, Texas. Each robot is equipped with an on-board computer, which receives commands from two Kinect for Windows sensors and computers. The robots are controlled by two individuals whose movements – punching, rotating, stepping forward and backwards – are interpreted by and relayed back to the robots, who in turn, slug it out, until one is struck and its pneumatic-controlled head springs up.

The use of Kinect for Windows for telepresence applications, like controlling a robot or other mechanical device, opens up a number of interesting possibilities. Imagine a police officer using gestures and word commands to remotely control a robot, exploring a building that may contain explosives. In the same vein, Kinect telepresence applications using robots could be used in the manufacturing, medical, and transportation industries.

Project Detroit asked the question, what do you get when you combine the world’s most innovative technology with a classic American car? The answer is a 2012 Ford Mustang with a 1967 fastback replica body, and everything from Windows Phone integration to built-in WiFI, Viper SmartStart security system, cloud services, augmented reality, Ford SYNC, Xbox-enabled entertainment system, Windows 8 Slate, and Kinect for Windows cameras built into the tail and headlights.

One of the key features we built for Project Detroit was the ability to read Kinect data including a video stream, depth data, skeletal joint data, and audio streams over the network using sockets (available here as an open source project). These capabilites could make it possible to receive an alert on your phone when someone gets too close to your car. You could then switch to a live video/audio stream, via a network from the Kinect, to see what they were doing. Using your phone, you could talk to them, asking politely that they “look, but not touch.”

While these technologies may not show up in production cars in the coming months (or years), Kinect for Windows technologies are suited for use in cars for seeing objects such as pedestrians and cyclists behind and in front of vehicles, making it easier to ease into tight parking spots, and enabling built-in electronic devices with the wave of a hand or voice commands.

It’s an exciting time to not only be a developer, but a business, organization or consumer who will have the opportunity to benefit from the evolving uses and limitless possibilities of the Kinect for Windows natural user interface.

Dan Fernandez
Senior Director, Microsoft Channel 9

Kinect for Windows Helps Girls Everywhere Dress Like Barbie

Kinect for Windows Team — Thu, 12 Apr 2012 16:18:00 GMT

I grew up in the UK and my female cousins all had Barbie. In fact Barbies – they had lots of Barbie dolls and ton of accessories that they were obsessed with. I was more of a BMX kind of kid and thought my days of Barbie education were long behind me, but with a young daughter I’m beginning to realize that I have plenty more Barbie ahead of me, littered around the house like landmines. This time around though, I’m genuinely interested thanks to a Kinect-enabled application.

This week, Barbie lovers in Sydney, Australia, are being given the chance to do more than fanaticize how they’d look in their favorite Barbie outfit. Thanks to Mattel, Gun Communications, Adapptor, and Kinect for Windows, Barbie The Dream Closet is here.

The application invites users to take a walk down memory lane and select from 50 years of Barbie fashions. Standing in front of Barbie’s life-sized augmented reality “mirror,” fans can choose from several outfits in her digital wardrobe—virtually trying them on for size.

The solution, built with the Kinect for Windows SDK and using the Kinect for Windows sensor, tracks users’ movements and gestures enabling them to easily browse through the closet and select outfits that strike their fancy. Once an outfit is selected, the Kinect for Windows skeletal tracking determines the position and orientation of the user. The application then rescales Barbie’s clothes, rendering them over the user in real time for a custom fit.

One of the most interesting aspects of this solution is the technology’s ability to scale - with menus, navigation controls and clothing all dynamically adapting so that everyone from a little girl to a grown woman (and cough, yes, even a committed father) can enjoy the experience. To facilitate these advancements, each outfit was photographed on a Barbie doll, cut into multiple parts, and then built individually via the application.

Of course, the experience wouldn’t be complete without the ability to memorialize it. A photo is taken and, with approval/consent from those photographed, is uploaded and displayed in a gallery on the Barbie Australian Facebook page. (Grandparents can join in the fun from afar!)

I spoke with Sarah Sproule, Director, Gun Communications about the genesis of the idea who told me, “We started working on Barbie The Dream Closet six months ago, working with our development partner Adapptor. Everyone has been impressed by the flexibility, and innovation Microsoft has poured into Kinect for Windows. Kinect technology has provided Barbie with a rich and exciting initiative that's proving to delight fans of all ages. We're thrilled with the result, as is Mattel - our client."

Barbie’s Dream Closet, was opened to the public at the Westfield Parramatta in Sydney today and will be there through April 15. Its first day, it drew enthusiastic crowds, with around 100 people experiencing Barbie The Dream Closet. It's expected to draw even larger crowds over the holidays. It’s set to be in Melbourne and Brisbane later this year.

Meantime, the Kinect for Windows team is just as excited about it as my daughter:

“The first time I saw Barbie’s Dream Closet in action, I knew it would strike a chord,” notes Kinect for Windows Communications Manager, Heather Mitchell. “It’s such a playful, creative use of the technology. I remember fanaticizing about wearing Barbie’s clothes when I was a little girl. Disco Ken was a huge hit in my household back then…Who didn’t want to match his dance moves with their own life-sized Barbie disco dress? I think tens of thousands of grown girls have been waiting for this experience for years…Feels like a natural.”

That’s the beauty of Kinect – it enables amazingly natural interactions with technology and hundreds of companies are out there building amazing things; we can’t wait to see what they continue to invent.

Steve Clayton
Editor, Next at Microsoft

Microsoft’s Kinect Accelerator Begins Today

Kinect for Windows Team — Mon, 02 Apr 2012 15:04:00 GMT

I am pleased to announce that the finalists for our Kinect Accelerator have arrived in ever-sunny Seattle and today are launching into a three-month program to build new products and business using Kinect. I can’t wait to see what they come up with – using Kinect, these teams have the ability to reimagine the way products are used, and perhaps even revolutionize entire industries along the way.

Kinect Accelerator is powered by TechStars, in close collaboration with the Microsoft BizSpark program; my team and I have been working closely with the BizSpark team and others in the Interactive Entertainment Business to help develop and bring this program to life. The response to the Kinect Accelerator has been phenomenal and we expect to see remarkable innovation coming out of the program.

We were hoping to receive 100 to 150 applications, with a goal of selecting the best ten. But the worldwide entrepreneurial community completely surprised us by submitting almost five hundred applications with concepts spanning nearly 20 different industries, including healthcare, education, retail, entertainment, and more.

There were so many clever and innovative ideas and so many great teams it was super challenging to narrow things down – we spent many, many hours in a rigorous and highly energetic review process. We finally landed on 11 finalists from five countries, chosen based on their experience, qualifications, and the potential benefit that could result from their Kinect Accelerator. The finalists are:

Freak'n Genius – Seattle, WA
GestSure Technologies – Toronto, Canada
IKKOS – Seattle, WA
Kimetric – Buenos Aires, Argentina
Jintronix Inc. – Montreal, Canada
Manctl – Lyon, France
NConnex – Hadley, MA
Styku - Los Angeles, CA
übi interactive – Munich, Germany
VOXON – New York, NY
Zebcare – Boston, MA

Each team will be mentored by entrepreneurs and venture capitalists as well as leaders from Kinect for Windows, Xbox, Microsoft Studios, Microsoft Research and other Microsoft organizations. The teams will spend the first several weeks ideating and refining their business concepts with input and advice from their mentors, followed by several weeks of design and development. They will present their results at an event at the end of June.

We were so amazed by the quality, caliber, and uniqueness of the applications and teams that we decided to reward the top 100 applicants that didn’t make it into the program with a complimentary Kinect for Windows sensor. I believe we are going to see great things from many of the folks that applied to the program and we wish them all the best.

We will share more information about the Kinect Accelerator teams and their applications on this blog in coming months. And for more information on the Kinect Accelerator program in general, go to KinectAccelerator.com.

Craig Eisler
General Manager, Kinect for Windows

And One More Thing About 1.5…

Kinect for Windows Team — Thu, 29 Mar 2012 20:29:00 GMT

Since our announcement of Kinect for Windows version 1.5 in “What’s Ahead: A Sneak Peek” there have been a few questions that have come up that I wanted to answer.

There have been some folks who have thought that 1.5 included new hardware. Version 1.5 is our new software release that is coming out in the same timeframe that we launch the current Kinect for Windows hardware in 19 additional countries. We will upgrade our software at a faster rate than we refresh our hardware.

We have built version 1.5 of our software with 1.0 compatibility at top of mind. Applications built using 1.0 will work on the same machine with an application built using 1.5 – this is something that we plan to do always, insuring that solutions built using older runtimes can always run side by side with solutions using new runtimes. Furthermore, we have maintained API compatibility for developers – applications that are currently being built using the 1.0 SDK can be recompiled using the 1.5 SDK without any changes required. No one has to wait for 1.5 to get a Kinect for Windows sensor or to start coding using the current SDK!

I love the enthusiasm for the 1.5 SDK and runtime, the new speech languages, and for the new countries we’re launching in – we can’t wait to deliver it to you.

Craig Eisler
General Manager, Kinect for Windows

What’s Ahead: A Sneak Peek

Kinect for Windows Team — Mon, 26 Mar 2012 16:38:00 GMT

The momentum continues for Kinect for Windows. I am pleased to announce that we will be launching Kinect for Windows in nineteen more countries in the coming months. We will have availability in Hong Kong, South Korea, and Taiwan in late May. In June, Kinect for Windows will be available in Austria, Belgium, Brazil, Denmark, Finland, India, the Netherlands, Norway, Portugal, Russia, Saudi Arabia, Singapore, South Africa, Sweden, Switzerland and the United Arab Emirates.

We are also hard at work on our 1.5 release, which will be available at the end of May. Among the most exciting new capabilities is Kinect Studio, an application that will allow developers to record, playback and debug clips of users engaging with their applications. Also coming is what we call “seated” or “10-joint” skeletal tracking, which provides the capability to track the head, neck and arms of either a seated or standing user. What is extra exciting to me about this functionality is that it will work in both default and near mode!

Also included in our 1.5 release will be four new languages for speech recognition – French, Spanish, Italian, and Japanese. In addition, we will be releasing new language packs which enable speech recognition for the way a language is spoken in different regions: English/Great Britain, English/Ireland, English/Australia, English/New Zealand, English/Canada, French/France, French/Canada, Italian/Italy, Japanese/Japan, Spanish/Spain and Spanish/Mexico.

In a future blog post, I’ll discuss the features and capabilities we are releasing in more detail. We are excited by the enthusiasm for Kinect for Windows, and will continue to work on bringing Kinect for Windows to more countries, supporting more languages with our speech engine, and continuing to evolve our human tracking capabilities.

Craig Eisler
General Manager, Kinect for Windows

Kinect for Windows Solution Leads to the Perfect Fitting Jeans

Kinect for Windows Team — Thu, 15 Mar 2012 15:36:00 GMT

The writer Mark Twain once said “We are alike, on the inside.” On the outside, however, few people are the same. While two people might be the same height and wear the same size, the way their clothing fits their bodies can vary dramatically. As a result, up to 40% of clothing purchased both online and in person is returned because of poor fit.

Finding the perfect fit so clothing conforms to a person’s unique body shape is at the heart of the Bodymetrics Pod. Developed by Bodymetrics, a London-based pioneer in 3D body-mapping, the Bodymetrics Pod was introduced to American shoppers for the first time today during Women’s Denim Days at Bloomingdale’s in Century City, Los Angeles. This is the first time Kinect for Windows has been used commercially in the United States for body mapping in a retail clothing environment.

Bloomingdale’s, a leader in retail innovation, has one of the largest offerings in premium denim from fashion-forward brands like J Brand, 7 for all mankind, Citizens and Humanity, AG, and Paige. The Bodymetrics services allows customers to get their body mapped and find jeans that fit and flatter their unique shape from the hundreds of different jeans styles that Bloomingdale’s stocks.

During Bloomingdale’s Denim Days, March 15 – 18, customers will be able to get their body mapped, and also become a Bodymetrics member. This free service enables customers to access an online account and order jeans based on their body shape.

“We’re very excited about bringing Bodymetrics to US shoppers,” explains Suran Goonatilake, CEO of Bodymetrics. “Once we 3D map a customer’s body, we classify their shape into three categories - emerald, sapphire and ruby. A Bodymetrics Stylist will then find jeans that exactly match the body shape of the customer from jean styles that Bloomingdale’s stocks.”

The process starts with a customer creating a Bodymetrics account. They are then directed to the Bodymetrics Pod, a secure, private space, where their body is scanned by 8 Kinect for Windows sensors arranged in a circle. Bodymetrics’ proprietary software produces a 3D map of the customer’s body, and then calculates the shape of the person, taking hundreds of measurements and contours into account. The body-mapping process takes less than 5 seconds.

Helping women shop for best-fitting jeans in department stores is just the start of what Bodymetrics envisions for their body-mapping technologies. The company is working on a solution that can be used at home. Individuals will be able to scan their body, and then go online to select, virtually try on, and purchase clothing that match their body shape.

Goonatilake explains, “Body-mapping is in its infancy. We’re just starting to explore what’s possible in retail stores and at home. Stores are increasingly looking to provide experiences that entice shoppers into their stores, and then allow a seamless journey from stores to online. And we all want shopping experiences that are personalized to us – our size, shape and style.”

Even though people may not be identical on the outside, we desire clothing that fits well and complements our body shapes. The Kinect for Windows-enabled Bodymetrics Pod offers a retail-ready solution that makes the perfect fit beautifully simple.

Kinect for Windows Team

Kinect for Windows Academic Pricing Now Available in the US

Kinect for Windows Team — Thu, 08 Mar 2012 23:55:00 GMT

Students, teachers, researchers, and other educators have been quick to embrace Kinect’s natural user interface (NUI), which makes it possible to interact with computers using movement, speech, and gestures. In fact, some of the earliest Kinect for Windows applications to emerge were projects done by students, including several at last year’s Imagine Cup.

One project, from an Imagine Cup team in Italy, created an application for people with severe disabilities that enables them to communicate, learn, and play games on computers using a Kinect sensor instead of a traditional mouse or keyboard. Another innovative Imagine Cup project, done by university students in Russia, used the Kinect natural user interface to fold, rotate, and examine online origami models.

To encourage students, educators, and academic researchers to continue innovating with Kinect for Windows, special academic pricing on Kinect for Windows sensors is now available in the United States. The academic price is $149.99 through Microsoft Stores.

If you are an educator or faculty with an accredited school, such as a university, community college, vocational school, or K-12, you can purchase a Kinect for Windows sensor at this price.

Find out if you qualify, and then purchase online or visit a Microsoft store in your area.

Kinect for Windows team

Kinect for Windows Applications Shown at Microsoft TechForum

Kinect for Windows Team — Wed, 29 Feb 2012 00:59:00 GMT

This year’s Microsoft TechForum provided an opportunity for Craig Mundie, Microsoft Chief Research and Strategy Officer, to discuss the company’s vision for the future of technology as well as showcase two early examples of third-party Kinect for Windows applications in action.

Mundie was joined by Don Mattrick, President of the Microsoft Interactive Entertainment Business, and his Chief of Staff, Aaron Greenberg, who demonstrated both of the third-party Kinect for Windows applications, including the Pathfinder Kinect Experience. This application enables users to stand in front of a large monitor, and use movement, voice, and gestures to walk around the 2013 Nissan Pathfinder Concept, examining the exterior, bending down and inspecting the wheels, viewing the front and back, and then stepping inside to experience the upholstery, legroom, dashboard, and other details.

Nissan worked with IdentityMine and Critical Mass to create the Kinect-enabled virtual experience, which was initially shown at the Chicago Auto Show in early February. The application is continuing to be refined, taking advantage of the Kinect natural user interface to enable manufacturers to showcase their vehicles in virtual showrooms.

“Using motion, speech, and gestures, people will be able to get computers to do more for them,” explain Greenberg. “You can imagine this Pathfinder solution being applied in different ways in the future - at trade shows, online, or even at dealerships - where someone might be able to test drive a physical car, while also being able to visualize and experience different configurations of the car through its virtual twin, accessorizing it, changing the upholstery, et cetera.”

Also demonstrated at TechForum was a new kind of shopping cart experience, which was developed by mobile application studio Chaotic Moon. This application mounts a Kinect for Windows sensor on a shopping cart, enabling the cart to follow a shopper - stopping, turning, and moving where and when the shopper does.

Chaotic Moon has tested their solution at Whole Foods in Austin, Texas, but the application is an early experiment and no plans are in place for this application to be introduced in stores anytime soon. Conceivably, Kinect-enabled carts at grocery stores, shopping malls, or airports could make it easier for people to navigate and perform tasks hands free. “Imagine how an elderly shopper or a parent with a stroller might be assisted by something like this,” notes Greenberg.

“The Kinect natural user interface has the potential to revolutionize products and processes in the home, at work, and in public places, like retail stores,” continues Greenberg. “It’s exciting to see what is starting to emerge.”

(Please visit the site to view this video)

Kinect for Windows team

Kinect for Windows is now Available!

Kinect for Windows Team — Wed, 01 Feb 2012 04:35:00 GMT

On January 9th, Steve Ballmer announced at CES that we would be shipping Kinect for Windows on February 1st. I am very pleased to report that today version 1.0 of our SDK and runtime were made available for download, and distribution partners in our twelve launch countries are starting to ship Kinect for Windows hardware, enabling companies to start to deploy their solutions. The suggested retail price is $249, and later this year, we will offer special academic pricing of $149 for Qualified Educational Users.

In the three months since we released Beta 2, we have made many improvements to our SDK and runtime, including:

Support for up to four Kinect sensors plugged into the same computer
Significantly improved skeletal tracking, including the ability for developers to control which user is being tracked by the sensor
Near Mode for the new Kinect for Windows hardware, which enables the depth camera to see objects as close as 40 centimeters in front of the device
Many API updates and enhancements in the managed and unmanaged runtimes
The latest Microsoft Speech components (V11) are now included as part of the SDK and runtime installer
Improved “far-talk” acoustic model that increases speech recognition accuracy
New and updated samples, such as Kinect Explorer, which enables developers to explore the full capabilities of the sensor and SDK, including audio beam and sound source angles, color modes, depth modes, skeletal tracking, and motor controls
A commercial-ready installer which can be included in an application’s set-up program, making it easy to install the Kinect for Windows runtime and driver components for end-user deployments.
Robustness improvements including driver stability, runtime fixes, and audio fixes

More details can be found here.

As I mentioned in an earlier blog post, without many years of intense R&D efforts, including research investments of hundreds of millions of dollars, and deep partnership between our research teams, software teams, hardware teams, manufacturing teams, and games studios, Kinect simply wouldn’t exist. Shipping Kinect for Windows was another cross-Microsoft effort: not only did the hardware and software teams work closely together to create an integrated solution, but our support, manufacturing, supply chain, reverse logistics, and account teams have all been working hard to prepare for today’s launch. As well, our research, speech, and Xbox NUI teams have contributed to making Kinect for Windows a better product. Microsoft’s ability to make these kinds of deep investments makes Kinect for Windows a product that companies can deploy with confidence, knowing you have our support and our ongoing commitment to make Kinect for Windows the best it can be.

Looking towards the future, we are planning on releasing updates to our SDK and runtime 2-3 times per year – in fact, the team is already hard at work on the next release. We are continuing to invest in programs like our Testing and Adoption Program and the Kinect Accelerator, and will work to create new programs in the future to help support our developer and partner ecosystem. We will also continue to listen to our developer community and business customers for the kinds of features and capabilities they need, as they re-imagine the future of computing using the power of Kinect.

Craig Eisler
General Manager, Kinect for Windows

Near Mode: What it is (and isn’t)

Kinect for Windows Team — Sat, 21 Jan 2012 01:59:00 GMT

There has been a lot of speculation on what near mode is since we announced it. As I mentioned in the original post, the Kinect for Windows device has new firmware which enables the depth camera to see objects as close as 50 centimeters in front of the device without losing accuracy or precision, with graceful degradation down to 40 centimeters.

The lenses on the Kinect for Windows sensor are the same as the Kinect for Xbox 360 sensor, so near mode does not change the field of view as some people have been speculating. As some have observed, the Kinect for Xbox 360 sensor was already technically capable of seeing down 50 centimeters – but with the caveat “as long as the light is right”.

That caveat turned out to be a pretty big caveat. The Kinect for Windows team spent many months developing a way to overcome this so the sensor would properly detect close up objects in more general lighting conditions. This resulted not only in the need for new firmware, but changes to the way the devices are tested on the manufacturing line. In addition to allowing the sensor to see objects as close as 40 centimeters, these changes make the sensor less sensitive to more distant objects: when the sensor is in near mode, it has full accuracy and precision for objects 2 meters away, with graceful degradation out to 3 meters. Here is a handy chart one of our engineers made that shows the types of depth values returned by the runtime:

In Beta 2, for an object 800-4000 millimeters from the sensor the runtime would return the depth value, and the runtime returned a 0 regardless if the detected depth was unknown, too near or too far. Our version 1.0 runtime will return depth values if an object is in the above cyan zone. If the object is in the purple, brown or white zones, the runtime will return a distinct value indicating the appropriate zone.

Additionally, in version 1.0 of the runtime, near mode will have some skeletal support, although not full 20-joint skeletal tracking (ST). The below table outlines the differences between the default mode and near mode:

We believe that near mode, with its operational envelope of 40 centimeters to 3 meters, will enable many new classes of applications. While full 20-joint ST will not be supported in near mode with version 1.0 of the runtime, we will be working hard to support ST in near mode in the future!

Craig Eisler
General Manager, Kinect for Windows

The Power of Enthusiasm

Kinect for Windows Team — Fri, 13 Jan 2012 00:43:00 GMT

When we launched Kinect for Xbox 360 on November 4^th, 2010, something amazing happened: talented Open Source hackers and enthusiasts around the world took the Kinect and let their imaginations run wild. We didn’t know what we didn’t know about Kinect on Windows when we shipped Kinect for Xbox 360, and these early visionaries showed the world what was possible. What we saw was so compelling that we created the Kinect for Windows commercial program.

Our commercial program is designed to allow our partners— companies like Toyota, Mattel, American Express, Telefonica, and United Health Group—to deploy solutions to their customers and employees. It is also designed to allow early adopters and newcomers alike to take their ideas and release them to the world on Windows, with hardware that’s supported by Microsoft. At the same time, we wanted to let our early adopters keep working on the hardware they’d previously purchased. That is why our SDK continues to support the Kinect for Xbox 360 as a development device.

As I reflect back on the past eleven months since Microsoft announced we were bringing Kinect to Windows, one thing is clear: The efforts of these talented Open Source hackers and enthusiasts helped inspire us to develop Kinect for Windows faster. And their continued ambition and drive will help the world realize the benefits of Kinect for Windows even faster still. From all of us on the Kinect for Windows team: thank you.

Craig Eisler
General Manager, Kinect for Windows

Starting February 1, 2012: Use the Power of Kinect for Windows to Change the World

Kinect for Windows Team — Tue, 10 Jan 2012 00:57:00 GMT

We’re thrilled at the sales we’ve seen for Kinect - 18 million sold in the past year – and we were honored to receive a Guinness World Record for the fastest selling consumer electronics device ever. As consumers, we may take devices like Kinect for granted, but in fact electronic devices are the fruit of a great deal of behind-the-scenes ingenuity and experimentation. Kinect is a shining example of this. Instead of mimicking the handheld motion-sensing controllers already on the market, Microsoft shattered the existing controller paradigm by inventing a new natural user interface system that enables advanced human tracking, gesture recognition, voice control and more. Our answer to the “wand” controller was no controller at all, or as we say, “YOU are the controller.”

Getting there wasn’t easy. Without many years of intense R&D efforts, including research investments of hundreds of millions of dollars, and the deep partnership between our research teams, software teams, hardware teams, manufacturing teams, and games studios, Kinect simply wouldn’t exist. And as amazing a piece of hardware as Kinect is, it is much more than that. At the heart of the Kinect experience lies sophisticated software that meaningfully deciphers the images and gestures captured by the 3D sensor as well as the voice commands captured by the microphone array from someone much further away than someone using a headset or a phone. More importantly, Kinect software can understand what each user means by a particular gesture or command across a wide range of possible shapes, sizes, and actions of real people.

The incredible amount of innovation on Kinect for Xbox 360 this past year shows the potential for Kinect as a platform for developers and businesses to build new and innovative offerings. Along with many others, we have only begun to explore the potential of this amazing technology. This proliferation of creative and imaginative new ideas for Kinect, which we call the Kinect Effect, will expand even further with our commercial release of Kinect for Windows.

Today, we are announcing that the new Kinect for Windows hardware and accompanying software will be available on February 1^st, 2012 in 12 countries (United States, Australia, Canada, France, Germany, Ireland, Italy, Japan, Mexico, New Zealand, Spain, United Kingdom), at a suggested retail price of US $249. Kinect for Windows hardware will be available, in limited quantities at first, through a variety of resellers and distributors. The price includes a one-year warranty, access to ongoing software updates for both speech and human tracking, and our continued investment in Kinect for Windows-based software advancements. Later this year, we will offer special academic pricing (planned at US $149) for Qualified Educational Users.

We love the innovation we have seen built using Kinect for Xbox 360 – this has been a source of inspiration and delight for us and compelled us to create a team dedicated to serving this opportunity. We are proud to bring technology priced in the tens of thousands of dollars just a few years ago to the mainstream at extremely low consumer prices. And although Kinect for Windows is still value-priced for the technology, some will ask us why it isn’t the same price as Kinect for Xbox.

The ability to sell Kinect for Xbox 360 at its current price point is in large part subsidized by consumers buying a number of Kinect games, subscribing to Xbox LIVE, and making other transactions associated with the Xbox 360 ecosystem. In addition, the Kinect for Xbox 360 was built for and tested with the Xbox 360 console only, which is why it is not licensed for general commercial use, supported or under warranty when used on any other platform.

With Kinect for Windows, we are investing in creating a platform that is optimized for scenarios beyond the living room, and delivering new software features on an ongoing basis, starting with “near mode” (see my earlier blog post for more about this). In addition to support for Windows 7 and the Windows 8 developer preview (desktop apps only), Kinect for Windows will also support gesture and voice on Windows Embedded-based devices and will enhance how data is captured and accessed within intelligent systems across manufacturing, retail and many more industries. We are building the Kinect for Windows platform in a way that will allow other companies to integrate Kinect into their offerings and we have invested in an approach that allows them to develop in ways that are dependable and scalable.

We have chosen a hardware-only business model for Kinect for Windows, which means that we will not be charging for the SDK or the runtime; these will be available free to developers and end-users respectively. As an independent developer, IT manager, systems integrator, or ISV, you can innovate with confidence knowing that you will not pay license fees for the Kinect for Windows software or the ongoing software updates, and the Kinect for Windows hardware you and your customers use is supported by Microsoft.

Although we encourage all developers to understand and take advantage of the additional features and updates available with the new Kinect for Windows hardware and accompanying software, those developers using our SDK and the Kinect for Xbox 360 hardware may continue to use these in their development activities if they wish. However, non-commercial deployments using Kinect for Xbox 360 that were allowed using the beta SDK are not permitted with the newly released software. Non-commercial deployments using the new runtime and SDK will require the fully tested and supported Kinect for Windows hardware and software platform, just as commercial deployments do. Existing non-commercial deployments using our beta SDK may continue using the beta and the Kinect for Xbox 360 hardware; to accommodate this, we are extending the beta license for three more years, to June 16, 2016.

We expect that as Kinect for Windows hardware becomes readily available, developers will shift their development efforts to Kinect for Windows hardware in conjunction with the latest SDK and runtime. The combination of Kinect for Windows hardware and software creates a superior development platform for Windows and will yield a higher quality, better performing experience for end users.

We are excited for the new possibilities that Kinect will enable on the Windows platform, and to see how businesses and developers reimagine their processes and their products, and the many different ways each Kinect could enrich lives and make using technology more natural for everyone.

Craig Eisler
General Manager, Kinect for Windows

FIRST Teams to Use Kinect for Windows to Control Robots

Kinect for Windows Team — Sat, 07 Jan 2012 19:51:00 GMT

Getting technology to do what you want can be challenging. Imagine building a remote-controlled robot in 6 weeks, from pre-defined parts, which can perform various tasks in a competitive environment. That’s the challenge presented to 2,500 teams of students who will be participating in the FIRST (For Inspiration and Recognition of Science and Technology) Robotics Competition.

The worldwide competition, open to students in grades 9-12, kicks off this morning with a NASA-televised event, including pre-recorded opening remarks from Presidents Clinton and G.W. Bush, Dean Kamen, founder of FIRST and inventor of the Segway, and Alex Kipman, General Manager, Hardware Incubation, Xbox.

Last year, several FIRST teams experimented with the Kinect natural user interface capabilities to control their robots. The difference this year is the Kinect hardware and software will be included in the parts kits teams receive to build their robots. Teams will be able to control their robots not only with joy sticks, but gestures and possibly spoken commands.

The first part of the competition is the autonomous period, in which robot can only be controlled by sensor input and commands. This is when the depth and speech capabilities of Kinect will prove extremely useful.

To help teams understand how to incorporate Kinect technologies into the design of their robot controls for the 2012 competition, workshops are being held around the country. Students will be using C# or C++ to program the drive stations of their robots to recognize and respond to gestures and poses.

In addition, Microsoft Stores across the country are teaming up with FIRST robotics teams to provide Kinect tools, technical support, and assistance.

While winning teams get bragging rights, all participants gain real-world experience by working with professional engineers to build their team’s robot, using sophisticated hardware and software, such as the Kinect for Windows SDK. Team members also gain design, programming, project management, and strategic thinking experience. Last but not least, over $15 million of college scholarships will be awarded throughout the competition.

“The ability to utilize Kinect technologies and capabilities to transform the way people interact with computers already has sparked the imagination of thousands of developers, students, and researchers from around the world,” notes FIRST founder Dean Kamen. “We look forward to seeing how FIRST students utilize Kinect in the design and manipulation of their robots, and are grateful to Microsoft for participating in the competition as well as offering their support and donating thousands of Kinect sensors.”

This morning’s kick-off of the 2012 FIRST Robotics Competition was a highly anticipated day. Approximately 2,500 teams worldwide were given a kit of 600-700 discrete parts including a Kinect sensor and the Kinect for Windows software development kit (SDK), along with the details and rules for this year’s game, Rebound Rumble. Learn how Kinect for Windows will play a role in this year’s game by watching the game animation.

(Please visit the site to view this video)

Kinect for Windows team

Kinect Accelerator Program Seeking Innovators

Kinect for Windows Team — Wed, 28 Dec 2011 22:34:00 GMT

In March, ten startups will converge on Seattle to start developing commercial and gaming applications that utilize Kinect's innovative natural user interface (NUI). As part of the Microsoft Kinect Accelerator program, they will have three months and a wealth of resources—including access to Microsoft and industry mentors—to develop, and then present their applications to angel investors, venture capitalists, Microsoft executives, media, and influential industry leaders.

Since launching in late November, the Kinect Accelerator has received hundreds of applications from over forty countries, proposing transformative, creative innovations for healthcare, fitness, retail, training/simulation, automotive, scientific research, manufacturing, and much more.

Applications are still being accepted, and the Kinect Accelerator team encourages you to apply. Learn more about the application process.

The Kinect Accelerator program is powered by TechStars, one of the most respected technology accelerator programs in the world. Microsoft is working with TechStars to leverage the absolute best startup accelerator methodologies, mentors, and visibility. If you are considering building a business based on the capabilities of Kinect, this is a great opportunity for you.

Dave Drach, Managing Director, Microsoft Emerging Business Team, explains that the Kinect Accelerator program is looking for creative startups that have a passion for driving the next generation of computing. “Starting in the spring of 2012, they will have three months to bring their ideas to life. What will emerge will be applications and business scenarios that we’ve not seen before,” comments Drach.

Read more about the Kinect Accelerator program.

Kinect for Windows team

Making Learning More Interactive and Fun for Young Children

Kinect for Windows — Thu, 22 Dec 2011 23:20:00 GMT

Although no two people learn in exactly the same way, the process of learning typically involves seeing, listening/speaking, and touching. For most young children, all three senses are engaged in the process of grasping a new concept.

For example, when a red wooden block is given to a toddler, they hear the words “red” and “block,” see the color red, and also use their hands to touch and feel the shape of the wooden block.

Uzma Khan, a graduate student in the Department of Computer Science at the University of Toronto, realized the Kinect natural user interface (NUI) could provide similar experiences. She used the Kinect for Windows SDK to create a prototype of an application that utilizes speech and gestures to simplify complex learning, and make early childhood education more fun and interactive.

The application asks young children to perform an activity, such as identify the animals that live on a farm. Using their hands to point to the animals on a computer screen, along with voice commands, the children complete the activities. To reinforce their choices, the application praises them when they make a correct selection.

(Please visit the site to view this video)

Using the speech and gesture recognition capabilities of Kinect enables children to not only learn by seeing, listening, and speaking; it lets them actively participate by selecting, copying, moving, and manipulating colors, shapes, objects, patterns, letters, numbers, and much more.

The creation of applications to aid learning for people of all ages is one of the many ways we anticipate Kinect for Windows will be used to enable a future in which computers work more naturally and intelligently to improve our lives.

Sheridan Jones
Business and Strategy Director, Kinect for Windows

Kinect for Windows – Building the Future

Kinect for Windows Team — Tue, 22 Nov 2011 18:55:00 GMT

Since announcing a few weeks ago that the Kinect for Windows commercial program will launch in early 2012, we’ve been asked whether there will also be new Kinect hardware especially for Windows. The answer is yes; building on the existing Kinect for Xbox 360 device, we have optimized certain hardware components and made firmware adjustments which better enable PC-centric scenarios. Coupled with the numerous upgrades and improvements our team is making to the Software Development Kit (SDK) and runtime, the new hardware delivers features and functionality that Windows developers and Microsoft customers have been asking for.

Simple changes include shortening the USB cable to ensure reliability across a broad range of computers and the inclusion of a small dongle to improve coexistence with other USB peripherals. Of particular interest to developers will be the new firmware which enables the depth camera to see objects as close as 50 centimeters in front of the device without losing accuracy or precision, with graceful degradation down to 40 centimeters. “Near Mode” will enable a whole new class of “close up” applications, beyond the living room scenarios for Kinect for Xbox 360. This is one of the most requested features from the many developers and companies participating in our Kinect for Windows pilot program and folks commenting on our forums, and we’re pleased to deliver this, and more, at launch.

Another thing we’ve heard from our pilot customers is that companies exploring commercial uses of Kinect want to operate with the assurance of support and future innovation from Microsoft. As part of Microsoft’s deep commitment to NUI, we designed the Kinect for Windows commercial program to give licensed customers access to ongoing updates in both speech and human tracking (where Microsoft has been investing for years), in addition to providing fully supported Kinect hardware for Windows. We’ve been captivated by the countless creative ways companies worldwide envision how their businesses and industries can be revolutionized with Kinect, and are proud to be helping those companies to explore the profound implications NUI has for the future.

Microsoft also has just launched a new initiative, the Kinect Accelerator incubation project run by Microsoft BizSpark. I will be serving as a Mentor for this program, along with a number of other folks from Microsoft. BizSpark helps software startups through access to Microsoft software development tools, connection to key industry players (including investors) and by providing marketing visibility. The Kinect Accelerator will give 10 tech-oriented companies using Kinect (on either Windows or Xbox360) an investment of $20,000 each, plus a number of other great perks. Applications are being accepted now through January 25th, 2012. At the end of the program, each company will have an opportunity to present at an Investor Demo Day to angel investors, venture capitalists, Microsoft executives (including me), media and industry influentials. I can’t wait to see what they (and maybe you?) come up with!

Craig Eisler
General Manager, Kinect for Windows

The Kinect Effect

Kinect for Windows — Wed, 09 Nov 2011 18:45:00 GMT

Have you heard about the “Kinect Effect” yet? It’s a term we started using around Microsoft shortly after the launch of Kinect last year to describe the amazing and creative ways Kinect was being applied to fields beyond gaming.

We’ve seen exploration with Kinect by artists, entertainers, retailers, educators, and physical therapists (just to name a few.) In fact, we’ve seen research in nearly every area imaginable.

Check out the (Please visit the site to view this file) to see what it’s all about. This video does a great job capturing the potential. And it’s this potential that drives the enthusiasm my team and I have for developing Kinect for Windows.

After just one short year, it still feels like every application is the first and each idea is new and fresh. We can’t wait to see what’s next when, early next year, Kinect for Windows will be available for commercial use. Then, we’ll see even more new ideas which will continue to inspire us, and others, to keep driving the innovation forward.

--Craig Eisler

General Manager, Kinect for Windows

It’s Official: Kinect for Windows is Coming Soon

Kinect for Windows — Fri, 04 Nov 2011 05:20:00 GMT

To commemorate the one-year anniversary of Kinect and the Kinect Effect, I sent an email to my team earlier this week. I’d like to quote for you what I said to them, “It all started with a revolutionary sensor and amazing software that turned voice and movement into magic. With that magical combination, last year the Interactive Entertainment Business at Microsoft showed the world how to re-imagine gaming. This year, we’re showing the world how to re-imagine entertainment. Next year, with Kinect for Windows, we will help the world re-imagine everything else.”

To mark the milestone, the Kinect for Windows team is celebrating with our own milestones: We’re starting up this blog, launching the official Kinect for Windows web site, and releasing beta 2 of the Kinect for Windows SDK. (And, yes, we will celebrate the anniversary more this evening– it’s been an amazing journey these past months!)

I know many of you are eagerly awaiting the Kinect for Windows commercial program coming in early 2012. My team is working hard to deliver a great product and I’m confident that it will be worth the wait.

We’ve already seen strong enthusiasm for Kinect among developers who have done amazing things with it in countless different ways, from education to healthcare, gaming to art installations, manufacturing to retail.

Currently, we have more than 200 companies taking part in our pilot program. They are telling us how Kinect for Windows will help them transform their products, their processes, their brands, and their businesses. Putting the power of Kinect + Windows into the hands of business leaders and technical visionaries will give them the tools they need to develop novel solutions for everything from training employees to visualizing data, from configuring a car to managing an assembly line.

The updated software development kit that we released today includes some great new features that help us get closer to realizing this vision, including faster skeletal tracking, better accuracy rate when it comes to skeletal tracking and joint recognition, and the ability to plug and unplug your Kinect without losing work/productivity.

Every day, I come to work and learn about another amazing application that a partner or other developer is doing with Kinect for Windows. I look forward to next year, when the potential goes exponential and everyone’s ideas, including yours, are part of that equation.

If you haven’t done so already, download the SDK and re-imagine the world with us.

--Craig Eisler
General Manager, Kinect for Windows