Although no two people learn in exactly the same way, the process of learning typically involves seeing, listening/speaking, and touching. For most young children, all three senses are engaged in the process of grasping a new concept.
For example, when a red wooden block is given to a toddler, they hear the words “red” and “block,” see the color red, and also use their hands to touch and feel the shape of the wooden block.
Uzma Khan, a graduate student in the Department of Computer Science at the University of Toronto, realized the Kinect natural user interface (NUI) could provide similar experiences. She used the Kinect for Windows SDK to create a prototype of an application that utilizes speech and gestures to simplify complex learning, and make early childhood education more fun and interactive.
The application asks young children to perform an activity, such as identify the animals that live on a farm. Using their hands to point to the animals on a computer screen, along with voice commands, the children complete the activities. To reinforce their choices, the application praises them when they make a correct selection.
Using the speech and gesture recognition capabilities of Kinect enables children to not only learn by seeing, listening, and speaking; it lets them actively participate by selecting, copying, moving, and manipulating colors, shapes, objects, patterns, letters, numbers, and much more.
The creation of applications to aid learning for people of all ages is one of the many ways we anticipate Kinect for Windows will be used to enable a future in which computers work more naturally and intelligently to improve our lives.
Sheridan JonesBusiness and Strategy Director, Kinect for Windows
As you might imagine, working in a nuclear power plant provides special challenges. One crucial aspect for any project is the need to minimize employee exposure to radiation by applying a standard known as As Low As Reasonably Achievable—ALARA for short.
How this works: Plant ALARA managers work with the maintenance groups to estimate how much time is required to perform a task and, allowing for exposure limits, they determine how many employees may be needed to safely complete it. Today, that work is typically done with pen and paper. But new tools from Siemens PLM Software that incorporate the Kinect for Windows sensor could change this by providing a 3-D virtual interactive modeling environment.
Kinect for Windows is used to capture realistic movement for use in the Siemens Teamcenter solution for ALARA radiation planning.
The solution, piloted at a US nuclear power plant last year, is built on Siemens’ Teamcenter software, using its Tecnomatix process simulate productivity product. Siemens PLM Software Tecnomatix provides virtual 3-D human avatars—“Jack” and “Jill”—that are integrated to model motion-controlled actions input with a Kinect for Windows sensor. This solution is helping to usher in a new era of industrial planning applications for employee health and safety in the nuclear industry.
"We're really revolutionizing the industry," said Erica Simmons, global marketing manager for Energy, Oil, and Gas Industries at Siemens PLM Software. "For us, this was a new way to develop a product in tandem with the industry associations. We created a specific use case with off-the-shelf technology and tested and validated it with industry. What we have now is a new visual and interactive way of simulating potential radiation exposure which can lead to better health and safety strategies for the plant."
Traditional pencil-and-paper planning (left) compared to the Siemens PLM Software Process Simulate on Teamcenter solution (right) with “Jack” avatar and Kinect for Windows movement input.
The Siemens Tecnomatix process planning application, integrated with the Kinect for Windows system, will give nuclear plant management the ability to better manage individual worker radiation exposure and optimize steps to reduce overall team exposure. As a bonus, once tasks have been recorded by using “Jack,” the software can be used for training. Employees can learn and practice an optimized task by using Kinect for Windows and Siemens “Jack” and “Jill”—safely outside of the radiation zone—until they have mastered it and are ready to perform the actual work.
"We wanted to add something more for the user of this solution in addition to our 3-D human avatars and the hazard zones created by our visual cartography; this led us to exploring what we could do with the Kinect for Windows SDK for this use case," said Dr. Ulrich Raschke, director of Human Simulation Technologies at Siemens PLM Software. “User feedback has been good so far; the addition of the Kinect for Windows system adds another level of interactivity to our application."
This Siemens solution grew out of a collaborative effort with Electric Power Research Institute (EPRI) and Fiatech industry association, which identified the need for more technologically advanced estimation tools for worker radiation dosage. Kinect for Windows was incorporated when the developers were tailoring the avatar system to the solution and exploring ways to make the user experience much more interactive.
"Collaboration with several key stakeholders and industry experts led to this innovative solution," said Phung Tran, senior project manager at EPRI. "We're pleased the industry software providers are using it, and look forward to seeing the industry utilize these new tools."
“In fact,” Tran added, “the tool is not necessarily limited to radiation work planning. It could help improve the management and execution of many operation, maintenance, and project-based tasks.”
Kinect for Windows team
Earlier this month, Geekwire announced the nominees for the 2012 Seattle 2.0 Startup Awards. We're honored Kinect for Windows was selected in the “Innovation of the Year” category, which looks at technologies, which are setting a course for ”where the world is going and the way of the future.”
Other nominees in this category are Symform, ExtraHop, LaserMotive, and Vioguard.
If you’re developing with the Kinect for Windows SDK and sensor, or simply a fan of the technology, cast your vote and help us become Seattle’s startup innovation of the year.
Voting ends Monday, April 23rd. Winners will be announced at the Seattle 2.0 Startup Awards bash on May 3 at the Experience Music Project (EMP) in Seattle.
Kinect for Windows Team
Kinect for Windows partners are finding new business opportunities by helping to develop new custom applications and ready-made solutions for various commercial customers, such as the Coca-Cola Company, and vertical markets, including the health care industry.
Several of these solutions were on display at the Microsoft Worldwide Partner Conference (WPC) in Toronto, Canada, where Kinect for Windows took the stage with two amazing demos as well as strong booth showings at the Solutions Innovation Center.
"Being part of the WPC 2012 event was a great opportunity to showcase our Kinect-based 3-D scanner, and the response was incredibly awesome, both on stage when the audience would spontaneously clap and cheer in the middle of the scan, and in the Kinect for Windows trade show area where people would stand in line to get scanned," said Nicolas Tisserand, co-founder of the France-based Manctl, one of the 11 companies in the Microsoft Accelerator for Kinect program.
Manctl's Skanect scanner software uses the Kinect sensor to build high quality 3-D digital models of people and objects, which can be sent to a 3-D printer to create detailed plastic extruded sculptures. "Kinect for Windows is a fantastic device, capable of so much more than just game control. It's making depth sensing a commodity," Tisserand added.
A demo from übi interactive in Germany uses the Kinect sensor to turn virtually any surface into a 3-D touchscreen that can control interfaces, apps, and games. "Kinect for Windows is a great piece of hardware and it works perfect[ly] with our software stack," reported übi co-founder David Hajizadeh. "As off-the-shelf hardware, it massively reduced our costs and we see lots of opportunities for business applications that offer huge value for our customers."
Snibbe Interactive created its SocialMirror Coke Kiosk to deliver a Kinect-based game in which players aim a stream of soda into a glass and then share videos of the experience with their social networks. "We were extremely excited to show off our unique Coca-Cola branded interactive experience and its unique ability to create instant ROI [return on investment] through our viral marketing component," reported Alan Shimoide, director of engineering at Snibbe.
InterKnowlogy developed KinectHealth to assist doctors with motion-controlled access to patient records and surgery planning tools. "A true game changer, Kinect for Windows allows our designers and developers to think differently about business cases across many verticals," noted Kevin Custer, the director of strategic marketing and partnerships at InterKnowlogy. "Kinect for Windows is not just how we interact with computers, but it offers unique ways to add gesture and voice to our natural user-interface designed software—the combination of which is changing lives of customers and users alike." "Avanade has already delivered several innovative solutions using Kinect, and we expect that demand to keep growing," said Ben Reierson, innovation manager at Avanade, whose Kinect for Virtual Healthcare includes video chat for connecting clinics to remote doctors for online appointments. "Customers and partners are clearly getting more serious about the possibilities of Kinect and natural user interfaces."
The following blog post was guest authored by Celeste Humphrey, business development consultant at nsquared, and Dr. Neil Roodyn, director of nsquared.
A company that is passionate about learning, technology, and creating awesome user experiences, nsquared has developed three new applications that take advantage of Kinect for Windows to provide users with interactive, natural user interface experiences. nsquared is located in Sydney, Australia.
At nsquared, we believe that vision-based interaction is the future of computing. The excitement we see in the technology industry regarding touch and tablet computing is a harbinger of the changes that are coming as smarter computer vision systems evolve.
Kinect for Windows has provided us with the tools to create some truly amazing products for education, hospitality, and events.
Education: nsquared sky spelling
We are excited to announce nsquared sky spelling, our first Kinect for Windows-based educational game. This new application, aimed at children aged 4 to 12, makes it fun for children to learn to spell in an interactive and collaborative environment. Each child selects a character or vehicle, such as a dragon, a biplane, or a butterfly, and then flies as that character through the sky to capture letters that complete the spelling of various words. The skeleton recognition capabilities of the Kinect for Windows sensor and software development kit (SDK) track the movement of the children as they stretch out their arms as wings to navigate their character through hoops alongside their wingman (another player). The color camera in the Kinect for Windows sensor allows each child to add their photo, thereby personalizing their experience.
nsquared sky spelling
Hospitality: nsquared hotel kiosk
The nsquared hotel kiosk augments the concierge function in a hotel by providing guidance to hotel guests through an intuitive, interactive experience. Guests can browse through images and videos of activities, explore locations on a map, and find out what's happening with a live event calendar. It also provides live weather updates and has customizable themes. The nsquared hotel kiosk uses the new gestures supported in the Kinect for Windows SDK 1.7, enabling users to use a “grip” gesture to drag content across the screen and a “push” gesture to select content. With its fun user interface, this informative kiosk provides guests an interactive alternative to the old brochure rack.
Kinect for Windows technology enables nsquared to provide an interactive kiosk experience for less than half the price of a similar sized touchscreen (see note).
nsquared hotel kiosk
Events: nsquared media viewer
The new nsquared media viewer application is a great way to explore interactive content in almost any environment. Designed for building lobbies, experience centers, events, and corporate locations, the nsquared media viewer enables you to display images and video by category in a stylish, customizable carousel. Easy to use, anyone can walk up and start browsing in seconds.
In addition to taking advantage of key features of the Kinect for Windows sensor and SDK, nsquared media viewer utilizes Windows Azure, allowing clients to view reports about the usage of the screen and the content displayed.
nsquared media viewer
Kinect for Windows technology has made it possible for nsquared to create applications that allow people to interact with content in amazing new ways, helping us take a step towards our collective future of richer vision-based computing systems.
Celeste Humphrey, business development consultant, andDr. Neil Roodyn, director, nsquared
____________Note: Based on the price of 65-inch touch overlay at approximately US$900 compared to the cost of a Kinect for Windows sensor at approximately US$250. For integrated touch solutions, the price can be far higher. Back to blog...
Styku, a Kinect Accelerator startup, set out to alter clothes shopping for retailers by using the Kinect for Windows sensor and software development kit to develop its Smart Fitting Room quickly, a new case study reports.
The technology will soon be used by Brooks Brothers, IM-Label, and other fashion retailers. Styku hopes to improve the shopping experience—reducing the problem of shoppers returning up to 40 percent of their online purchases and offering a faster, less expensive body scanning solution. Additionally, military apparel contractors appreciate the improved measurement capability of Kinect for Windows with the Styku software—estimated to be up to 400 percent more accurate—which could save soldiers' lives, thanks to better fitting body armor.
Customers can quickly visualize the fit and fabric characteristics of garments over digital renderings of their bodies that are created by scanning their body with the Kinect for Windows sensor. The scan lasts only one second—reducing the risk that a fidgety customer will compromise the scan’s accuracy. Clothing is rendered in 3-D, and customers can use gesture to rotate, view a custom-fit color map, and compare multiple sizes.
"Kinect for Windows had exactly the sensors that we needed, in a small package," said Pierre Du Charme, vice president of Software Engineering for Styku. "The SDK was easy to learn and gave us the tools to quickly implement a full-featured application."
Twelve weeks ago, I announced that the Microsoft Accelerator for Kinect had opened its doors and the 11 participating teams had arrived in Seattle. Yesterday, the program concluded with Demo Day—an all-day event attended by more than 150 investors and journalists—where each of the startups presented their business plans and applications.
From the beginning, we believed this program was going to be amazing: we had hoped to receive 100 to 150 applications, but ended up with nearly 500 from more than 60 countries. There were so many amazing, creative ideas from a whole range of talented, successful people. As I said in a previous post, getting to the finalists was super challenging.
The teams who came here to Seattle—leaving jobs, families, university, and the comforts of their daily lives—did not disappoint. Their energy, drive, and innovative thinking were a constant source of inspiration to me and the folks across Microsoft that worked with them.
There were a lot great moments at Demo Day; here are just a few of many:
I think all the Kinect Accelerator companies have done an outstanding job the past 12 weeks and have bright futures ahead. These 11 teams are helping accelerate and push the boundary of what’s possible with Kinect for Windows, and inspiring others to think creatively about what the future looks like when Kinect-enabled, touch-free NUI experiences are commonplace.
Thanks to all of the teams that participated in the Accelerator and to the many others who applied. Keep up the great work!
Craig EislerGeneral Manager, Kinect for Windows
Last week, winners of GeekWire’s fourth annual Seattle 2.0 Startup Awards were announced. Seattle has a vibrant startup community, and this is a very popular event within that community. I attended the awards ceremony, and it was amazing to see how much energy and excitement was in the room – and the EMP (Experience Music Project) where the event was held was packed.
There were entrepreneurs of all types and levels of experience (I even ran into Ray Ozzie!), venture capitalists, CEOs, and so much IQ and passion in one place it was a rush – the same rush I feel every Monday when I spend time at the Kinect Accelerator in Microsoft’s Westlake office.
We were thrilled when Kinect for Windows was named “Innovation of the Year” – not because we won (which is great!), but because it was the popular vote of the startup community. This is the same community that is continuing to deliver so many amazingly different ideas and products that have Kinect for Windows at the center. In so many ways, Kinect for Windows is “Innovation of the Year” because of the innovators who are using it: Thank you.
Learn more about all of the winners of the Seattle 2.0 Startup Awards. And thanks to all of you who voted for Kinect for Windows!
Swivel Close-Up, a Kinect for Windows-based kiosk from FaceCake, lets customers visualize themselves in small accessories such as makeup, sunglasses, and jewelry.
Microsoft Kinect for Windows has been playing an increasingly important role in retail, from interactive kiosks at stores such as Build-A-Bear Workshop, to virtual dressing rooms at fashion leaders like Bloomingdale's, to virtual showrooms at Nissan dealerships. This year's National Retail Federation (NRF) Convention and Expo, which took place earlier this week, showcased several solutions that provide retailers with new ways to drive customer engagement, sales, and loyalty.
Trend watchers have noted significant shifts in how consumers shop—often blending online and in-store investigation by using phones, tablets, kiosks, and computers in addition to good old-fashioned salesperson interaction. Brick-and-mortar stores, which are facing vigorous competition from online resellers, are embracing new technologies like Kinect for Windows to help drive sales and retention—and to delight and surprise customers with fun, interactive shopping experiences. Even better, customers can get more accurate and personalized information about whether a specific product is right for them—whether it's an article of clothing or a piece of furniture—reducing dissatisfaction and inconvenient returns.
"This past holiday season, we’ve seen retailers get much more tech savvy in how they engage customers and offer more flexibility in how they shop," said Kinect for Windows Senior Channel Development Manager Michael Fry. "As the lines between traditional and digital shopping channels continue to blur, retailers must seek new ways to deliver the most value and earn loyalty through compelling, seamless experiences across all touch points with their customers. Technologies like Kinect for Windows help retailers engage customers with interactive shopping experiences that are not only fun, but also increase important bottom-line business results—increasing engagement, awareness, and brand value while making it easier to select the best products for them."
At a hospitality event during NRF, Kinect for Windows partner Avanade showed one such innovation: their "shoppable storefront," created for my-wardrobe.com in Norway. Customers can walk up to the showroom window and—even after business hours—interact with the Kinect for Windows sensor to browse the store catalog, view pricing, and scan a Quick Response (QR) code to quickly purchase the product online via mobile phone. See a video of how it works.
"Consider the possibilities within the store, they're almost endless with a technology like Kinect for Windows," said John Konczal, director of service line marketing at Avanade. "You could build a guide for customers to find more information about products and quickly locate them in the store. If an item is not available, order it for shipment and pick-up at the nearest store. The interactivity, simplicity, and responsiveness of this technology can really help retailers differentiate their stores from the competition."
Avanade also demonstrated Natural User Observation of Retail Displays (NUO), which provides a cost-effective solution for retailers by gathering real-time customer response and behavior. This allows retail managers to do things like determine where customers are spending their time in the store, identify trends, and gather demographic and customer behavior as they interact with store displays. Avanade reports that the solution integrates into existing store and back-office IT systems and provides dashboards and data-rich reporting for improving product placement, marketing effectiveness, and overall display performance.
Another of our partners, FaceCake Marketing Technologies, Inc., which developed Swivel, the 3-D virtual dressing room that's been featured at Bloomingdale's, showed NRF attendees the newest enhancements to their Swivel software. The enhancements, which work in conjunction with the latest Kinect for Windows SDK, include face-tracking and a feature called real-time Compare, which allows you to contrast two looks in a full-motion visualization of yourself in two dresses (or any type of clothing) side-by-side. Sizing is now even more accurate, and FaceCake also added multi-user functionality that allows, for example, a bride to see herself, virtually, in various wedding dresses at the same time as her bridesmaids see themselves in their bridesmaid dresses.
We also featured another exciting new product from FaceCake in our booth: Swivel Close-Up. This Kinect for Windows-based kiosk, which operates within a two-foot environment, lets customers try on much smaller accessories than clothing including makeup, sunglasses, and jewelry. Earrings dangle and twist beautifully as a shopper tries them on virtually, and consumers now have the opportunity to try on a limitless number of lip colors without lipstick ever touching their lips.
"We can now provide an extended Try-On solution that is real-time, 3-D, and full motion as opposed to just uploading a static image and then modifying it," said FaceCake CEO Linda Smith. "The result is a lifelike representation that's just like looking in a mirror—your dream dressing room mirror powered by Swivel and Kinect for Windows! It's both efficient and fun for the customer."
One of the key themes of this year's NRF event was putting customers at the center of retail marketing, something that Kinect for Windows accomplishes readily, thanks to its ability to quickly entice customers into virtual shopping spaces within actual storefronts, making it easier than ever for them to find, experience, and purchase products that are right for them.
"Staying competitive in retail today means putting customers at the heart of the business and seeking new ways to deliver value in the store," Fry said. "A Kinect for Windows retail display immediately puts the focus on the shopper, delivering uniquely personalized results that drive both sales and customer satisfaction."
Earlier this week at the 2012 Seattle Interactive Conference, Oscar Murillo, user experience architect for Kinect for Windows, kicked off a six-person panel discussion about the transformational power of voice and gesture technology with a demonstration that showed participants how much the Kinect sensor has grown beyond its gaming roots.
"Kinect for Windows is a premier technology that enables users to interact with systems without touching a user interface" noted Murillo. "Human-to-human interactions are fluid and multimodal. With Kinect for Windows, we see human-computer interactions that are coming closer to mirroring the way humans naturally interact: effortless, transparent, and contextual communication between users and technology—by using voice and gesture—are now becoming possible. We see interactions that are as natural as human beings themselves." Murillo illustrated how, by using Kinect for Windows, he can control an environment with his body and voice. The sensor changed his appearance and even placed him in different environments by using "real-time green screening" to provide a museum setting and an abstract landscape of cubes and spheres with which he could interact. Murillo tracked 100 different points on his face and showed both thermal and radiant scanning. The use of these and other emerging techniques provides "a novel way for users to interact with products, brands, environments, services, and each other," Murillo added.
After Murillo’s presentation, Steve Clayton, editor of Next at Microsoft blog, moderated a panel of NUI thought leaders from around the world who are using Kinect for Windows in their work.
Academy Award-winning visual effects designer John Gaeta, who developed the "bullet time" effects in The Matrix Trilogy, also worked on Kinect for Windows in its early stages. His company, Float Hybrid Entertainment, develops interactive displays and participates in the Kinect for Windows advisory board.
"The thing that is interesting is the human interface part of it," observed Gaeta. "To allow people to have some sort of method to reflect themselves back, and that there can be a two-way relationship between the average person and a machine."
Scott Snibbe—founder of Snibbe Interactive and a world-renowned interactive media entrepreneur, researcher, and artist—is also an early pioneer with Kinect, which he has been working with since 2006. "With NUI, we can finally put the person in control of the computer instead of the computer controlling the person," he explained. "Humans are first and foremost social; Kinect for Windows can power social NUIs that respond to gesture and voice—the same way humans communicate with each other."
Matt Von Trott, digital director and partner at Assembly Ltd., noted the growing appeal of Kinect for Windows to the advertising industry. "In advertising these days, you need to make something that does more than make people talk about it," he said.
James Ashley, presentation layer architect on the Emerging Experiences team at the international digital agency Razorfish and a Microsoft Most Valued Professional for the Kinect sensor, remarked that he was skeptical the first time he saw the early concept videos for the project that eventually became Kinect for Windows. He didn’t believe it could really work. But it did, and the results were magical. "People want it and clients want it," he explained.
David Kung, vice president of business development at Oblong Industries, Inc. and former Disney Imagineer, notes that the bar for entry is much lower than with previous technological advances. "What's most exciting is how the developer community is adopting at a very low investment," he said. "Not just at a highly expensive R&D level."
"We still have a while to go before we get to true multi-modal—we are crawling still for sure," Kung added. "We can envision a time where technology could potentially answer a child's question while looking out the car window, 'What is that?' With NUI, GPS, and other advancements, such scenarios are possible."
This year's Seattle Interactive Conference (October 29 and 30) connected about 4,000 entrepreneurs, developers, and online business professionals who are all aspiring to explore the latest online technology and emerging trends. "We're straight-up geeks who just love technology, noted SIC co-founded Mark Peterson. “So, partnering with Microsoft and Kinect for Windows just made sense."