Download Research Tools
Well, now it is. Today, we are pleased to announce the launch of Code Hunt, a browser-based game for anyone who is interested in coding. We built Code Hunt to take advantage of the fact that any task can be more effective and sustainable when it’s fun. And Code Hunt is fun! It uses puzzles, which players explore by means of clues presented as test cases. Players iteratively modify their code to match the functional behavior of secret solutions. Once their code matches, lights flash and sounds play, letting players know that they have “captured” the code. Players then get a score, which depends on how elegant their solution is, and are encouraged to move on to the next puzzle or level.
When we demoed Code Hunt a few months ago, we were amazed at the interest it elicited across groups at Microsoft, from those involved with K-12 education to those focused on college recruiting. However, today we want to talk about how Microsoft Research Asia used Code Hunt during their annual Beauty of Programming (BOP) event, a competition that attracts thousands of students in the Greater China Region (GCR).
In the past, the BOP competition gave students specifications for problems and then checked their solutions automatically using a test suite. This is the traditional approach: students pit their wits against each other—and against the clock—to create a solution to a defined problem. While this kind of coding is similar to what they will encounter in courses or later in their careers, it isn’t necessarily fun.
Code Hunt is different. Instead of giving students a problem and comparing their solutions to a set of fixed test cases, Code Hunt does the opposite: it presents an empty slate to the user and a set of constantly changing test cases. It thus teaches coding as a by-product of solving a problem that is presented as pattern matching inputs and outputs. The fun is in finding the pattern. Fun is seen as a vital ingredient in accelerating learning and retaining interest during what might be a long and sometimes boring journey towards obtaining a necessary skill—or in this case, winning a competition. The GCR team recognized that Code Hunt would not only make the BOP competition more fun, but it would also enable them to check the solutions more quickly and accurately.
With considerable optimism, we opened Code Hunt to BOP competitors in April. In three rounds, 2,353 students scored in the game, and the contestants solved an average of 55.7% of the puzzles. Since Code Hunt runs on Microsoft Azure, we have all the statistics. We could see that, on average, it took players 41 tries to capture the code for puzzles. However, we were really interested in the 350 top students who solved all of the puzzles—even the most difficult ones. These students needed only 7.6 tries on average to solve a puzzle, showing that Code Hunt can reliably surface the better coders. From these students, 13 were selected to proceed to the finals, and we wish them luck.
Code Hunt was developed by a team in Microsoft Research led by Principal Development Lead Nikolai Tillmann and Principal Research Software Engineer Peli de Halleux. It is based on Pex, Microsoft Research’s state-of-the-art implementation of dynamic symbolic execution (analyzing a program to determine what inputs cause each part of a program to execute), which is available as a Power Tool in Microsoft Visual Studio.
We look forward to Code Hunt’s further application and would be happy to receive inquiries regarding competitions or courses. But remember, anyone can play Code Hunt—for fun or to hone their coding skills. Just go to www.codehunt.com and start coding!
—Judith Bishop, Director of Computer Science, Microsoft Research, and Guobin Wu, Research Program Manager, Microsoft Research Asia
A very interesting concept, but I believe the puzzles are too biased towards the skills found in the Microsoft Research group: a lot of math, logic puzzles, etc.
I don't feel a better coder if I guess the requirement for a puzzle from just one test case, nor do I feel a bad coder because I can't find the correlation between some numbers.
In no real life scenario one would have to guess what a class does from the unit tests some other coder did and that are revealed one by one. I really appreciate the game, I spent my whole day playing, but I would have preferred to know what I am working towards and only then have the unit tests verify my work.
On the other hand, I can see this as being a two-parter game. The second part would be to get the description of the problem and create unit tests for the class that should solve it. Then run all submitted classes against your tests.
Although I admit it would be difficult to test the tests automatically, I see a much better value in learning the thoroughness of creating valid tests with 100% coverage than guessing functionality from partial test results.
This isn't a learning tool. It will let you fail hundreds of times with no hints or clues. Don't bother with this.
I'm honestly really quite disappointed with this. It's a fantastic concept but the execution is very flawed.
For anyone who would like to play this, know that it's not a tool to learn but more there to test your knowledge of C# and programming. And at that, the tests don't show you all the required cases in order to figure out the solutions immediately.
I need a little tutorial for the loop with array game Level 02.01
can you send me data to firstname.lastname@example.org?
Thank you a lot!
I'm blogging some hints at:
07.04 - the system thinks this java solution is poor to reverse a string. I believe the usage of char arrays with reversion is not really better.
return (new StringBuilder(s)).reverse().toString();;
07.07 couldn't handle s.replaceAll("[aueio]", ""));
08.01 throwing "path bounds exceeded (path bounds exceeded)" because it gives too high parameter for int calculation (64 loool).
Interesting try...! Like the looks, good interface... Just started... :)
Thanks everyone for the comments and thanks for playing Code Hunt. One of the objectives of Code Hunt was to present a gaming alternative to traditional coding – hence the player is not given a specification or told what to do. The game is to find the pattern in the test cases and thereby solve the puzzle. As such, Code Hunt exercises logic as well as coding skills.
The underlying engine that drives Code Hunt is at its best when it works with logical puzzles. The early sectors are numerical, but later on they become quite challenging. Learning to deal with the results of unit tests is a very important skill in debugging programs, and one that is actively taught in college classes. Code Hunt has proven to challenge coders at all levels, but we are always looking to improve it. Feel free to post feedback about specific puzzles directly on www.codehunt.com. We hope you will enjoy the game!
Microsoft Research Connections team
Posted by Microsoft Research
Our most popular blog posts of 2014 reflect the breadth of our research and our collaborative efforts across multiple product groups as well as with external organizations worldwide. From 3-D visualization to unveiling