Richard Conwayby Richard Conway, UK Windows Azure User Group

In our day to day jobs it’s sometimes hard to find (or make) the time to really innovate. Hackathons can be a way to try new tech, learn from your peers and have some fun along the way. Some people enjoy them so much you’ll see them at nearly every hackathon, they’re practically addicted!

Back in April this year we ran our first hackathon in conjunction with Data Science London, a strong London community of data scientists and academics that meet once a month. We had several discussions about what to base the hackathon challenge on but decided that finding out who the 100 most influential people in Britain are was a good challenge. With a dataset supplied by PeerIndex, the teams worked through the night to determine whether their models fit the data.

We had nearly 250 people turn up, who formed themselves into 138 teams, including virtual teams from across the globe. UKWAUG members attended in droves and some had their first experience of data science and used languages such as Python and frameworks like scikit-learn. Andy Cross, Microsoft Big Data evangelist Wenming Ye and I gave an introduction to Windows Azure and how Microsoft views Big Data for anyone who wasn’t familiar with Windows. Despite some attendees having little knowledge of Microsoft’s cloud and data offerings many people were overwhelmed with the fact that we could spin up Linux virtual machines on demand and install Python and related toolkits in Windows Azure within minutes. Those who hadn't set their subscription up eagerly logged on to Windows Azure to get their free compute time for the weekend and spun up Extra Large Linux VMs to give them the compute power they need to crunch large amounts of data with Random Forest, SVD and Logisitic Regression algorithms to name but a few. We also demonstrated HDInsight, a joint effort between Microsoft and Hortonworks to get Hadoop to run on Windows and Windows Azure. Everyone was incredibly impressed that we could use Machine Learning libraries such as Mahout on the Microsoft stack backed by vast amounts of clustered compute.

We also had the benefit of having Don Syme and Kenji Takeda from Microsoft Research that gave awe inspiring talks on F# and what MSR does in general. We discovered that the Kinect was born from a single email from the Xbox team to a member of MSR and led to a machine learning revolution in pixel analysis from a concept paper MSR had published! Most attendees weren't even aware that Microsoft Research had published a Machine Learning library for C# developers called Infer.NET, a library my company has been using for testing for the last 8 months. Don Syme spoke about F# type providers and showed how you could use F# to get access to typed datasets such as those provided by the World Bank. We also had a talk on how F# and Excel could be used together to script and get data with ease. These talks must have been impacting because many attendees went away to try F# in their solutions especially when they learnt that they could use R and Python libraries from F#.


Teams starting the challenge

We had two prize categories which were Data Visualisation and Data Analytics. The latter one was more abstract since it revolved around best fits for a model and used Kaggle to index the entries. The former showed how data could be presented to users to show something meaningful in a colourful way. Two BizSparkcompanies won the former challenge. Overall seven prizes were won, donated by Microsoft, Nokia and others.

Given the success and the size of this hackathon we've decided to run another one on 27th July and again at the end September. You can register here, plus get a flavour of the previous hackathon look at http://www.bigdatahackathon.com. Follow me on twitter (@azurecoder) or keep on the ukwaug.net website if you want to find out more about the July hackathon.

The September hackathon will be a weekend event for about 400 people. The "flavour" of the hackathon will be smart cities and look at the data generated by smart devices and mobile devices and work out how to use this for various problems. This hackathon will be different from the last one in that we will be looking to couple data scientists with app developers. As such this will be a chance to mix and match and learn a little about Big Data if you come from an app background.

In the meantime please feel free to attend our other events or summer bootcamps at http://ukwaug.net in both London, Manchester and shortly Edinburgh.

<<Author Bio>>

Richard Conway is one of the founders of the UK Windows Azure Users Group. He is a Microsoft MVP for Windows Azure and can be followed @azurecoder. He works for Elastacloud which is a wholly Windows Azure consultancy focussing on distributed computing in the cloud, most notably Big Data and Big Compute workloads. He is the author of the popular open source component Fluent Management (http://nuget.org/packages/Elastacloud.AzureManagement.Fluent) which can be used to manage VMs, Cloud Services, Storage, SQL, Websites and Mobile Services in Windows Azure.