The goal of this site is to put relevant and applicable tools and information at the fingertips
With this blog we want to inform you on our latest initiatives.
Enjoy reading and stay tuned!
Last week I attended The 2012 Eduserv Symposium. The event was focused on 'Big Data' and discussed whether Big Data represents a challenge or an opportunity and how we can best make use of it. This year's event was held at the Royal College of Physicians.
The key message from the introduction was the importance of uncoupling the issues of Big Data from the industry hype. Throughout the event the Gartner 3 V’s (velocity, volume and variety of data ) were discussed in depth.
Volume as a defining attribute of Big Data
Whilst it is fairly obvious that data volume is the primary attribute of big data, people often ask for a definitive quantity in GB, TB, PB etc. that would qualify as big data.
Whilst simplest answer is to give a data volume, for example 50TBytes which today present represents a reasonably large and expensive dataware house –this answer of course changes as the technology changes over time due to Moore’s Law.
But it’s also worth thinking about what it is you are looking as data, for instance for a large library of photographs the actual data contained in all the images themselves is very large for instance a RAW File from a Nikon D300 is about 25MB, so library with 2 million such images would be about 50TBytes , but the meta data describing those images isn’t that large perhaps 2GBytes.
So to someone actually searching the images for content, e.g. using facial recognition find all the photos of Adele, then that is a big data problem, but if the photos have already been labelled and tagged as being of Adele then that isn’t really a big data challenge as you are only search the Meta data.
Data Feed Velocity as a defining attribute of Big Data
Big data can be described by its velocity or speed. Or you may prefer to think of it as the frequency of data generation or frequency of data delivery. For example, think of the stream of data coming off of any kind of sensor, say thermometers sensing temperature, microphones listening for movement in a secure area, or video cameras scanning for a specific face in a crowd. This isn’t new; many firms have been collecting click stream data off of Web sites for years, using streaming data to make purchase recommendations to Web visitors. With sensor and Web data flying at you relentlessly in real time, data volumes get big in a hurry.
Technologies such as Stream Insight are well positioned for certain types of streaming data, whilst other applications may need specialist development or tools.
Data Feed Variety.
A proliferation of data types from social, machine to machine, and mobile sources add new data types to traditional transactional data. Data no longer fits into neat, easy to consume structures. New types include content, geo-spatial, hardware data points, location based, log data, machine data, metrics, mobile, physical data points, process, RFID’s, search, sentiment, streaming data, social, text, and web. The addition of unstructured data such as speech, text, and language increasingly complicate the ability to categorize data. Some technologies that deal with unstructured data include data mining, text analytics, and noisy text analytics.
A key message was Big Data and the role of the Data Scientist, was not limited to the computer Scientist of the future as Big Data is of use to all disciplines.
Please find below a quick overview deck of how we see the Big Data opportunity at Microsoft.
A interesting case study is Klout Data Services Firm Uses Microsoft BI and Hadoop to Boost Insight into Big Data
Klout wanted to give consumers, brands, and partners faster, more detailed insight into hundreds of terabytes of social-network data. It also wanted to boost efficiency. To do so, Klout deployed a business intelligence solution based on Microsoft SQL Server 2012 Enterprise and Apache Hadoop. As a result, Klout processes data queries in near real time, minimizes costs, boosts efficiency, increases insight, and facilitates innovation.
Klout helps clients make sense of the hundreds of terabytes of data generated each day by more than 1 billion signals on 15 leading social networks including Facebook and LinkedIn. The data that Klout analyzes is generated by the more than 100 million people who are indexed by the firm. This includes Klout members and the people that they interact with on social sites. Individuals join Klout to understand their influence on the web, which is rated on a scale from 1 to 100. They also sign up to participate in campaigns where they can receive gifts and free services. More than 3,500 data partners also join Klout to better understand consumers and network trends including changes in demand and how peoples’ influence might affect word-of-mouth advertising.
To deliver the level of insight that customers seek and yet meet the budget constraints of a startup firm, Klout maintained a custom infrastructure based on the open-source Apache Hadoop framework, which provides distributed processing of large data sets. The solution included a separate silo for the data from each social network. To manage queries, Klout used custom web services, each with distinct business logic, to extract data from the silos and deliver it as a data mashup.
Maintaining Hadoop and the custom web services to support business intelligence (BI) was complex and time-consuming for the team. The solution also hindered data insight. For example, accessing detailed information from Hadoop required extra development, and so mashups often lacked the level of detail that users sought. In addition, people often waited minutes, or sometimes hours, for queries to process, and they could only obtain information based on predetermined templates.
Klout wanted to update its infrastructure to speed efficiency and support custom BI. Engineers sought technologies that could deliver mission-critical availability and still scale to meet big-data growth and performance requirements.
In 2011, Klout decided to implement a BI solution based on Microsoft SQL Server 2012 Enterprise data management software and the open-source Hive data warehouse system. Based on employees’ previous experience with the Microsoft BI platform, Klout also knew that SQL Server offers excellent compatibility with third-party software and it can handle the data scale and query performance needed to manage big-data sets.
In August 2011, engineers implemented a data warehouse with Hive, which consolidates data from all of the network silos hosted by Hadoop. In addition, Klout deployed SQL Server 2012 on a system that runs the Windows Server 2008 R2 Enterprise operating system to take advantage of Microsoft SQL Server 2012 Analysis Services. Engineers use it to manage all business logic required to facilitate multidimensional online analytical processing (MOLAP). Data is stored in multidimensional cubes, which helps preserve detail and speed analysis. To provide high availability, Klout replicates the database to a secondary system using SQL Server 2012 AlwaysOn.
At the time that Klout was initially deploying its solution, SQL Server 2012 and Hive could not communicate directly. To work around this issue, engineers set up a temporary relational database that runs MySQL 5.5 software. It includes data from the previous 30 days and serves as a staging area for data exchange and analysis. Klout engineers are currently working to implement the new open database connectivity driver in SQL Server 2012 to directly join Hive with SQL Server 2012 Analysis Services. In addition, to enhance insight Klout plans to work with Microsoft to incorporate other Microsoft BI tools into its solution, such as Microsoft SQL Server Power Pivot for Microsoft Excel.
With its new solution, Klout expects to boost efficiency, reduce expenses, expand insight, and support innovation.
Speeds Efficiency and Cuts Costs By taking advantage of the Microsoft platform for BI, users will be able to get the data they seek in near real time.
Klout is implementing the flexible and scalable infrastructure it needs to continue to push the limits of data analysis.
This case study is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.
Microsoft and partners NetApp and Citrix announced upcoming native support for FreeBSD support on Windows Server Hyper-V.
This move continues Microsoft’s commitment to extend support across platforms to the Windows Server Hyper-V solution, making it easier for more customers to realize the benefits of server virtualization and more easily adopt cloud computing.
This will allow FreeBSD to run as a first-class guest on Windows Server Hyper-V. The drivers and associated source code will be released early this summer under the BSD license, and will initially work with FreeBSD 8.2 and 8.3 on Windows Server 2008 R2.
You can read more about this on the Openness blog.
You can read the full interview with Joe CaraDonna, the Technical Director of Core Operating Systems at NetApp, here.
We have been working closely with Steven Johnston and András Sóbester members of the Astra Research team from Southampton University over the past year, since last April when the team ASTRA team launched the first Windows Phone device with huge success.
Over the past year, the ASTRA Atmospheric Science Through Robotic Aircraft research team, are now using a number of Microsoft technologies including, Microsoft .Net Gadgeteer, Windows Embedded, Windows Azure and Windows Phone.
The Astra team recently launched a new project which included Windows Azure and a Nokia Lumia 800 device, the device was mounted onto a weather balloon to carry out some experiments whilst in flight. The balloon and device reached a height of 105,000 feet, with a temperatures of -61C and maintained data connectivity at 8km above Earth with the Windows Azure cloud completing data capture and analysis of the flight, the results will be used to help create balloon flight trajectory prediction software that can then be used to guide other missions, like weather balloons.
The experiments completed by ASTRA (Atmospheric Science Through Robotic Aircraft) project investigates new technologies for making low cost observations of the parameters of the atmosphere and their latest experiment, further details of the project can be found at Nokia Conversations blog,
The balloon and payload travelled for 2 hours and 22 minutes on a journey over South Wales en route to Cornwall. It took some quite pretty pictures along the way too.
A small selection of the 2GB image set is available to download.
Back in January this year, we announced Visual Studio Achievements for Developers – a fun way bringing Xbox LIVE-like points to everyday programming tasks. The fascinating program makes sure your students hard day’s work doesn’t go unnoticed and after over 80,000 downloads since the initial release, As your aware from my previous blog we have now extended the game to enroll developers working on Windows Azure.
The Visual Studio Achievements Extension has now been updated to include fifteen new achievements, all focused on exercising features of Windows Azure. Using the extension, various achievements are unlocked based on your activity. When you unlock an achievement, Visual Studio lets you know visually with a pop-up. In addition, your Channel 9 profile is updated with any achievements you earn. So, head over to Channel9, sign up for an account and download the plugin.
There are 15 new Azure achievements, such as publishing to Windows Azure from Visual Studio (Heading into the Cloud), using page blobs (Attack of the Blob), using SQL Azure (Database Darling) and configuring start up tasks (It’s My Party). Two of the achievements – Phone in the Cloud and Game in the Cloud – require use of Windows Azure toolkits.
You might also want to check Visual Studio Achievements for Windows Phone – a Windows Phone app that allows you to view Visual Studio Achievements on the go.
The Windows Phone application developed by Den Delimarsky allows viewing Visual Studio Achievements on the go. The source code for the application is available on Codeplex. You can choose to pull the source and compile a XAP yourself, and then side-load it on your phone.
Interestingly, Den has authored a great series of blog posts about this project on DZone with details on each of the different areas, passing on some lessons learned in building this app.
Visual Studio Achievements for Windows Phone – Developing the core Visual Studio Achievements for Windows Phone – What it looks like Visual Studio Achievements for Windows Phone – User details & official project page Visual Studio Achievements for Windows Phone – Storage Visual Studio Achievements for Windows Phone – Basic User Comparison Visual Studio Achievements for Windows Phone – Final notes before the beta
You can also Download the app directly from the Windows Phone MarketPlace the app is called VSA.
Microsoft Small Basic is, in my opinion, one of the really cool platforms for introducing programming to younger students. A FREE simple IDE, a simple subset of BASIC, and built in turtle graphics all resonate with me. And a number of teachers around the world seem to agree.
What is really great is that we have some Excellent curriculum support. This includes a complete set of PowerPoint presentation with notes for teachers . There are other resources available
Microsoft Small Basic puts the fun back into computer programming. With a friendly development environment that is very easy to master, it eases students of all ages into the world of programming.
Read the Small Basic FAQ
Download Small Basic from DreamSpark.com
Use the Getting Started Guide to start learning Small Basic
Use the curriculum to expand your knowledge
Read sample chapters from e-books to dive deep into Small Basic
Ask Questions in the Forums
Stay informed about Small Basic by reading our blog
Small Basic Curriculum Resources – Tutorials and Guides.
If you use this curriculum or anything else associated with Small Basic we would love to hear from you? Also any feedback about what we can do better or differently would be appreciated.
Microsoft Student to Business http://www.microsoft.com/studentstobusiness/
Coming soon – Career Week! Week commencing May 14th
Join the discussion on Microsoft student experience Facebook group, the Facebook group will host a themed week around career-related topics. As part of the career week, students will be able to participate in a live Facebook chat with Kevin Otten, Microsoft University Recruiter.
Microsoft IT Academy category has been added to the S2B portals.
This new category will expose jobs requiring MS certifications, encouraging more students to complete courses to increase their employability.
F# for Education and Innovation Workshop
The F# for Education and Innovation workshop will be held at the University of Pisa on June 28, 2012.
The free, one-day workshop provides a forum for attendees to learn and explore the use of F# 3.0 for education, research, and industry. Don Syme, from Microsoft Research Cambridge, will present the keynote session. The workshop will highlight the latest developments for information-rich programming in F# 3.0, delivered through Microsoft Visual Studio 11 and Try F#. The workshop is preceded by a free F# tutorial session on June 27.
If you are interested in attending this workshop, contact Kenji Takeda Microsoft Research.
TouchDevelop is a novel software development environment that lets users write programs for Windows Phone directly on a smartphone, without the need of a separate PC. The Windows Phone app can be used in the classroom—along with accompanying videos, a book, and slides—to teach programming concepts. TouchDevelop is ideal for courses on mobile computing, as it reduces the time required to write apps. It is also relevant for some compiler and computer-language courses.
Windows Phone Loan Programme
To help students gain access to this revolutionary new programming language and environment, we have established a program to lend academics Windows Phones for use in a course for one semester. If your interested please send a short proposal to firstname.lastname@example.org. The proposal should include the name of the instructor, the course website, the number of students, and how the smartphones will be used.
· Learn more about TouchDevelop
· Read about the TouchDevelop programming environment
· Learn programming with TouchDevelop
Windows 8 offers unparalleled new opportunities for application developers to build and sell apps world-wide via the Windows Store. In this event, we'll deliver developer-level, demo-driven sessions that give you an accelerated entry into what it means to design, develop and publish exciting, modern, polished, world-ready applications for next-generation devices running Windows 8.
Were hosting a special one day event for UK Students only at
1. Beginners Welcome.
2. A familiarity with .NET development and Visual Studio would be advantageous but not required.
Register Now to avoid disappointment
Agenda (subject to change):
09:30 – 10:00 - Registration Opens
10:00 – 11:00 - Windows 8 for Modern App Development
11:00 – 12:00 - Metro Design Language
12:00 – 12:30 - Lunch
12:30 – 13:30 - "Metro Style" Apps - The Power of the Device
13:30 – 14:30 - “Metro Style" Apps - The Power of the Cloud
14:30 – 15:00 - From Idea to Store – Next Steps on the Journey
“Imagine a world where technology helps solve the toughest problems.”
The Microsoft Imagine Cup is the world’s premier student technology competition. It provides an opportunity for students to use their creativity, passion, and knowledge of technology to help solve global challenges and make a difference in the world. While competing for cash and prizes, students gain real-life experiences, make new friends, and change the world.
Now in its tenth year, the Imagine Cup has grown to be a truly global competition focused on finding solutions to real-world problems. 358,000 students representing 183 countries and regions registered for the Imagine Cup 2011 Competition. The Imagine Cup is one way Microsoft encourages the brightest young minds to join together and use technology to take on the toughest problems facing our world today, whether it is combatting diseases, improving education or creating disaster communication systems.
UK Entries for 2012 Imagine Cup Competition
Alperton Community School
Anglia Ruskin University
Banff and Buchan
Belfast Metropolitan College
Birkbeck College, University of London
Birmingham City University
Birmingham Metropolitan college
Blackpool & the Fylde College
Bsix Brooke House Sixth Form College
Canterbury Christ Church University
Cardiff Metropolitan University
City College Plymouth
City of Glasgow College
City of Sunderland College
City of Wolverhampton University
City University London
College of North West London
Coopers Company And Coborn School
De Montford University (Leicester)
Dumfries and Galloway College
Edge Hill University
Edinburgh's Telford College
Forth Valley College
Glasgow Caledonian University
Gunnersbury Catholic School
Haberdashers' Aske's Boys School
Heriot Watt University
Highdown School and Sixth Form Centre
Holy Cross College, Bury.
Imperial College London
Isle of Wight College/Chichester University
John Flamsteed Community School
Ken Stimpson Community School
King Edward VI Camp Hill School for Boys
King George IV College
Kings College London
Kirkham Grammar School
Leeds Metropolitan University
Leeds University Business School
Liverpool Hope University
Liverpool John Moores University
London Metropolitan University
London School of Commerce
London South Bank University
Manchester Metropolitan University
Meridian Business School
Mill Hill Country High School
Monkseaton High School
Newcastle Upon Tyne University
Nottingham Trent University
Oriel High School
Queen Marry University of London
Queens University Belfast
Reid Kerr College
Robert Gordon University
Sir John Deane's 6th Form College
South Downs College
South Essex College Basildon Campus
St Andrews University
St Leonard's RC Comprehensive School
St Pauls Catholic College
St Peter's Sixth Form College
St Wilfrids Sixth Form
St.Augustine High School
Sutton Coldfield College
The Coopers' Company and Coborn School
The Grange School
The Sutton Academy
The University of Birmingham
The University of Bradford
The University of Nottingham
The University of Salford
The University of Sheffield
The University of the West of England
The University of the West of Scotland
Titanic Quarters Belfast BMC (Belfast Metropolitan College)
Torquay Boys' Grammar School
Trinity Saint David
Tyne Metropolitan College
University of Aberdeen
University of Abertay Dundee
University of Bath
University of Bedfordshire
University of Birmingham
University of Bolton
University of Bradford
University of Brighton
University of Bristol
University of Cambridge
University of Central Lancashire
University College London, UCL
University of Chester
University of Derby
University of Dundee
University of Durham
University of Edinburgh
University of Essex
University of Glamorgan
University of Gloucestershire
University of Greenwich
University of Hertfordshire
University of Huddersfield
University of Hull
University of Kent
University of Kingston
University of Leeds
University of Leicester
University of Lincoln
University of Liverpool
University of Newcastle upon Tyne
University of Northumbria
University of Nottingham
University of Manchester
University of Oxford
University of Plymouth
University of Reading
University of Salford
University of Sheffield
University of Southampton
University of St Andrews
University of Strathclyde
University of Sunderland
University of Surrey
University of Teesside
University of the West of England
University of the West of Scotland
University of Ulster
University of Wales
University Of Westminster
University of Wolverhampton
University of West London
West London College
Workington Sixth Form
Ysgol Emrys ap Iwan, Abergele