The goal of this site is to put relevant and applicable tools and information at the fingertips
With this blog we want to inform you on our latest initiatives.
Enjoy reading and stay tuned!
Microsoft Press have a number of FREE Ebooks now available covering topics from SQL Server 2012, Office 2010 and Windows Phone, there is something here for everyone.
Many of our free ebooks are offered in three formats - PDF, Mobi (Kindle) and epub. Hope you find them useful!
Last week I attended The 2012 Eduserv Symposium. The event was focused on 'Big Data' and discussed whether Big Data represents a challenge or an opportunity and how we can best make use of it. This year's event was held at the Royal College of Physicians.
The key message from the introduction was the importance of uncoupling the issues of Big Data from the industry hype. Throughout the event the Gartner 3 V’s (velocity, volume and variety of data ) were discussed in depth.
Volume as a defining attribute of Big Data
Whilst it is fairly obvious that data volume is the primary attribute of big data, people often ask for a definitive quantity in GB, TB, PB etc. that would qualify as big data.
Whilst simplest answer is to give a data volume, for example 50TBytes which today present represents a reasonably large and expensive dataware house –this answer of course changes as the technology changes over time due to Moore’s Law.
But it’s also worth thinking about what it is you are looking as data, for instance for a large library of photographs the actual data contained in all the images themselves is very large for instance a RAW File from a Nikon D300 is about 25MB, so library with 2 million such images would be about 50TBytes , but the meta data describing those images isn’t that large perhaps 2GBytes.
So to someone actually searching the images for content, e.g. using facial recognition find all the photos of Adele, then that is a big data problem, but if the photos have already been labelled and tagged as being of Adele then that isn’t really a big data challenge as you are only search the Meta data.
Data Feed Velocity as a defining attribute of Big Data
Big data can be described by its velocity or speed. Or you may prefer to think of it as the frequency of data generation or frequency of data delivery. For example, think of the stream of data coming off of any kind of sensor, say thermometers sensing temperature, microphones listening for movement in a secure area, or video cameras scanning for a specific face in a crowd. This isn’t new; many firms have been collecting click stream data off of Web sites for years, using streaming data to make purchase recommendations to Web visitors. With sensor and Web data flying at you relentlessly in real time, data volumes get big in a hurry.
Technologies such as Stream Insight are well positioned for certain types of streaming data, whilst other applications may need specialist development or tools.
Data Feed Variety.
A proliferation of data types from social, machine to machine, and mobile sources add new data types to traditional transactional data. Data no longer fits into neat, easy to consume structures. New types include content, geo-spatial, hardware data points, location based, log data, machine data, metrics, mobile, physical data points, process, RFID’s, search, sentiment, streaming data, social, text, and web. The addition of unstructured data such as speech, text, and language increasingly complicate the ability to categorize data. Some technologies that deal with unstructured data include data mining, text analytics, and noisy text analytics.
A key message was Big Data and the role of the Data Scientist, was not limited to the computer Scientist of the future as Big Data is of use to all disciplines.
Please find below a quick overview deck of how we see the Big Data opportunity at Microsoft.
A interesting case study is Klout Data Services Firm Uses Microsoft BI and Hadoop to Boost Insight into Big Data
Klout wanted to give consumers, brands, and partners faster, more detailed insight into hundreds of terabytes of social-network data. It also wanted to boost efficiency. To do so, Klout deployed a business intelligence solution based on Microsoft SQL Server 2012 Enterprise and Apache Hadoop. As a result, Klout processes data queries in near real time, minimizes costs, boosts efficiency, increases insight, and facilitates innovation.
Klout helps clients make sense of the hundreds of terabytes of data generated each day by more than 1 billion signals on 15 leading social networks including Facebook and LinkedIn. The data that Klout analyzes is generated by the more than 100 million people who are indexed by the firm. This includes Klout members and the people that they interact with on social sites. Individuals join Klout to understand their influence on the web, which is rated on a scale from 1 to 100. They also sign up to participate in campaigns where they can receive gifts and free services. More than 3,500 data partners also join Klout to better understand consumers and network trends including changes in demand and how peoples’ influence might affect word-of-mouth advertising.
To deliver the level of insight that customers seek and yet meet the budget constraints of a startup firm, Klout maintained a custom infrastructure based on the open-source Apache Hadoop framework, which provides distributed processing of large data sets. The solution included a separate silo for the data from each social network. To manage queries, Klout used custom web services, each with distinct business logic, to extract data from the silos and deliver it as a data mashup.
Maintaining Hadoop and the custom web services to support business intelligence (BI) was complex and time-consuming for the team. The solution also hindered data insight. For example, accessing detailed information from Hadoop required extra development, and so mashups often lacked the level of detail that users sought. In addition, people often waited minutes, or sometimes hours, for queries to process, and they could only obtain information based on predetermined templates.
Klout wanted to update its infrastructure to speed efficiency and support custom BI. Engineers sought technologies that could deliver mission-critical availability and still scale to meet big-data growth and performance requirements.
In 2011, Klout decided to implement a BI solution based on Microsoft SQL Server 2012 Enterprise data management software and the open-source Hive data warehouse system. Based on employees’ previous experience with the Microsoft BI platform, Klout also knew that SQL Server offers excellent compatibility with third-party software and it can handle the data scale and query performance needed to manage big-data sets.
In August 2011, engineers implemented a data warehouse with Hive, which consolidates data from all of the network silos hosted by Hadoop. In addition, Klout deployed SQL Server 2012 on a system that runs the Windows Server 2008 R2 Enterprise operating system to take advantage of Microsoft SQL Server 2012 Analysis Services. Engineers use it to manage all business logic required to facilitate multidimensional online analytical processing (MOLAP). Data is stored in multidimensional cubes, which helps preserve detail and speed analysis. To provide high availability, Klout replicates the database to a secondary system using SQL Server 2012 AlwaysOn.
At the time that Klout was initially deploying its solution, SQL Server 2012 and Hive could not communicate directly. To work around this issue, engineers set up a temporary relational database that runs MySQL 5.5 software. It includes data from the previous 30 days and serves as a staging area for data exchange and analysis. Klout engineers are currently working to implement the new open database connectivity driver in SQL Server 2012 to directly join Hive with SQL Server 2012 Analysis Services. In addition, to enhance insight Klout plans to work with Microsoft to incorporate other Microsoft BI tools into its solution, such as Microsoft SQL Server Power Pivot for Microsoft Excel.
With its new solution, Klout expects to boost efficiency, reduce expenses, expand insight, and support innovation.
Speeds Efficiency and Cuts Costs By taking advantage of the Microsoft platform for BI, users will be able to get the data they seek in near real time.
Klout is implementing the flexible and scalable infrastructure it needs to continue to push the limits of data analysis.
This case study is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.
I had a interesting discussion this morning with a business school looking at introducing Mobile App development one of the key things within their assessment is the proof of prototypes and wireframes we got onto discussing the opportunity and advantages of using Microsoft Expression, SketchFlow.
SketchFlow addresses three pain points in software sketching and prototyping:
It makes it easier to experiment with ideas for dynamic user interaction. It facilitates communication of design ideas and intent between designers and other stakeholders. It makes it easier to gather, evaluate and use stakeholder feedback on design ideas.
The mission of the Expression team is to help designers to create great user experiences. We believe that this mission needs to begin with the early exploration phases of design. SketchFlow,
SketchFlow is a set of tools for Expression Blend to create sketches and prototypes of interactive content and applications, giving Blend a new role in pre-production and design phases of the development process.
SketchFlow is informal and quick, enabling you to sketch out plentiful ideas for dynamic interactions in a cost effective manner. SketchFlow also supports the evolution of your rough sketches into living and breathing prototypes that can be as real as you need them to be.
SketchFlow is part of Expression Blend so FREE from www.dreamspark.com
Throughout the software industry, we historically have not spent much effort on the earlier stages of design. This lack of upfront investment in design often leads to increased cost due to a need for expensive rework at a later stage and to products that may not satisfy our customers.
Great design for software needs a great design process, and ways to make the exploration, communication and evaluation of interaction and design fast and effective are an important part of great process.
Sketching, and, at a slightly later stage of the design process, prototyping, are wonderful techniques to explore a multitude of ideas quickly, without excessive investment and emotional attachment. Software interaction is highly dynamic, and over the last years we have consistently have pushed more in the direction of rich, dynamic visuals and interaction methods. Expectations have risen greatly. However, it is far from easy to create meaningful design studies and sketches of dynamic interactivity entirely with traditional tools. There are many reasons for that, including the fact that most design tools have been created as production tools, focused on the creation of final production assets, where precision, quality and finish matter. Another reason is that most tools still treat UI as something that is just a small derivation from an otherwise static comp.
SketchFlow provides a way to showcase your prototype to others using the SketchFlow Player. The SketchFlow Player allows you to explore the prototype from the first moment on, even while it consists of nothing but a few rough sketches. The player lets you navigate your prototype, run animations that illustrate how your prototype might work, or switch into different states of your UI, all without wiring up actual UI elements. This lowers the cost of evaluating ideas in early stages before much expense has been incurred.
But showing a prototype to others is only part of the review process, so the SketchFlow Player provides tools to collect feedback from reviewers. Reviewers can provide feedback either as text, like the comments in the Feedback tab in the screenshot below, or as ink, like the red arrow in the screenshot below. Then you can incorporate this feedback into the prototype using the Feedback panel in Expression Blend, allowing you to iterate on your design using suggestions from your team.
Tutorials on Sketchflow http://www.microsoft.com/design/toolbox/school/tutorials.aspx
For more information, watch a video on Sketchflow
Touchdevelop is an ideal teaching tools for schools, touch develop allows you to program a Windows Phone directly. You do not need a separate PC. Scripts can perform various tasks similar to regular apps. Any TouchDevelop user can install, run, edit, and publish scripts. You can share your scripts with other people by publishing them to the TouchDevelop script bazaar, or by submitting them as an app to the Windows Phone Marketplace.
How its being used for Education
Ninety eighth-grade students at Mill Creek Middle School in Kent, Washington, learned how to write smartphone programs with TouchDevelop, a Microsoft Research programming environment that enables users to create and test programs on Windows Phone. Most of the students had no prior experience with programming, but after tinkering with TouchDevelop for an hour, students came up with a variety of innovative and fun programs that they proudly demonstrated to their peers.http://research.microsoft.com/apps/video/default.aspx?id=157112&s=0&e=150
Teach TouchDevelop Resources
Resources and Book on TouchDevelopment
Last week Geoff and I attended a number of University graduation showcase events across the country, firstly thank you for the invites and its great to see the outcomes of students projects and the use of Microsoft Technology. A number of students and academics also asked me some interesting questions in relation to the development of Windows 8 and specifically how to include some aspects of Windows 8 Metro into the curriculum. One of the easiest ways of implementing Windows 8 into your teaching is to simply add in a small work item of converting an existing app to Metro as a learning exercise, this may also be great a MSc or MPhil project for an existing application or service.
There are a lot of useful resources out there on porting existing applications to Metro apps, a successfully ported app will of course run beautifully on Windows 8 and this may be a great way of introducing/exposing students to the Metro UI and design principles:
Porting a website
Porting a Windows Phone app
Hope the above give you some food for thought of how you may start thinking about the opportunities of Windows 8 and the Metro UI.
We have been working closely with Steven Johnston and András Sóbester members of the Astra Research team from Southampton University over the past year, since last April when the team ASTRA team launched the first Windows Phone device with huge success.
Over the past year, the ASTRA Atmospheric Science Through Robotic Aircraft research team, are now using a number of Microsoft technologies including, Microsoft .Net Gadgeteer, Windows Embedded, Windows Azure and Windows Phone.
The Astra team recently launched a new project which included Windows Azure and a Nokia Lumia 800 device, the device was mounted onto a weather balloon to carry out some experiments whilst in flight. The balloon and device reached a height of 105,000 feet, with a temperatures of -61C and maintained data connectivity at 8km above Earth with the Windows Azure cloud completing data capture and analysis of the flight, the results will be used to help create balloon flight trajectory prediction software that can then be used to guide other missions, like weather balloons.
The experiments completed by ASTRA (Atmospheric Science Through Robotic Aircraft) project investigates new technologies for making low cost observations of the parameters of the atmosphere and their latest experiment, further details of the project can be found at Nokia Conversations blog,
The balloon and payload travelled for 2 hours and 22 minutes on a journey over South Wales en route to Cornwall. It took some quite pretty pictures along the way too.
A small selection of the 2GB image set is available to download.
.NET Gadgeteer can be used in schools to help students make gadgets and learn C# programming along the way. These lesson plans give teachers all the material needed to run 8 1-hour sessions using .NET Gadgeteer. You will also need the hardware, details of which are given at http://netmf.com/gadgeteer.
The material consists of 8 sessions relating to 3 projects:
The aim of each session is:
Back in January this year, we announced Visual Studio Achievements for Developers – a fun way bringing Xbox LIVE-like points to everyday programming tasks. The fascinating program makes sure your students hard day’s work doesn’t go unnoticed and after over 80,000 downloads since the initial release, As your aware from my previous blog we have now extended the game to enroll developers working on Windows Azure.
The Visual Studio Achievements Extension has now been updated to include fifteen new achievements, all focused on exercising features of Windows Azure. Using the extension, various achievements are unlocked based on your activity. When you unlock an achievement, Visual Studio lets you know visually with a pop-up. In addition, your Channel 9 profile is updated with any achievements you earn. So, head over to Channel9, sign up for an account and download the plugin.
There are 15 new Azure achievements, such as publishing to Windows Azure from Visual Studio (Heading into the Cloud), using page blobs (Attack of the Blob), using SQL Azure (Database Darling) and configuring start up tasks (It’s My Party). Two of the achievements – Phone in the Cloud and Game in the Cloud – require use of Windows Azure toolkits.
You might also want to check Visual Studio Achievements for Windows Phone – a Windows Phone app that allows you to view Visual Studio Achievements on the go.
The Windows Phone application developed by Den Delimarsky allows viewing Visual Studio Achievements on the go. The source code for the application is available on Codeplex. You can choose to pull the source and compile a XAP yourself, and then side-load it on your phone.
Interestingly, Den has authored a great series of blog posts about this project on DZone with details on each of the different areas, passing on some lessons learned in building this app.
Visual Studio Achievements for Windows Phone – Developing the core Visual Studio Achievements for Windows Phone – What it looks like Visual Studio Achievements for Windows Phone – User details & official project page Visual Studio Achievements for Windows Phone – Storage Visual Studio Achievements for Windows Phone – Basic User Comparison Visual Studio Achievements for Windows Phone – Final notes before the beta
You can also Download the app directly from the Windows Phone MarketPlace the app is called VSA.
Over the past few weeks we have been travelling across the country to run Windows 8 events.
So Get started building Windows 8 Apps
1. Apply to join our UK Windows 8 for Application Developer LinkedIn Group.
2. We will reply asking you for some details of what it is you are building and then admit you into the group.
3. You build a great app following the guidance that we have online and which meets the Store certification requirements.
4. We will use the LinkedIn Group to invite you to an Application Excellence Lab at Microsoft in Reading where you’ll work with an engineer to ensure certification requirements are being met and that the app is providing a great experience to the user.
5. You leave the lab with a registration code that lets you begin the app submission process at the Windows Store
Getting Started building an App
Here are some recommended links to help with planning & designing Metro style apps…
· Watch 8 traits of great Metro style apps session from Build, to understand the key platform features and Metro design principles
· Read Planning Metro style apps, which walks you through the planning process to ensure your app has clear goals and a compelling “great at” statement
· If you have an existing site/app, read these design case studies to explore how to reimagine your web site or iPad app using Metro design principles
· Point your designers at the UX guidelines, checklists, PhotoShop templates etc on http://design.windows.com.
Key topics to help you design your first app are:
Grid system and app silhouette
Text and typography
Additionally there is lots of content, tools, samples and downloads on http://dev.windows.com. You can download the app samples and use the forums to ask questions.
Here are set of great Bookmarks in alphabetical order
Microsoft and partners NetApp and Citrix announced upcoming native support for FreeBSD support on Windows Server Hyper-V.
This move continues Microsoft’s commitment to extend support across platforms to the Windows Server Hyper-V solution, making it easier for more customers to realize the benefits of server virtualization and more easily adopt cloud computing.
This will allow FreeBSD to run as a first-class guest on Windows Server Hyper-V. The drivers and associated source code will be released early this summer under the BSD license, and will initially work with FreeBSD 8.2 and 8.3 on Windows Server 2008 R2.
You can read more about this on the Openness blog.
You can read the full interview with Joe CaraDonna, the Technical Director of Core Operating Systems at NetApp, here.