John C. Hancock's blog

Analyzing data gathered using Groove forms

johnchanc — Sun, 15 Apr 2007 22:22:00 GMT

Recently I had to come up with a solution for analyzing data gathered from a consumer research survey. Imagine that a company wants to send researchers out to shopping malls and interview people to find out how much they know about a new product. In the old days, this process would involve paper questionnaires that they would fill out for every person they interviewed, and then a long data capture process at the very end. The company wants much faster access to analyze the results as they are gathered, so they arm the researchers with Tablet PCs. Now the challenge is to find an application to get the data from the field researchers into the cube for analysis.

This could involve building a custom client application, but that would involve writing custom code to manage local data storage on the tablets and synch it up with a central database – too much effort. A creative solution uses Groove forms to gather the data and rely on the built in secure synchronization so that researchers can synch up even from coffee shops with public Internet connections. MSDN Magazine has published my article on how to build this solution, and how to use the Groove web services API to get the data from the Groove workspace into the database – see http://msdn.microsoft.com/msdnmag/issues/07/05/Groove/default.aspx

Social relationships and data mining

johnchanc — Sun, 25 Feb 2007 15:12:00 GMT

Studying the social relationships between people is an area that is getting an increasing amount of attention. Sociologists often turn to concepts from the field of network analysis to help them understand and quantify these ideas, such as how central a particular person is within their social network, or which subgroups of people are closely related (see Social network).

These techniques involve applying complex algorithms to large sets of data. What we need is a platform that can store the data, give us a framework to implement the required logic, and a flexible way of presenting the data to users. SQL 2005 has all the necessary components, so I've attached a paper that describes how to start tackling this area. The paper walks you through creating the data mining model using the Microsoft Association Rules algorithm, developing the stored procedures using C#, and then reporting on the data mining model.

Practical Business Intelligence with SQL Server 2005

johnchanc — Sun, 13 Aug 2006 18:44:00 GMT

Our new book Practical Business Intelligence with SQL Server 2005 will be available from September, so this blog is a place for me to post any helpful information that didn't make it into the book. So, if you send me questions here or through my web site at http://www.johnchancock.net, I'll post more detailed explanations and real world tips on this blog.