Jamie's Junk : Different

More Data Mining in the Cloud?

JamieMac — Tue, 12 May 2009 22:46:00 GMT

Since we released the Tech Preview of the Cloud Data Mining Addins last year, I've heard about and talked with Michael Zeller of Zementis. Zementis has been very active in shaping the PMML (Predictive Modeling Markup Language) standard for representing...(read more)

Those kids won't eat anything!

JamieMac — Fri, 21 Nov 2008 00:15:53 GMT

I did my BI Power Hour demo at PASS 2008 yesterday and it featured my twin boys Bowen and Logan.

Logan (right) has an ASD (Autism Spectrum Disorder) that limits his diet (no milk products, gluten, or soy) and Bowen has some sensory issues, but that doesn't explain their extreme pickiness with food. For example, I made a rice-yogurt-blueberry smoothie for Logan and he just looked at it and said "yucky!". Ugh.

So, I decided to make a worksheet listing foods they these kids will eat and won't eat. I used attributes of Color, Type, and Processed, along with a column indicating whether or not they will actually eat the food. Of course, I had to answer to myself disturbing questions such as "what color are hot dogs?", but I got through it.

Then I used the Prediction Calculator from the new Table Analysis Tools Excel addin for SQL Server 2008. The Prediction Calculator creates a little widget in Excel that allows you to enter in input values and based on your costs. Running the Prediction Calculator is as simple as selecting your table, clicking the Prediction Calculator button on the Table Analyze ribbon, and then choosing the column and value you want to predict.

There's actually a little more work to do after you run the tool, and that is specifying your costs and profits. Your costs are the cost you incur for getting the answer wrong, and a profit is the profit you make when you get the answer right. The grid below is in Prediction Calculator Report that is created after running the tool. In this case, I figured that if I guessed that my kids would eat some food and was wrong, it would cost me the 5 bucks for the food which would be wasted, therefore I set the False Positive Cost to 5. Furthermore, I figured that if I guessed correctly that they wouldn't eat a food, I saved the money and the 5 bucks would still be in my pocket, so I set the True Negative Profit to 5 as well.

Doing so, gave me a profit chart that looks like below - which is a problem. Basically, what a "always rising" chart says is that you should always say "no" to achieve the highest profit - which makes sense since I can only lose money by saying "yes" and only gain money by saying "no". Essentially my laptop analysis tells me that my kids are simply too picky and I should just make them starve! Hah! My laptop apparently has never had kids!

Anyway, I decided that there's some nominal value for my kids eating, so I changed the parameters a bit. I figured that my kids complaining that I didn't get a food that they wanted causes me the psychological cost of 1 dollar (or maybe the real cost of going back to the store of a dollar, however you want to see it), and I set the False Negative Cost to 1. Also I decided the value of my kids not getting a sugar imbalance and (literally) bouncing off the walls is a "peace of mind" profit of a dollar, so I set the True Positive Profit to 1 as well. This gives me a better behaved profit chart with a peak like below.

Once I've set my costs, I can go to the Prediction Calculator sheet that was created and select my inputs and see if my kids will actually eat the food. In this case, "Yellow, unprocessed, grains and nuts" doesn't exceed the threshold of 642, so the answer is no. Yay! I saved 5 bucks because my kids won't eat corn :(.

Of course, this isn't very useful for me in this format - it's not like I'm going to lug my laptop around the grocery store plugging in values for every product I see on the shelf. So to get around this I use the new, experimental Cloud Data Mining Service. The web interface contains many of the Table Analysis Tools, including the Prediction Calculator, and you can access data from CSV files, SQL Data Services, or another way which is not entirely obvious and not documented by simply pasting your data from Excel to the web.

Once you've pasted your data, I run the Prediction Calculator just like I did in Excel. There's one small difference, however, in the result is that I have the HTML fragment for the calculator itself. Therefore I can make my own web site with the calculator embedded inside.

Once I have my website (which in this case is at

http://www.sqlserverdatamining.com/PASS2008BIPowerHourDemo.htm) I can access the Prediction Calculator from any web-enabled device - like my phone, which I can carry around the grocery store and determine that my kids will eat .... brown.....processed.....meat.... oh yay....

Time's running out for your free data mining book!

JamieMac — Thu, 20 Nov 2008 22:52:00 GMT

The survey is now closed. Thank you.

Tomorrow is the last day you can fill out this survey for a chance to win one of ten copies of Data Mining with SQL Server 2008. I used the the Data Exploration tool in the SQL Server 2008 Data Mining Client for Excel, and saw that it takes most people less than 15 minutes to fill it out.

(time to take survey in seconds)

Regarding the book, just today I received my sample copies and I was surprised at how much bigger it is than the 2005 book! It rounds out at 636 pages - I remember last version we were running up against publisher defined page limits and we cut back material to make it fit. This time, we just wrote what needed to be written and the publisher agreed to let us, the authors, make the decisions on how long the book should be. I'm really happy about how the text turned out this time - we still don't have a review on Amazon, so hopefully the public will agree!

Get a FREE AUTOGRAPHED copy of Data Mining with SQL Server 2008

JamieMac — Sat, 08 Nov 2008 05:44:00 GMT

The survey is now closed. Thank you

I checked on Amazon today and Data Mining with SQL Server 2008 is finally in stock!

However, if you are currently using SQL Server Data Mining, before you rush out and buy a copy, please take this opportunity to win a FREE COPY AUTOGRAPHED BY BOGDAN AND ME by filling out a middling short survey on how you use the product. I will be using the survey responses to determine the variety of creative ways customers are implementing SQL Server Data Mining solutions, and, of course, as the information churns in our brains, it means that you could have an impact on what happens in the product down the road. So not only do you have a chance to win a book, your input could change history!

In any case, I'm not giving away just one, but TEN copies of the book to random responders by November 21. So, if you are a SQL Server Data Mining user, please fill out the survey and get your name in the drawing.

Oh, and feel free to go ahead and buy the book, if you win, you'll have an extra loaner copy!! :)

Data Mining Addins on Channel 9

JamieMac — Tue, 03 Jun 2008 11:48:00 GMT

Wee Hyong, a SQL Server MVP in Singapore discusses with Channel 9 how he improved the event experience for the Heroes Community launch using the Data Mining Addins. The first half of the video is the interview and then he presents a demo of the event data using the SQL Server 2008 Data Mining Addins for Office

SQL Data Mining

Data Mining Case Studies

JamieMac — Tue, 03 Jun 2008 11:36:29 GMT

As I'm in the mood for finally responding to frequent requests, here is a list of case studies referring to SQL Server Data Mining on Microsoft.com. Note that these are "marketing" case studies and are notoriously sparse on details, and that's aggravated by marketing focusing more on the relational engine than the data mining engine (in fact many of them refer to data mining without any details, but so is life). In any case, you can use these whenever you get a response such as "but how do we know if SQL Server Data Mining is enterprise ready?" or "But does anyone like us use SQL Server Data Mining?" or similar questions.

Location	Industry	Case Study
Philippines	Media & Entertainment	Wireless Services Firm Doubles Response Rates with SQL Server 2005 Data Mining 10/23/05
Canada	Professional Services	Microsoft Partner Helps Internet-Monitoring Company Crawl 750 Million Pages Monthly 01/17/07
France	Professional Services	E-Commerce Provider Boosts Competitiveness with Microsoft Technology and Partnership 02/04/07
France	Retail	French Retailer Deploys Powerful Solution to Handle 1 Million Orders a Day 09/04/06
Israel	Healthcare	Microsoft SQL Server 2005 Data Mining Helps Clalit Preserve Health and Save Lives 11/03/05
France	Manufacturing	New Business Intelligence Solution Reduces Customer Churn by 25 to 30 Percent 03/16/06
United States	Government	The Illinois Department of Transportation Saves Lives with Microsoft Business Intelligence Solution 21-Apr-08
Germany	Transportation & Logistics	Global Air Freight Company Analyzes Mission Critical Logistics Data with SQL Server 2005 02/01/06
United Kingdom	Media & Entertainment	Scalable Data Warehouse Helps U.K. Lifestyle Group Consolidate Multiple Data Sources in Just Three Months 11/07/06
Italy	Financial Services	New Information System Increases Efficiency and Helps Double Profits at Italian Bank 02/16/07
United Kingdom	Education	Analytical Tools Help Research Group Provide New Insight Into How Darwin Developed His Theory of Evolution 11/07/06
United Kingdom	Education	Research Group Unlocks the Secrets of Darwinian Research with Mapping and Database Software 18-Mar-08
United States	Manufacturing	Xerox Develops Scalable, Hosted Solution to Optimize Global Print Fleet Management 11/11/05
United States	Professional Services	Database Products Help Real Estate Service Evaluate Millions of Homes Daily, Cut Costs 08/24/06
United States	Financial Services	Zurich in North American Anticipates U.S.$25 Million Gain with SQL Server 2005 Solution 01/22/07

FREE Data Mining training by Rafal Lukawiecki

JamieMac — Tue, 03 Jun 2008 11:13:51 GMT

Many people ask me fore pointers to data mining and this fell into my lap over the past few days. I've blogged about Rafal before when he first took up speaking on data mining - now he's created a lot of new material and presented to audiences around the world. Below is excerpted from a mail I received about Rafal's presentations grouped up for training purposes. It's not the deeply specific training that some of you have asked for, but listening to Rafal is always inspirational, so if there are other audiences you need to capture, have them watch these videos!

Some additional info about Rafal:

· Rafal Lukawiecki is consistently one of the top rated speakers at Microsoft Events

· These sessions were presented to 20 Countries across the EMEA region reaching more than 3,000 IT Pros who rated the content at 8.2 out of 9!

In Data Mining and Business Intelligence for Enterprises, Rafal Lukawiecki aims to show IT Professionals how data mining can be used in IT infrastructure to support real business scenarios demystifying the perception that Data Mining is complex, untested or only for specialists. This has become possible since Microsoft has taken the technology to new levels making it accessible to all. In four easy-to-understand yet packed with practical information sessions you will learn about what Data Mining and Business Intelligence can do for you, how to deploy and manage it, how to use it, and how to make it available to other parts of your IT environment.

Visit TechNet Spotlight: www.microsoft.com/technetspotlight

Video on Demand, Video Downloads, PowerPoint Presentations, Audio and more

Data Mining and Business Intelligence for Enterprises

Available Videos

Data Mining and Business Intelligence for Enterprises:

http://www.microsoft.com/emea/spotlight/event.aspx?id=99

Links to Session Videos:

Ø Introduction to Data Mining- Rafal Lukawiecki, Strategic Consultant, Project Botticelli Ltd

Ø Working with Data Mining- Rafal Lukawiecki, Strategic Consultant, Project Botticelli Ltd

Ø Using Data Mining in your IT Systems (Part 1)- Rafal Lukawiecki, Strategic Consultant, Project Botticelli Ltd

Ø Using Data Mining in your IT Systems (Part 2), Rafal Lukawiecki, Strategic Consultant, Project Botticelli Ltd

Screencasts, more screencasts, plus some scripts.

JamieMac — Fri, 14 Mar 2008 11:05:00 GMT

Oddly enough, this week was a banner week for data mining screencast references. I took some internal strategy training, and was asked by several attendees for links to the screencasts that I made (on the download page) and received two (maybe three) independent requests for the scripts for those screencasts.

Additionally I was in an e-mail conversation with Professor Emeritus Bill Burrows from University of Washington and discovered coincidentally that he has created his own excellent set of screencasts for the addins! Wacky! I must say that his voice and style is much more pleasant than my own, plus it's not delivered from the perspective of the development team, so you should check them out or send colleagues that way to get introduced to the topic.

And, if you're interested, the scripts from my screencasts are attached - plus a bonus script that I never actually got around to recording!

"Supercrunchers" and Microsoft Data Mining

JamieMac — Wed, 12 Mar 2008 09:22:00 GMT

Ian Ayres, Yale Law school professor and author of SuperCrunchers, talks about data driven decisions and the Data Mining Addins in this Gartner podcast. He also mentions some other company, but we won't talk about them :) Give it a listen!

A data mining survey

JamieMac — Tue, 04 Mar 2008 21:38:00 GMT

Karl Rexer of Rexer analytics has taken up a hobby of implementing a somewhat unscientific survey of data mining users to determine trends in the industry - see what kinds of software people are using - all that kind of stuff. In Karl's own words:

"We would greatly appreciate it if you would:

1) Participate in this survey, and

2) Tell other data miners about the survey (forward this email to them)."

Since I don't have everyone's email address, the next best thing is posting on my blog. You can participate in the survey here and use access code GSU74. The survet is anonymous, but there's a place at the end to leave your e-mail if you want to get a copy of the results.

If you have any questions about the survey you can ask Karl himself at krexer@RexerAnalytics.com.

Not the kind of surprise I was expecting....

JamieMac — Thu, 07 Jul 2005 09:27:00 GMT

I'm in Amsterdam now for TechEd EMEA. Everytime you visit a different country there are lots of little surprises - even if you 've been there before. Take for example yesterday, when I got off the plane. As any one would expect, the first thing you do after a 10 hour flight that ends with a 30 minute taxi is head for the restroom, so I did. As I entered the main restroom in the shopping area in Shiphol airport, there was a large picture of their air traffic control tower with the unnerving caption "Surprising Toilets!" You know, after that flight - in an economy seat - the last thing in the world I wanted a surprise from was my toilet. I almost left right there - I could hold it for another hour until I got to the hotel - I really could.

Anyway, it turns out the surprise was all of the "surprising" information and paintings about the airport and Amsterdam that you would find on the walls of the restroom. The toilets, although a bit different than their North American counterparts, were thankfully unsurprising. If you really want to learn more about the airport, go here - they sell great cheese there.

Oh, and I didn't have to wait.....