Jamie's Junk : Addins

New Cloud Data Mining Addin Usability Feature (well kind of)

JamieMac — Fri, 08 May 2009 02:11:00 GMT

Bogdan did an awesome job putting together the modified Data Mining Addin for the cloud. We saw quite a few downloads of his new addin, but surprisingly and unfortunately, didn't see a lot of usage! Strange? Maybe. Bogdan thought about the issue for a...(read more)

Those kids won't eat anything!

JamieMac — Fri, 21 Nov 2008 00:15:53 GMT

I did my BI Power Hour demo at PASS 2008 yesterday and it featured my twin boys Bowen and Logan.

Logan (right) has an ASD (Autism Spectrum Disorder) that limits his diet (no milk products, gluten, or soy) and Bowen has some sensory issues, but that doesn't explain their extreme pickiness with food. For example, I made a rice-yogurt-blueberry smoothie for Logan and he just looked at it and said "yucky!". Ugh.

So, I decided to make a worksheet listing foods they these kids will eat and won't eat. I used attributes of Color, Type, and Processed, along with a column indicating whether or not they will actually eat the food. Of course, I had to answer to myself disturbing questions such as "what color are hot dogs?", but I got through it.

Then I used the Prediction Calculator from the new Table Analysis Tools Excel addin for SQL Server 2008. The Prediction Calculator creates a little widget in Excel that allows you to enter in input values and based on your costs. Running the Prediction Calculator is as simple as selecting your table, clicking the Prediction Calculator button on the Table Analyze ribbon, and then choosing the column and value you want to predict.

There's actually a little more work to do after you run the tool, and that is specifying your costs and profits. Your costs are the cost you incur for getting the answer wrong, and a profit is the profit you make when you get the answer right. The grid below is in Prediction Calculator Report that is created after running the tool. In this case, I figured that if I guessed that my kids would eat some food and was wrong, it would cost me the 5 bucks for the food which would be wasted, therefore I set the False Positive Cost to 5. Furthermore, I figured that if I guessed correctly that they wouldn't eat a food, I saved the money and the 5 bucks would still be in my pocket, so I set the True Negative Profit to 5 as well.

Doing so, gave me a profit chart that looks like below - which is a problem. Basically, what a "always rising" chart says is that you should always say "no" to achieve the highest profit - which makes sense since I can only lose money by saying "yes" and only gain money by saying "no". Essentially my laptop analysis tells me that my kids are simply too picky and I should just make them starve! Hah! My laptop apparently has never had kids!

Anyway, I decided that there's some nominal value for my kids eating, so I changed the parameters a bit. I figured that my kids complaining that I didn't get a food that they wanted causes me the psychological cost of 1 dollar (or maybe the real cost of going back to the store of a dollar, however you want to see it), and I set the False Negative Cost to 1. Also I decided the value of my kids not getting a sugar imbalance and (literally) bouncing off the walls is a "peace of mind" profit of a dollar, so I set the True Positive Profit to 1 as well. This gives me a better behaved profit chart with a peak like below.

Once I've set my costs, I can go to the Prediction Calculator sheet that was created and select my inputs and see if my kids will actually eat the food. In this case, "Yellow, unprocessed, grains and nuts" doesn't exceed the threshold of 642, so the answer is no. Yay! I saved 5 bucks because my kids won't eat corn :(.

Of course, this isn't very useful for me in this format - it's not like I'm going to lug my laptop around the grocery store plugging in values for every product I see on the shelf. So to get around this I use the new, experimental Cloud Data Mining Service. The web interface contains many of the Table Analysis Tools, including the Prediction Calculator, and you can access data from CSV files, SQL Data Services, or another way which is not entirely obvious and not documented by simply pasting your data from Excel to the web.

Once you've pasted your data, I run the Prediction Calculator just like I did in Excel. There's one small difference, however, in the result is that I have the HTML fragment for the calculator itself. Therefore I can make my own web site with the calculator embedded inside.

Once I have my website (which in this case is at

http://www.sqlserverdatamining.com/PASS2008BIPowerHourDemo.htm) I can access the Prediction Calculator from any web-enabled device - like my phone, which I can carry around the grocery store and determine that my kids will eat .... brown.....processed.....meat.... oh yay....

Time's running out for your free data mining book!

JamieMac — Thu, 20 Nov 2008 22:52:00 GMT

The survey is now closed. Thank you.

Tomorrow is the last day you can fill out this survey for a chance to win one of ten copies of Data Mining with SQL Server 2008. I used the the Data Exploration tool in the SQL Server 2008 Data Mining Client for Excel, and saw that it takes most people less than 15 minutes to fill it out.

(time to take survey in seconds)

Regarding the book, just today I received my sample copies and I was surprised at how much bigger it is than the 2005 book! It rounds out at 636 pages - I remember last version we were running up against publisher defined page limits and we cut back material to make it fit. This time, we just wrote what needed to be written and the publisher agreed to let us, the authors, make the decisions on how long the book should be. I'm really happy about how the text turned out this time - we still don't have a review on Amazon, so hopefully the public will agree!

Data Mining in the Cloud

JamieMac — Mon, 25 Aug 2008 23:27:00 GMT

In case the title of my last post wasn't provocative enough, I'll post it again! Here's an excerpt "access the Table Analysis Tools for Excel 2007 by connecting to our hosted data mining service"...

If you don't want to read the long-winded post and just want to get the goods, go here

KDD 2008 and Incredibly Awesome SQL 2008 Data Mining Demos

JamieMac — Mon, 25 Aug 2008 21:48:00 GMT

Bogdan and Raman are currently outside of Las Vegas at the KDD 2008 (Knowledge Discovery and Data Mining) conference. At the conference they are showing all the goodness that is SQL Server 2008 Data Mining, but also a "secret project" that Bogdan has been working on all summer - Data Mining in the Cloud. That's right - access to the awesome data mining functionality of SQL Server 2008 with no local Analysis Services server.

Currently the technology demonstration allows you to access the Table Analysis Tools for Excel 2007 by connecting to our hosted data mining service, meaning you can use the Table Analysis Tools anywhere at anytime without connectivity to your local IT infrastructure. Additionally there is a web interface that allows you to upload a limited amount of data and play with the tools without even having Excel! Currently only a few of the tools are implemented in the web interface. If for any reason you haven't had a chance to grab those amazing Table Analysis Tools and see what SQL Server Data Mining can do for you - run, don't walk, over to http://www.sqlserverdatamining.com/cloud/ and try them out right now!

Not to limit ourselves to revolutionary cloud data mining technology, the data mining team here in SQL Server also created another demo application that clearly shows the amazing advances in our Time Series algorithm. Shuvro and Tatyana put together an incredible application that front ends the time series capabilities of SQL Server 2008. In this application, you not only get a simple interface for creating time series models resulting in a chart showing the historical and forecasted data, but you can also hover over any forecasted points and it will highlight all the data points in that series and other series that have an impact on the prediction! Even more, you can click and drag forecasted values to see how they change future predictions! Even more, the application will show you all the DMX statements that it uses to create these models and generate these forecasts. This demo is truly an incredible piece of work. You can find out more about the demo and how to download it at http://www.sqlserverdatamining.com/ssdm/Home/TipsTricks/tabid/61/Default.aspx?id=383.

Screencasts, more screencasts, plus some scripts.

JamieMac — Fri, 14 Mar 2008 11:05:00 GMT

Oddly enough, this week was a banner week for data mining screencast references. I took some internal strategy training, and was asked by several attendees for links to the screencasts that I made (on the download page) and received two (maybe three) independent requests for the scripts for those screencasts.

Additionally I was in an e-mail conversation with Professor Emeritus Bill Burrows from University of Washington and discovered coincidentally that he has created his own excellent set of screencasts for the addins! Wacky! I must say that his voice and style is much more pleasant than my own, plus it's not delivered from the perspective of the development team, so you should check them out or send colleagues that way to get introduced to the topic.

And, if you're interested, the scripts from my screencasts are attached - plus a bonus script that I never actually got around to recording!

"Supercrunchers" and Microsoft Data Mining

JamieMac — Wed, 12 Mar 2008 09:22:00 GMT

Ian Ayres, Yale Law school professor and author of SuperCrunchers, talks about data driven decisions and the Data Mining Addins in this Gartner podcast. He also mentions some other company, but we won't talk about them :) Give it a listen!