Jamie's Junk : Talks

Those kids won't eat anything!

JamieMac — Fri, 21 Nov 2008 00:15:53 GMT

I did my BI Power Hour demo at PASS 2008 yesterday and it featured my twin boys Bowen and Logan.

Logan (right) has an ASD (Autism Spectrum Disorder) that limits his diet (no milk products, gluten, or soy) and Bowen has some sensory issues, but that doesn't explain their extreme pickiness with food. For example, I made a rice-yogurt-blueberry smoothie for Logan and he just looked at it and said "yucky!". Ugh.

So, I decided to make a worksheet listing foods they these kids will eat and won't eat. I used attributes of Color, Type, and Processed, along with a column indicating whether or not they will actually eat the food. Of course, I had to answer to myself disturbing questions such as "what color are hot dogs?", but I got through it.

Then I used the Prediction Calculator from the new Table Analysis Tools Excel addin for SQL Server 2008. The Prediction Calculator creates a little widget in Excel that allows you to enter in input values and based on your costs. Running the Prediction Calculator is as simple as selecting your table, clicking the Prediction Calculator button on the Table Analyze ribbon, and then choosing the column and value you want to predict.

There's actually a little more work to do after you run the tool, and that is specifying your costs and profits. Your costs are the cost you incur for getting the answer wrong, and a profit is the profit you make when you get the answer right. The grid below is in Prediction Calculator Report that is created after running the tool. In this case, I figured that if I guessed that my kids would eat some food and was wrong, it would cost me the 5 bucks for the food which would be wasted, therefore I set the False Positive Cost to 5. Furthermore, I figured that if I guessed correctly that they wouldn't eat a food, I saved the money and the 5 bucks would still be in my pocket, so I set the True Negative Profit to 5 as well.

Doing so, gave me a profit chart that looks like below - which is a problem. Basically, what a "always rising" chart says is that you should always say "no" to achieve the highest profit - which makes sense since I can only lose money by saying "yes" and only gain money by saying "no". Essentially my laptop analysis tells me that my kids are simply too picky and I should just make them starve! Hah! My laptop apparently has never had kids!

Anyway, I decided that there's some nominal value for my kids eating, so I changed the parameters a bit. I figured that my kids complaining that I didn't get a food that they wanted causes me the psychological cost of 1 dollar (or maybe the real cost of going back to the store of a dollar, however you want to see it), and I set the False Negative Cost to 1. Also I decided the value of my kids not getting a sugar imbalance and (literally) bouncing off the walls is a "peace of mind" profit of a dollar, so I set the True Positive Profit to 1 as well. This gives me a better behaved profit chart with a peak like below.

Once I've set my costs, I can go to the Prediction Calculator sheet that was created and select my inputs and see if my kids will actually eat the food. In this case, "Yellow, unprocessed, grains and nuts" doesn't exceed the threshold of 642, so the answer is no. Yay! I saved 5 bucks because my kids won't eat corn :(.

Of course, this isn't very useful for me in this format - it's not like I'm going to lug my laptop around the grocery store plugging in values for every product I see on the shelf. So to get around this I use the new, experimental Cloud Data Mining Service. The web interface contains many of the Table Analysis Tools, including the Prediction Calculator, and you can access data from CSV files, SQL Data Services, or another way which is not entirely obvious and not documented by simply pasting your data from Excel to the web.

Once you've pasted your data, I run the Prediction Calculator just like I did in Excel. There's one small difference, however, in the result is that I have the HTML fragment for the calculator itself. Therefore I can make my own web site with the calculator embedded inside.

Once I have my website (which in this case is at

http://www.sqlserverdatamining.com/PASS2008BIPowerHourDemo.htm) I can access the Prediction Calculator from any web-enabled device - like my phone, which I can carry around the grocery store and determine that my kids will eat .... brown.....processed.....meat.... oh yay....

Silverlight Viewers, Data Mining Road Show, and TechEd

JamieMac — Fri, 22 Feb 2008 20:27:00 GMT

A enthusiast in China took a suggestion all the way and has implemented at least a decision tree viewer in Silverlight - I don't read Chinese, but you can see it here

Rafal Lukawiecki - consistantly one of the top TechEd Europe speakers - has been traveling around doing an extensive data mining roadshow. Here is a review of his show in Israel, but you don't have to hear just the critics - you can also hear Rafal's point of view in this interview or even see last year's TechEd presentation. I was there and was amazed - I had provided a lot of the material to Rafal and presented it for a "good" response, but hearing Rafal deliver it was inspirational!

You don't have to stop there, in April, Rafal's show is going to London, and rumor tells me you'll have the chance to see him at TechEd in Orlando this year in both the developer week and the IT week speaking both on Data Mining and Security.

PASS Past and Present

JamieMac — Wed, 13 Sep 2006 09:46:00 GMT

If you're interested in a broad overview of data mining functionality in SQL Server 2005, check out my PASS 2005 presentation on IT's Showtime with the compelling and exciting name "Data Mining with SQL Server 2005." Interestingly enough, I remember getting feedback that some people thought the presentation would be about things like implementing algorithms inside SQL stored procedures and were "pleasantly surprised" to find out that all that data mining functionality was just built in.

Now, if you want to see something completely different - something that very few have had the chance to catch a glimpse, come to PASS 2006 in Seattle. I will be showing the new Data Mining Addins for Office 2007 that I've posted a little tiny bit about. If you come, you'll get to see it all! (and if we happen to be distributing CTP 2 of SQL Server 2005 SP2, you could even take it home - but no promises :) ) If you see this and don't think these addins aren't the coolest thing since sliced bread, than you must not think sliced bread is all that cool....

Commentary on one of my TechEd talks

JamieMac — Thu, 22 Jun 2006 10:24:00 GMT

In this post, Teo Lachev writes about his impressions of Tech Ed 2006 including my Reporting and Analysis Services talk, among others. I'll post a bit more about my other talks (with screenshots!) later on.

Boston next week

JamieMac — Thu, 08 Jun 2006 10:49:00 GMT

Next week I'll be in Boston at TechEd, so if you're there, come by and say Hi! On Wednesday, I'll be doing a talk on reporting and Analysis Services. Later that day, I'll do a chalk talk on the subtleties in the CREATE MINING MODEL statement. (Hopefully they'll find me a projector for this one so you won't have to suffer my handwriting....)

On Thursday, I'll be showing the "world premier" of our Data Mining Addins for Office 2007 to be released later this year. These addins are composed of Table Analytic tools for Excel 2007, Data Mining Client tools for Excel 2007, and Data Mining Templates for Visio 2007. Want to know more - see me at Teched!

Slides from my 2 day training course in Copenhagen

JamieMac — Thu, 27 Apr 2006 23:02:00 GMT

Last year on short notice I pulled together a two day data mining training session for about 30 people in Copenhagen. I've been getting requests for slides "from my book" (and from other places), and since I never had any, I felt these would do. You can get them from this link at sqlserverdatamining.com (free logon required). Their kind of sparse, but they at least provide somewhat of an inventory of the functionality in the product.

BI Training in LA towards the end of June

JamieMac — Mon, 24 Apr 2006 23:43:00 GMT

MSEvents is hosting a hands-on lab in June where you can learn about BI for IT professionals. From this training (according to their site) you will learn how to create

An Analysis Services cube

A SQL Server 2005 Integration Services package

SQL Server Reporting Services Report

Decision Tree and Naïve Bayes Data Mining Models

A report user the new ad-hoc report tool, Report Builder

On PASS, PDC and new Downloadable Demos

JamieMac — Sat, 01 Oct 2005 00:24:00 GMT

I've just finished uploading some new demos that I've shown at PDC and PASS. Both of these demos show how to use VBA to integrate data mining functionality into Excel. Their pretty cool - you should check them out. Bogdan wrote the DMPredict function and I wrote the clustering sample.

As I'm writing this PASS is finishing up - I actually should be on the way to the airport, but I still have a couple minutes....

PASS was excellent as usual and I'm fired up about the response to the data mining tools in SQL 2005 - there's a whole new audience out there that has advanced research technology available to them for the first time. It will be exciting to see how these communities can push and pull this tech to do things that no one has ever imagined. A PDC'er that commented to be two weeks back in LA, called the advances in Data Mining and OLAP in SQL Server 2005 Analysis Services a "seismic" event. It's good to be in AS!

Go ahead - if you've downloaded the September CTP, check out those demos! Gotta catch my flight!

Me on "TV"

JamieMac — Fri, 12 Aug 2005 10:46:00 GMT

I just watched the MSDN TV episode I did on Data Mining and I'm less embarassed than I thought I would be... [:D]

If you've watched any of the data mining webcasts it's old hat, but if you haven't it's a good teaser to get you started - after which you should watch the Intro to DM Webcast

Video from TechEd 2005

JamieMac — Sat, 16 Jul 2005 01:49:00 GMT

I just saw Euan Garden's video of me from TechEd Orlando this year - you can watch it streaming here - or for the true collector, you can download and save the video here.

New data mining sample available

JamieMac — Thu, 14 Jul 2005 00:41:00 GMT

Those of you who read my blog likely know that our site went down again (not because I said so, but because there's a high correlation between people who read my blog and visit sqlserverdatamining.com) This time the reason was a bit more benign - in moving the box to it's new location, the hard drive shook loose and needed to be reseated. In the process, we added a second drive and ordered some more memory so we won't have these issues anymore - or as much, at least.

Anyway to commemorate the occasion of the site's rebirth I added a new sample to our downloads section - this time a sample plug-in viewer that provides a generic method of browsing the content of any model - regardless of the algorithm. It's a great tool for those who want to understand the content layout of the algorithms so you can write your own applications - or for those writing their own algorithms that need to debug how you are generating your content. Also, the source code is included for those who want to create your own plug-in viewers as well.

Along the same topic, if you missed my programming webcast, Intelligent Applications: Embedding Data Mining in Your Application , it is now available on-demand.

Hope you enjoy it! Keep on mining....

Busy busy month

JamieMac — Fri, 01 Jul 2005 21:20:00 GMT

I've kind of gotten out of the blog habit since before TechEd in June. Before TechEd I was disconnected for a week on a "vacation" visiting family for a niece-in-law's graduation. Boy, I could have blogged a lot that week - but only if I could guarantee that nobody from the family would have ever read it.... Anyway....

For the past few weeks I've been transitioning into my new role as Dev Manager for Analysis Services. I will still be doing all the data mining stuff that I've been driving for that past six years but will now also be responsible for the development of the OLAP side of the project. Luckily I have a team of undisputed experts at my disposal that will make this job easy.

SQLServerDataMining.com is still down. We've removed the machine from our previous hosting location to upgrade and are installing it in it's new home later today. There's been quite a few changes between Beta 2 and the June CTP so it took a couple days to upgrade from the beta and from VS7 to VS8 and fix up the applications that are running on the system. Having the newer bits will allow us to add some more interesting applications and new samples which we will do as much as we can. I'm also going to implement "video tips and tricks" where my team and I will do some screen captures and narrations showing specific tasks.

Next week I will be going to TechEd EMEA in Amsterdam - which is always great fun. My kids love the picture of the little cars.....

My scheduled Webcast on integrating data mining with the other BI technologies in the SQL Server box will be delivered by Zhaohui Tang - the data mining Program Manager - be sure to watch it next Thursday - you can register here

Many who watched my webcast from yesterday have requested the code samples I demonstrated. If I can't get them up on SSDM today, I'll follow up with a blog post containing the code snippets.

Intro to Data Mining Webcast Tomorrow 5/25 9:30

JamieMac — Wed, 25 May 2005 01:28:00 GMT

If you've ever wanted to learn what Data Mining is all about - what it is and what you can do with it, be sure to catch my webcast tomorrow. You can register for it here. I'll be giving a complete introduction to the topic, so no previous knowledge is required. After the webcast there will be a Q&A session, so any questions you have will be answered as well.

Hope to see you tomorrow!

Webcasts, Murphy's Law and our Newsletter

JamieMac — Sat, 30 Apr 2005 04:24:00 GMT

You know it always happens - you can run a demo everyday for two years and then when you're giving a webcast available to the world - something goes horribly wrong. OK, so it wasn't horribly wrong, but I did manage to flub it up - you can check out me tripping over myself here.

I figured out that eventually there were two problems. First, I found the bug before I ran the webcast, but only changed the code on my slide and not actually in the code (the code in the webcast is a newer, cleaner version than that on SQLServerDataMining.com). The problem was that I returned the minimum probability ratio of the input attributes and not the likelihood of the case.

Second was that we currently don't implement smoothing in our clustering prediction - something we will fix in a future Community Technical Preview drop. What smoothing does is ensure that no predictions of known states would ever go to 0. During the demo, I had set Age to 25, Education Years to 25 and Relation to Head of House as Son. My data mining enhanced application came back and correctly identified Education Years as the incorrect field. However, when I raised Education Years to 50, it said that Relation To HOH was out of whack. I guess it's true that noone with 50 years of education is likely to be living with his dad, but it's not what I expected. Looking into the details showed that the probability distribution was on the order of 10^-83, but the probability of "Son" was 0. Zero pretty much trumps 10^-83 - smoothing will mitigate this effect.

Anyway, Raman Iyer on my team has released the latest issue of The Data Miner. This month's issue includes the following topics

SQL Server Data Mining Wins New Converts in Denmark
SQL 2005 Data Mining Programmability Article on MSDN
Apollo Data Tech Launches Predictive Analytics Solution Based on SQL Server 2005 DM
New Visualization Tools for Clustering and Decision Tree Models
Microsoft Hometown IT Mag Highlights SQL 2005 Data Mining
Data Mining Webcasts: New and Upcoming

Cool Eweek article and a Webcast today!

JamieMac — Tue, 26 Apr 2005 21:47:00 GMT

There's a cool article on eWeek talking wbout Whidbey and Yukon that focuses on our BI stuff. Theres a quote that Data Mining has "matured so much that the SQL Server 2000 Data Mining engine looks like a simple prototype by comparison. "

Today at 1:00 PM PST I will be delivering a Webcast about embedded smarts into client apps using data mining. I will be using the data validation demo from sqlserverdatamining.com as the framework for the talk. If you want to listen in, you can register at http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032271565&Culture=en-US

I'll let you know when/where the recorded version will be for those who can't make it.