Jamie's Junk : Time Series

KDD 2008 and Incredibly Awesome SQL 2008 Data Mining Demos

JamieMac — Mon, 25 Aug 2008 21:48:00 GMT

Bogdan and Raman are currently outside of Las Vegas at the KDD 2008 (Knowledge Discovery and Data Mining) conference. At the conference they are showing all the goodness that is SQL Server 2008 Data Mining, but also a "secret project" that Bogdan has been working on all summer - Data Mining in the Cloud. That's right - access to the awesome data mining functionality of SQL Server 2008 with no local Analysis Services server.

Currently the technology demonstration allows you to access the Table Analysis Tools for Excel 2007 by connecting to our hosted data mining service, meaning you can use the Table Analysis Tools anywhere at anytime without connectivity to your local IT infrastructure. Additionally there is a web interface that allows you to upload a limited amount of data and play with the tools without even having Excel! Currently only a few of the tools are implemented in the web interface. If for any reason you haven't had a chance to grab those amazing Table Analysis Tools and see what SQL Server Data Mining can do for you - run, don't walk, over to http://www.sqlserverdatamining.com/cloud/ and try them out right now!

Not to limit ourselves to revolutionary cloud data mining technology, the data mining team here in SQL Server also created another demo application that clearly shows the amazing advances in our Time Series algorithm. Shuvro and Tatyana put together an incredible application that front ends the time series capabilities of SQL Server 2008. In this application, you not only get a simple interface for creating time series models resulting in a chart showing the historical and forecasted data, but you can also hover over any forecasted points and it will highlight all the data points in that series and other series that have an impact on the prediction! Even more, you can click and drag forecasted values to see how they change future predictions! Even more, the application will show you all the DMX statements that it uses to create these models and generate these forecasts. This demo is truly an incredible piece of work. You can find out more about the demo and how to download it at http://www.sqlserverdatamining.com/ssdm/Home/TipsTricks/tabid/61/Default.aspx?id=383.

How can we mine? Let me count the ways...

JamieMac — Tue, 20 Nov 2007 00:18:10 GMT

Recently I received some customer feedback that SQL Server Data Mining "doesn't have enough algorithms." More specifically, the comment was that we have the same capabilities are other Data Mining providers, we just "hide" many facilities as algorithm parameters rather than separating out each as a named algorithm. So let's count the Microsoft algorithms a few different ways to work this out.

First - let's go by the box. This is the list of algorithms as specified in Books Online

Microsoft Decision Trees
Microsoft Clustering
Microsoft Naive Bayes
Microsoft Association Rules
Microsoft Neural Networks
Microsoft Time Series
Microsoft Sequence Clustering
Microsoft Linear Regression
Microsoft Logistic Regression

So that's nine - count 'em nine algorithms. But that's just one way. If you look at my book, Data Mining with SQL Server 2005 written with Zhaohui Tang, there are only seven algorithms! What? You say! How can it be?

Let me explain. During the development of SQL Server 2005, we realized a couple of tricks; 1) linear regression was the same as our tree algorithm, just forced to not split; and 2) logistic regression was the same as our Neural Nets, just with zero hidden layers. However, we got similar feedback - people want more algorithms, and specifically these ones, so we set up two "new algorithms" by forcibly setting parameters on the Decision Tree and Neural Network algorithms and voila! we shipped with nine named algorithms. It would have been hard to fill up two entire chapters explaining that last sentence, so Zhaohui and I decided just to stick to the seven core algorithms.

Anyway, this posting isn't really about how to count less algorithms, I really wanted to show you how to count more. When we set about designing SQL Server Data Mining, we really and truly tried to make data mining operations simpler. We thought at the time, rightly or wrongly, that the more options end users have, the more complicated and difficult the product would be to use. Therefore, we tried to determine the best behavior in a class, and make more advanced options available through parameters.

For example, take our clustering algorithm. We assumed that if people wanted clustering, most likely didn't care about the details of the algorithm, they just wanted to get the job done, and that those people who wanted more would look for it (the design principal - make the simple things simple, and the complex things possible). So we bundled up different flavors of clustering into a single package that many vendors would have broken apart. So let's start counting with clustering.

Our default clustering behavior is EM (Expectation Maximization) clustering using the Bradley-Fayyad scalable framework

Setting a parameter changes that to a K-Means clustering implementation using the same framework

3+4

Setting the same parameter another way, provides non-scalable versions of the two clustering varieties. (I know it's hard to swallow that the non-scalable versions count as separate algorithms, but if you started with the vanilla versions and added scalability, then of course you would consider those versions as new algorithms - I'm just working backwards here.

Let's move to our Decision Tree algorithm and we will consider our classification tree as one algorithm.

But our Decision Tree also predicts continuous and counts as a regression tree, so we will count that as another algorithm.

Oops! Our Decision Tree also creates full linear regressions at each of the leaf nodes. To get the typical regression tree behavior you need to make sure that none of the continuous inputs have the REGRESSOR flag and you get yet another algorithm.

Oh yeah, our trees allow for multiple targets in each model, allowing the discovery and display of interrelated patterns through our dependency net. I've seen other vendors advertise such functionality as an "algorithm" so there's our #8.

How about collaborative filtering with Trees - just slap a PREDICT flag on a nested table, and you have a complete recommendation system. Let's call it Associative Trees

Naive Bayes.

11+12

If we're going to count Associative Trees, we also have "Associative Bayes". I guess the multiple target interrelated pattern thing counts here as well.

Association Rules. A-priori style

It seems odd to count association rules twice since we can do predictions with it, but nobody else does it (or didn't before - correct me if I'm wrong), so Predictive Association Rules makes the cut.

15+16+17+18

Well if we're going to go and call predictive association an algorithm, we had better do the same for our clustering algorithm. Granted, clustering doesn't make a great classifier or estimator, but the great Highlight Exceptions functionality of the Data Mining addins comes from this ability. Yes, we can do nested table prediction as well with clustering, but I wouldn't recommend it to my mom, so I won't take another four for that.

19+20+21+22+23

Neural Networks, Sequence Clustering, Time Series, Linear Regression and Logistic Regression. Yeah, yeah, I could get into varieties here, but I think you get the point.

So by that count, and not being too creative (trust me, I can do more) we're looking at 23 algorithms in SQL Server 2005 Data Mining. There are a few more options coming up in SQL Server 2008 that are worth discussing as well.

The time series of SQL Server 2007 uses the ARTXP algorithm - "Auto Regression Trees with Cross Predict". In 2008, we're adding ARIMA as well, for algorithm #24.

And yet again with Time Series, the default mode of operation is to blend ARTXP and ARIMA results in an intelligent way to maximize accuracy and stability for #25.

Arbitrarily there are 23 algorithms in SQL 2005 and 25 in SQL 2008, with the option of teasing out even more varieties depending on how you apply parameters and flags to the base nine (or seven - depending on how you count!). Next time someone quips that SQL Server only has "nine" algorithms, tell them that's just the packaging - each of those nine provides a wealth of value in each box.

New Time Series Features for SQL Server 2008

JamieMac — Mon, 22 Oct 2007 18:46:00 GMT

The time series algorithm has had some major changes for SQL Server 2008. We've ramped up the algorithm and added prediction against new data. While the new data prediction is just as cool as the algorithm changes, I'll have to save that for later - this post I'll just tell you about the algorithm.

As you may or may not know, the time series algorithm in SQL Server 2005 uses the ARTXP algorithm from Microsoft Research. ARTXP stands for "Auto Regression Trees with Cross Predict" and of course, since you have to have an 'X' in any acronym worth its salt, "cross" is represented by an 'X'. ARTXP uses similar regression trees as are found in our decision tree algorithm to predict future values based on past history, including the past history of other series. It turns out that ARTXP, as verified through testing by Microsoft Research and the product team, is wickedly accurate at predicting the next step of a series - better than all other algorithms we've tried. However, as you move away from the time horizon, depending on your data, the predictions can grow unstable fairly quickly. Veteran users of the algorithm know how that's handled by the system - it agressively cuts off the prediction stream - i.e. it just stops.

We heard some frustration from users that hit this, and knew we had to do something. We didn't want to lose the accuracy or cross predictive ability of the ARTXP algorithm, but needed to control stability. Additionally, we've had many requests for more "industry standard" algorithm implementations simply because they are well known (not because they are necessarily better). So what did we do?

In SQL Server 2008 Data Mining, we have enhanced the Time Series algorithm to include not only an implementation of ARTXP, but also an implementation of ARIMA (Auto Regressive Integrated Moving Average). The ARIMA algorithm is well-known and although all autoregressive algorithms have the potential for instability in long-term predictions, ARIMA tends to hold out for most data sets. So, does this mean that you as the user have to decide between short-term accuracy and long term stabililty? The answer is a resounding NO! The SQL Server 2008 Time Series algorithm by default builds both an ARTXP model and an ARIMA model. For predictions, a weighted averaging scheme is used to favor the short term accuracy of ARTXP and the long term stability of ARIMA - essentially the best of both worlds!

To control this new behavior we have add three new parameters:

FORECAST_METHOD - controls which algorithms are used with MIXED being the default, but you can specify ARIMA or ARTXP as well.

PREDICTION_SMOOTHING - controls the mixture of the ARTXP and ARIMA results. Setting this value closer to 0 favors ARTXP more heavily, whereas setting it closer to 1 favors ARIMA more.

INSTABILITY_SENSITIVITY - ok, this one has nothing to do with ARIMA, just your feedback on ARTXP. If you specify ARTXP as the forecast method, you can now control the threshold at which the algorithm will cut off predictions. Setting this to 0 will turn of instability checks altogether. (By the way, with ARTXP if your regression formulae have coefficients that are all less than 1, you will never have a stability problem. Stability issues only occur when there are coefficients greater than 1.

If you want to try out the new Time Series algorithm, it is available in the July 2007 CTP (and later) of SQL Server 2008 which you can find under www.microsoft.com/sql/2008. In a later post (I know better than to specify when) I'll talk about the new prediction methods that you can apply to Time Series models, making these model reusable in a way never available before in SQL Server.

Help with Time Series on OLAP - another great blog with SQL Server Data Mining

JamieMac — Sat, 10 Jun 2006 21:22:00 GMT

Dejan Sarka writes about how to get "Cases from Multiple Dimensions of OLAP Cubes in the Time Series Algorithm" - an issue that has stymied many users. He also has a range of great posts on other DM and BI topics as well - worth a good read!

Forecasting on the fly using Analysis Services Data Mining and Reporting Services

JamieMac — Thu, 23 Feb 2006 00:12:00 GMT

Just found this great article on MSDN showing how to use session Time Series models in RS to forecast while generating a report. Cool beans!

Time Series Prediction

JamieMac — Tue, 29 Mar 2005 01:06:00 GMT

Last week I was playing around with the Time Series algorithm - ok I was showing a customer - and I wanted to show the prediction language used to display the charts. So I used the tools and showed them the basic prediction query for forecasting, but I wanted to go further and show them cool stats like the variance and standard deviation of each predicted value. So I tried, and failed, and tried again, and failed, and after a few more furtive attempts had to admit to myself that I just didn't know (shame shame) how to get the standard deviation for a time series prediction.

But I didn't give up there - since you can see the deviations in the chart viewer, obviously you can get them from the server. I fired up my handy-dandy SQL Profiler, connected to my Analysis Server and viola! I pulled the query right out of the log! Immediately afterwards I had one of those "oh yeah" moments where you realize that you knew (or should have guessed) all along but had a brain dead moment and simply spaced.

In any case, the syntax you need to use is not trivial and no one not on the development team is likely to guess out of thin air, so I wrote up this tip explaining the convoluted syntax for the world to see. Once you see it, you won't likely forget it, and if you do - grab the profiler and puul it out, just like the dev team does!