StreamInsight is a platform being released alongside SQL Server 2008 R2 to enable the development of Complex Event Processing (CEP) applications. So what does that mean?

If you have something which is giving output as an unbounded data stream (the example shown at PDC09 was a power monitor to a laptop, but this can be anything from windows event logger to web-services to financial data to monitoring devices etc.), you can take that data in form of events as a stream. You can then adapt it into a CEPStream (one of the classes given in the library) which enables powerful filtering, joining and transforming via LINQ, then take the results from this Query to Adapt to an output device (a physical device, a database, application etc.) as a stream and push it out.

Here there are 3 stages:

1. Get the data from the source and Adapt it to be consumed
2. Query (Filter/Join/etc.) the data using LINQ
3. Take the Queried data, Adapt it so the target may consume it, and Push it out.

These 3 stages are split out into 3 parts.  An Input Adapter for initializing the source stream, reading data, creating events from that data and Enqueuing those events to be processed.  An Output Adapter for initializing the output stream, Dequeuing processed events, Adapting the events to be readable by output stream and writing the data to the output stream.  Finally the server, which creates the input and output adapters, defining the queries to do the transform and hooking the streams together.

From a development point of view this is helpful as it is built on the .Net Framework so the adapters and queries can be written in a familiar language, the use of Streams within .Net is standard for consuming resources and LINQ provides powerful and readable modifications which again is familiar to most .Net developers.

Event Driven Applications

In the normal Request/Response model for a Data Driven Application the user does something which returns something back from a web-service/database etc. In an Event Driven Application, the user is a subscriber to a feed of data, and it’s the data change which will update the user and not the user’s request. This is useful in applications which want to consume data as the sensors / feeds driving it produce readings and data.

(Introduction to Microsoft SQL Server 2008 R2 StreamInsight / White Paper)

Getting Started

I will be going into further details of the technology in later posts, but to start with there were 2 talks at PDC09 which walk through some demos:

Introduction to Microsoft SQL Server 2008 R2 StreamInsight

Advanced Microsoft SQL Server 2008 R2 StreamInsight

Also the November SDK is available here which contains some samples of the different thing StreamInsight enables:

November CTP StreamInsight SDK

Events Have Shapes

If you look at any of the examples you will notice that for each type of Adapter there appear to be 3 different types; a Point, Edge and Interval Adapter, each extending one of:

PointInputAdapter, EdgeInputAdapter and IntervalInputAdapter or

PointOutputAdapter, EdgeOutputAdapter and IntervalOutputAdapter.

This is because an event may have a different shape depending on what that event is…

Point

The simplest to understand is PointEvent (a class under Microsoft.ComplexEventProcessing in the SDK)

microsoft.complexeventprocessing.pointevent

It is an event which happens at a specific point in time (hence it only has a StartTime property). e.g. A reading from a temperature sensor.

Interval

An interval is an event where you know the starting and stopping time of the given event (e.g. Process Running on a monitoring device)

microsoft.complexeventprocessing.intervalevent

It has a StartTime and an EndTime.

Edge

An edge event is where at reading the data in, you know if something has started or finished, but unlike the interval, you don’t know when the opposite happened, e.g. you know when it started or when it finished but not both.

microsoft.complexeventprocessing.edgeevent

This, like the Interval has a StartTime and an EndTime, but also has an EdgeType enumeration which allows you to specify when reading which time is valid (either it is starting, or stopping).

I will be delving further into StreamInsight in further blog posts, and as I explore StreamInsight further.