How to Manage Microsoft StreamInsight with PowerShell

How to Manage Microsoft StreamInsight with PowerShell

  • Comments 0

Are you running the StreamInsight host (StreamInsightHost.exe) or do you have an application that hosts the StreamInsight engine? If so, you might wonder how an administrator could retrieve manageability information or access metadata in the StreamInsight engine. It is possible that you often cannot afford and do not want to develop a full-blown application in Microsoft Visual Studio every time just to look at diagnostic information. Also, scripting capabilities are crucial because they allow you to automate administration tasks.

In those scenarios, PowerShell is the tool of the trade – and it works very well for StreamInsight. PowerShell allows you to perform administration tasks without the need to write a full-blown application in Visual Studio – and typically it is also your preferred choice for building powerful scripts. Below is the sequence of steps in PowerShell 2.0 to pull diagnostic views from the StreamInsight host while running the ObjectModel sample application on the host. You can find the sample code based on the ObjectModel sample at the end of this post.

This already works with our CTP2 bits - enjoy!

PS C:\> [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.ComplexEventProcessing")


GAC    Version        Location

---    -------        --------

True   v2.0.50727     C:\Windows\assembly\GAC_MSIL\Microsoft.ComplexEventProcessing\\Micro...

The first step above loads the StreamInsight client API into PowerShell. As you can see, StreamInsight is installed on the machine and we are picking up the assembly from the GAC.

PS C:\> $server = [Microsoft.ComplexEventProcessing.Server]::Connect("http://localhost/StreamInsight") 

With this step, we are establishing a connection to the running StreamInsight host process (make sure you have it started) and keep the connection in our server object. Let’s now have a quick look at all the members of the server object:

PS C:\> $server | gm



   TypeName: Microsoft.ComplexEventProcessing.Server


Name                    MemberType Definition

----                    ---------- ----------

ClearDiagnosticSettings Method     System.Void ClearDiagnosticSettings(System.Uri name)

CreateApplication       Method     Microsoft.ComplexEventProcessing.Application CreateApplication(string name)

CreateManagementService Method     Microsoft.ComplexEventProcessing.ManagementService.IManagementService CreateManag...

Dispose                 Method     System.Void Dispose()

Equals                  Method     bool Equals(System.Object obj)

GetDiagnosticSettings   Method     Microsoft.ComplexEventProcessing.DiagnosticSettings GetDiagnosticSettings(System....

GetDiagnosticView       Method     Microsoft.ComplexEventProcessing.DiagnosticView GetDiagnosticView(System.Uri name)

GetHashCode             Method     int GetHashCode()

GetType                 Method     type GetType()

SetDiagnosticSettings   Method     System.Void SetDiagnosticSettings(System.Uri name, Microsoft.ComplexEventProcessi...

ToString                Method     string ToString()

Applications            Property   System.Collections.Generic.IDictionary`2[[System.String, mscorlib, Version=2.0.0....

IsEmbedded              Property   System.Boolean IsEmbedded {get;}

Note the method for getting diagnostics views in the list above. Let’s invoke this to get event manager and plan manager statistics from the host: 

PS C:\> $dv = $server.GetDiagnosticView("cep:/Server/EventManager")

PS C:\> $dv


Key                                                 Value

---                                                 -----

All Events Count                                    19

All Events Memory (bytes)                           249856



PS C:\> $dv = $server.GetDiagnosticView("cep:/Server/PlanManager")

PS C:\> $dv


Key                                                 Value

---                                                 -----

Query Count                                         14

Stream Count                                        50

Operator Count                                      38 

Since we are running the ObjectModel sample, we can also pull statistics for the traffic sensor query: 

PS C:\> $dv = $server.GetDiagnosticView("cep:/Server/Application/ObjectModelSample/Query/TrafficSensorQuery")

PS C:\> $dv


Key                                                 Value

---                                                 -----

Query State                                         Suspended

Start Time                                          9/22/2009 5:34:02 PM

End Time                                            9/22/2009 5:34:03 PM

Stream Count                                        0

Operator Count                                      0

Total Incoming Event Count                          553

Total Consumed Event Count                          553

Total Produced Event Count                          192

Total Outgoing Event Count                          192

Last Incoming Event System Time                     9/22/2009 5:34:02 PM

Last Consumed Event System Time                     9/22/2009 5:34:02 PM

Last Produced Event System Time                     9/22/2009 5:34:03 PM

Last Outgoing Event System Time                     9/22/2009 5:34:03 PM

Total Consumed Events Latency (milliseconds)        14527.833

Total Produced Events Latency (milliseconds)        62457.0953

Total Outgoing Events Latency (milliseconds)        63553.2049

Last Produced CTI Timestamp                         12/31/9999 11:59:59 PM

Stream Event Count                                  0

Stream Memory Including Events (bytes)              0

Operator Index Event Count                          0

Operator Event Memory (bytes)                       0

Operator Index Memory (bytes)                       65870

Total Number of Times Operator Scheduled            708

Total Operator CPU Usage (milliseconds)             670 

This diagnostic view lists lots of useful stuff. You can find a detailed discussion of the individual elements in the Operations topic of the StreamInsight help file. Some fields that I always look into are the event counts and in particular the last CTI timestamp. Those are key indicators of how lively my system is.

As mentioned above, this example assumes that the query is running in a StreamInsight server which exposes its Web Service endpoint. Starting from our samples, there are two ways to do this.

  1. Expose the Web Service of an embedded server. We will cover this in a separate blog posting.
  2. Use the StreamInsightHost.exe.

In order to run the Object Model sample against a separate a StreamInsight host instead of an embedded server, you need to tweak it a little bit. These steps are described in the README.txt that is enclosed in the samples package. Attached is the complete code of the file ObjectModel.cs with the necessary changes. Note that you will have to edit the paths of the input csv files in that code.

Attachment: ObjectModel.cs
Leave a Comment
  • Please add 8 and 4 and type the answer here:
  • Post