thinkWork @ komalk

Now that we are aware of the internals of syslog module, internals of managed discovery data mapper module, this blog-post aims at providing an example of how various modules in OM 2007 can be connected, configured and leveraged for various purposes.

Let us define the health of a host such that any syslog message generated by it with a severity less than 4 indicates something bad. So, how do we make this happen in OM 2007? We would first need a managed type. So, lets define that:

SCOM 2007 resource kit tools has been released and can be found here. 

Two of these are of particular interest for AEM:

AEM Validation: This tool can help validate that AEM is working once it has been enabled on a SCOM 2007 installation. This tool can be found here.

AEM Generic Mapping: This tool maps a lot of unknown application errors to more specific information. This tool basically comprises of a new MP that needs to be imported into the SCOM system.

I have posted a few updates to the MPs attached to the 2 blogs:

• AEM: Changing Threshold Values (via MP)
• Syslog Module

 Kindly look at them if you have used the prior MPs before.

In the last blog post, it was described how overrides for AEM applications/error groups can be changed via submitting discovery data for Microsoft.SystemCenter.CM.AEM.MonitorOverride through a management pack. Same can be achieved via writing code against the SDK service. Below is a snippet of code that can help achieve the same.

Various error groups and applications that report crashes over period of time can be seen in AEM Views.

The state of each error group is controlled by 3 thresholds: error/crash/hit count, unique users affected, unique computers affected. By default, the threshold value for all these 3 parameters is 50. In other words, if any error group has a error count and/or unique users affected count and/or unique computers affected count exceeding 50 then the health of corresponding error group goes red affecting it's availability aspect.

The state of each application is also controlled by 3 direct thresholds: error/crash/hit count, unique users affected, unique computers affected. But it is also affected by state of any of the error groups it hosts/encompasses. The threshold for the 3 basic thresholds is 50 here too.

These basic threshold values can be changed by submitting an instance of Microsoft.SystemCenter.CM.AEM.MonitorOverride for each error group/application whose threshold values need to be changed. This class type has 4 properties: ManagedEntityId and new value for the 3 thresholds. The ManagedEntityId can correspond to either ID of a watson bucket/application error group or ID of an application.

Attached is a sample/demo MP that shows how to change the threshold values. Let's analyze various parts of the MP.

The above discovery tells us that the new threshold values for all the 3 basic monitors is going to be 100 for all applications whose applicationName is foo.exe irrespective of their applicationVersion.

Let's look at the composition of the module type ApplicationThresholdDefiner used above.

There are 4 modules used: Scheduler (System.Scheduler from System.Library MP) followed by OleDbProbe, followed by Filter (System.ExpressionFilter from System.Library MP) and then followed by DiscoveryMapper.

The Scheduler module is configured to 15 minutes in the MP and can be changed. What does this indicate? This indicates that if new applications are discovered by the system which meet the criteria defined by user, then in 15 minutes (worst case), a threshold override would also be submitted to the system.

The OleDbProbe module retrieves the managed entity id for one or more applications meeting given criteria.

The Filter module ensures that there's at least one application known to OpsMgr via AEM that meets given criteria.

The DiscoveryMapper module is responsible for submitting override data to the system in form of instances of the class Microsoft.SystemCenter.CM.AEM.MonitorOverride defined in Microsoft.SystemCenter.ClientMonitoring.Library MP.

Similar composition has been provided in attached MP to change threshold values for 1 or more error groups.

It should be noted that the state change for all error groups and applications is computed every 15 minutes. As a result, with the attached sample MP, it can take upto 30 minutes to reflect the override changes in the OpsMgr console.

This max. value can be reduced to 15 minutes by reducing the configuration of scheduler module from 15 minutes to 30 seconds or something like that. However, reducing the configuration for scheduler module would have an impact on performance as that determines how frequently the OpsMgr DB is hit with queries as defined in the compositions in the sample MP.

However, based on user requirements, the value can even be bumped up or changed to a schedule basis as well.

The earlier post on Ole-Db module was all about what the module does. This post is more about what one can do with the module. Specifically, this post would talk about how connection strings can be constructed dynamically using RunAs accounts. This post will also demonstrate (with examples) how other modules can use data generated by Ole-Db module.

A new RunAs profile is defined in MP (Management Pack) as follows and using OpsMgr Console, one should be able to add various RunAs accounts for different machines to it.

The above RunAs profile can be used to make connections to DB in a couple of ways:

1. An easy way to this is to specify credentials explicitly in the ConnectionString configuration element of Ole-Db Probe module as follows:

2. Another way is to create a composite module type and associating the RunAs profile to that module type. This way one can use trusted connections, i.e., 'Integrated Security=true'. 

See attached MP for more information.

In order to use output from OleDb module as configuration for other module, one should first understand the structure of OleDb data. More about it can be found here. OleDb module can be connected to any module that takes either System.BaseData or System.OleDbData as its input type (both datatypes defined in System.Library MP). Some example modules from System.Library MP that can be connected to OleDb would be System.Secure.CommandExecuterProbe, System.ExpressionFilter, System.Event.GenericDataMapper, etc.

Suppose, if we need the result of 1st column and 2nd row to be passed to the next module, it can be referred in the configuration as:

The configuration for System.ExpressionFilter module would look something like:

The configuration for System.Secure.CommandExecuterProbe module would look something like:

The AEM feature in Operations Manager 2007 is next step from CER 2.0. A training video is available here.

As such, AEM falls under the umbrella of Client Monitoring which in its current state also emcompasses other features like Customer Experience Improvement Program (CEIP). A white paper on Client Monitoring can be read here.

Operations Manager SDK contains helper classes for programming with AEM under Microsoft.EnterpriseManagement.Monitoring.ClientMonitoring namespace.

There's a lot available online to read about AEM (one such read is here) and so the next few blogs on AEM would focus on how you can customize AEM as per your needs.

The Ole-DB module is a probe action module. It is defined in System.Library Management Pack (MP) as System.OleDbProbe. This module takes in the following as various configuration elements:

• ConnectionString
• This is a mandatory string element used for establishing a connection.
• Example Value: Provider=SQLOLEDB;Database=master;Server=.;IntegratedSecurity=SSPI
  
• Query
• This is an optional string element which when provided is used to query against the provider (info purveyed above).
• Example Value: SELECT COUNT(*) FROM SYSCOLUMNS
  
• GetValue
• This is an optional boolean element indicating whether the result set from query should be outputted or not.
• Example Value: true
  
• IncludeOriginalItem
• This is an optional boolean element. Since this module is a probe-action module, it basically needs a trigger in most cases. However, in some scenarios, the input item data needs to be preserved and this element indicates that. A value of true would cause the input item's xml to be outputted.
• Example Value: true
  
• OneRowPerItem
• If multiple rows are returned by the query string, then setting this optional boolean element would cause multiple items to be outputted each having only one row.
• Example Value: true
  
• DatabaseNameRegLocation
• This is an optional string element. There might be cases where the database name is not available to you and needs to be read from a registry location (based off HKLM). In short, the DatabaseName is not provided in ConnectionString and the registry location information can be provided here.
• Example Value: SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\Setup\DatabaseName
  
• DatabaseServerNameRegLocation
• This is an optional string element. And just like DatabaseNameRegLocation, this element allows listing a registry location under HKLM from where Server information can be read from and not from ConnectionString.
• Example Value: SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\Setup\DatabaseServerName

The output from this module is OleDb data defined in the System.Library MP as System.OleDbData and has the following elements in it:

Attached is a demo MP demonstrating how Ole-Db module can be used to generate alerts associated with management server when it takes too long to execute a query.

The Syslog module is a data source module. It listens to syslog datagrams (UDP packets) on specified port (default being 514 as per IETF standards.). It is defined in System.ApplicationLog Management Pack (MP) as System.ApplicationLog.SysLogReader.

The output from this module is Syslog data defined in the ApplicationLog MP as System.ApplicationLog.SysLogData and has the following elements in it:

• Facility
• Severity
• Priority
• PriorityName
• TimeStamp
• HostName
• Message

Each syslog packet received by the module contains a max of 1024 bytes and they are to be interpreted as ASCII characters. A best effort is made to parse these characters into the above xml elements. The module needs only one piece of configuration: the port number at which the syslog module should be listening.

Attached is a demo MP demonstrating how syslog module can be used to generate alerts when it receives a high severity syslog message.

Read about Syslog in MOM2000/2005 here.

The aim of this posting is to explain with examples how discovery data can be created out of any other data-item produced by modules and be submitted to OpsMgr.

Let us assume that we have an application that acts as a server for various agents deployed in our system. We would like to collect information about various agents that contact the server and run monitoring rules against them. These agents need to be modeled as a class whose instances/objects identify various agents.

The class could be defined in Management Pack (MP) as follows:

As can be seen above, class MyAgent has one key Property ‘AgentID’ of type int and 5 non-key properties of type string: AgentName, AgentVersion, AgentOwner, LastMessageReceived and Comments.

To submit instances (discovery data) for this class, we would have the following section in MP:

The above text in MP tells that there would be a discovery rule running targeted towards System.Computer that would generate instances of type ‘MyAgent’. These instances would be submitted to the system by an instance of AgentDataGenerator module.

A discovery rule expects DiscoveryData. Therefore, output of AgentDataGenerator module should be System!System.Discovery.Data and nothing else. If we have a module that outputs a different type, how do we convert it to DiscoveryData/instance of MyAgent?

Let’s look at our module first.

From the above definition looks like AgentExplorer is implemented as class MyServer in namespace ServerNamespace in ServerAssembly and it outputs AgentData. AgentData being defined as follows:

AgentData is implemented as AgentData in ServerAssembly. AgentData would definitely be inheriting from DataItemBase defined in Microsoft.EnterpriseManagement.HealthService.dll. As a result of this inheritance, AgentData would also provide serialization/deserialization of the contents into/from xml.

Let’s assume the xml representation (serialized version of the datatype) looks like below:

Now, let us look as to how we can connect the above module to a managed discovery data mapper module and generate discovery data. In short, we would be defining the module from table 2.

Now, let us look at configuration for DiscoveryMapper module (defined in above table) closely. (Schema is defined in a previous blog posting).

The TimeGenerated element specifies the time stamp associcated with discovery data.

DiscoveryType element can be AddUpdate/Snapshot/Remove based on kind of discovery operations performed.

DiscoverySourceType element specified source of discovery data submitted to the system. In the sample case, it’s a rule. However, discovery data can be submitted via task/user/system as well.

SourceId is also required besides the DiscoverySourceType. $MPElement$ retrieves the Id of rule in which it is being used.

SourceManagedEntityId is managed entity Id of the target in this case, which is System!System.Computer.

The Properties element section in configuration allows to evaluate a bunch of xpath expressions from serialized data item (e.g., see table 5). The evaluation can be conditional as well. The ‘If’ attribute requires an xpath expression whose evaluates to a non-empty string will cause the included properties to be evaluated. There can be multiple Evaluate element sections. The mapper would build a list of property elements (as permitted by the ‘Evaluate/if’ attribute) referenced by an ID associated with xpath expressions. Xpath functions are permitted. (See VersionValue). ID can refer to a list as well here (See OwnverValue).

Explanation for PropertySets element is not required for this sample case.

The ClassInstances section allows creation of multiple class instances based on evaluation of ‘If’ attribute. If the ‘If’ attribute permits, then instance for specified ‘TypeId’ will be created. The Properties associated with it are defined as name-value pairs. Name takes an Id to the property of the class whose value needs to be populated. The Value element can take values either from configuration, values from propertyID’s defined earlier (see IDValue, NameValue, etc), values hard-coded (See Comments) and values from a member of ID referencing a list (See AgentOwner).

Also, make note of Optional attribute defined on Property element (See  Comments). When set to true, the mapper would not include that particular property in submission list if the value evaluates to an empty string.

RelationshipInstances can be discovered similarly. See schema for more info.

See all the pieces combined together in attached MP.

Following is the schema of configuration for discovery data managed mapper module.

Hi there,

This is Komal Kashiramka and I am here to help you with/give you more insight into some of the cool features in System Center Operations Manager 2007.

Specifically, I can help in depth with:

• Client Monitoring (AEM [Agentless Exception Monitoring], SQM, etc.)
• Ole Db Template/Module
• Syslog Module
• Computer Verification Module
• Generic Managed Modules: Discovery-data Mapper, Event Mapper, Http-Listener/Uploader

Keep reading and have fun! This is just the beginning...