• System Center Platform Team

    Using SDK Criteria for Querying Instances

    • 4 Comments

    Criteria offers a way of querying over instances and types with a specific filter. This post covers the different ways of using criteria to query instances. Querying over instances involves querying over specific instances or instances that are part of a relationship in a projection. There are two different classes that offer these functionalities.

       

         namespace Microsoft.EnterpriseManagement.Common

        public class EnterpriseManagementObjectCriteria : QueryCriteria<EnterpriseManagementObjectCriteria>, IObjectCriteria

        {

             }

       

        public class ObjectProjectionCriteria

        {

             }

     

    Class EnterpriseManagementObjectCriteria works on pure instances. Say, you wanted to query for work items and were looking for all work items that satisfied a particular condition.


    Class ObjectProjectionCriteria is used for querying over instances based on some condition that spans over a relationship. Say, you wanted to query for all incidents that were assigned to a certain user with some criteria acting on the relationship “IncidentAssignedToUser”.


    Constructors of both these classes (EnterpriseManagementObjectCriteria and ObjectProjectionCriteria) work on an input criteria parameter which is a schema based XML string, the schema of which varies slightly between both the classes.


    Most likely your query will fall into one of these categories which correspond to a specific type of schema element in your criteria XML.Please refer to more specific examples below for a given scenario. These descriptions and the examples that followed are explained with an assumption that readers have knowledge of core concepts such as MP definitions, type projections, relationships and various types of relationships such as reference or containment.

    • You are querying over an instance (that exists by itself or in a relationship) evaluating a property on it for a specific value. (Please refer to  SimpleCriteriaType and CriteriaOperatorType in the schema)
    • You are querying over an instance (that exists by itself or in a relationship) evaluating a property on it based on a regular expression. (Please refer to  RegExCriteriaType and RegexOperatorType in the schema) 
    • You are querying over an instance (that exists by itself or in a relationship) evaluating a property in a similar fashion such as the SQL IN/NOT IN clause. (Please refer to type InType/NotInType in the schema)
    • You want to check if an instance actually exists and not are necessarily filtering on a particular value. (Please refer to type UnaryCriteriaType and UnaryOperatorType in the schema)
    • Your query spans over some form of Containment relationship. (Please refer to  ContainsCriteriaType,NotContainsCriteriaType, ContainedCriteriaType, NotContainedCriteriaType in the schema)
    • You are querying over an instance (that exists by itself or in a relationship) evaluating a property on it based on a full text expression. (Please refer to  FullTextCriteriaType and FullTextOperatorType in the schema)
    • You have a NOT expression in your criteria (Please refer to NotType Criteria).
    • Your criteria involves performing a combination of the above functions. (Please refer to And Type, Or Type in the schema)

    The combination of all examples stated below will give an understanding of some core concepts in criteria.

     

    Example 1: Querying for an instance on some property using simple expression criteria, using EnterpriseManagementObjectCriteria class.

     

    Consider a query for retrieving all instances of a certain class based on some property value. Say, fetching all incidents whose title contains the text “critical”. Since this is purely evaluating a particular property on the criteria we can construct an SDK criteria of type “SimpleCriteriaType” for the purpose. As per the schema all criteria start with an “ExpressionType”

     

    All property evaluation is of the format: Context/ [Type]/ [PropertyName] (All start with a Context tag which is required and denotes the type of instance we are querying for, [Type] is replaced by the name of the type we are querying on, and in this case “System.WorkItem.Incident” and [PropertyName] is the name of the property which in this case is “Title”.

     

    We replace the inner text of the <Value> tag with the actual value to be compared with. Since we are performing a Like operation the value is appended with % as in %critical%.

     

    This forms criteria Xml as below:

     

    <Criteria xmlns="http://Microsoft.EnterpriseManagement.Core.Criteria/">

      <Expression>

        <SimpleExpression>

          <ValueExpressionLeft>

            <Property>$Context/Property[Type='System.WorkItem.Incident']/Title$</Property>

          </ValueExpressionLeft>

          <Operator>Like</Operator>

          <ValueExpressionRight>

            <Value>%critical%</Value>

          </ValueExpressionRight>

        </SimpleExpression>

      </Expression>

    </Criteria>

     

    This piece of code creates an EnterpriseManagementObjectCriteria object:

     

    ManagementPack incidentManagementPack = managementGroup.GetManagementPack("System.WorkItem.Incident.Library", "31bf3856ad364e35", “1.0.0.0”);

     

    ManagementPackClass incident = managementGroup.EntityTypes.GetClass("System.WorkItem.Incident", incidentManagementPack);

     

     

    EnterpriseManagementObjectCriteria criteria = new EnterpriseManagementObjectCriteria(criteriaXml,incident, incidentManagementPack,managementGroup);

     

    (CriteriaXml here indicates the xml based criteria string above)

     

    You could also opt to use the constructor below, however since the management pack is excluded, the string criteria has to include the guid values for the class names and property names instead of the name.

     

    EnterpriseManagementObjectCriteria criteria = new EnterpriseManagementObjectCriteria(criteriaXml, incident,managementGroup);

     

    The same criteria with guid values for the type and property will look as shown below:

     

    <Criteria xmlns="http://Microsoft.EnterpriseManagement.Core.Criteria/">

      <Expression>

        <SimpleExpression>

          <ValueExpressionLeft>

            <Property>$Context/Property[Type=8C61EEAB-9F24-0EB5-5B4A-BF4DD59C0EF3']/ 0C7DC36E-876F-8B2B-C53B-A6297208E6F0$</Property>

          </ValueExpressionLeft>

          <Operator>Like</Operator>

          <ValueExpressionRight>

            <Value>%critical%</Value>

          </ValueExpressionRight>

        </SimpleExpression>

      </Expression>

    </Criteria>

     

     

    Example 2: Querying on related instances using Object Projection Criteria and usage of unary expression

     

    The above example talks about retrieving instances of a particular class, however say you wanted to retrieve those instances based on a certain relation. Consider querying for all incidents that were related to a config item. These are instances that are part of the relationship “System.WorkItemRelatesToConfigItem”. Note that we need to query over this relationship here.  Also, we are looking for incidents that are related to any config items that exist and not necessarily a config item of a particular property value. Hence we will be using “UnaryExpression” criteria in this case.

     

    As per the schema the unary expression criteria looks like below:

     

    <Criteria xmlns="http://Microsoft.EnterpriseManagement.Core.Criteria/">

      <Expression>

        <UnaryExpression>

          <ValueExpression>

            <GenericProperty Path="$Context/Path[Relationship='WorkItem!System.WorkItemRelatesToConfigItem']$">Id</GenericProperty>

          </ValueExpression>

          <Operator>IsNotNull</Operator>

        </UnaryExpression>

      </Expression>

    </Criteria>

     

    Unary expressions contain just an expression and an operator to check for null. Notice how we are using a generic property tag instead of a property tag here. This is used for querying over generic properties that are common to all instances and not just properties specific to that particular instance. Following are the list of generic properties that we can be used in this format.

     

     

    public enum EnterpriseManagementObjectGenericPropertyName

           {

            Id,

            Name,

            Path,

            FullName,

            DisplayName,

            LastModified,

            TimeAdded,

            LastModifiedBy

           }

     

     

    As long as the property is one of the above we can use a <GenericProperty> tag and need not specify the type in the criteria. The attribute Path indicates the relationship on which we are using the criteria. The Path usually starts with “Context/” and is followed by one or more relationships. We have just one in this case. You can use the above sdk criteria to build an Object Projection Criteria object.

     

    Add a projection that defines the relationship - System.WorkItemRelatesToConfigItem in a custom management pack as below:

     

    <TypeProjection ID="WiRelatesToCiProj" Accessibility="Public" Type="CoreIncident!System.WorkItem.Incident">

      <Component Path="$Target/Path[Relationship='WorkItem!System.WorkItemRelatesToConfigItem']$" Alias="Rel" />

    </TypeProjection>

     

     

    The following code creates an ObjectProjectionCriteria :

     

    ManagementPack workItemMp = managementGroup.GetManagementPack("System.WorkItem.Library", "31bf3856ad364e35", new Version("1.0.0.0"));

    ManagementPack projMp = managementGroup.GetManagementPack("custom.Mp",null, new Version("1.0.0.0"));

    ManagementPackTypeProjection workItemRelConfigProj =managementGroup.EntityTypes.GetTypeProjection("System.WorkItem.Incident.RelatesToConfig", projMp);

     

    ObjectProjectionCriteria projCriteria = new ObjectProjectionCriteria(criteriaxml, workItemRelConfigProj, projMp, managementGroup);

      

        (criteria xml is the above mentioned xml string).


    Just like EnterpriseManagementObjectCtieria you can also use the constructor that does not take a management pack, but will need to specify guids for all elements.

     

    Example 3: Using typeconstraints, references and tokens in criteria. Usage of And/Or criteria.

     

    Consider the scenario where you are not only querying for workitems related to config items but for those that are related to a specific config item based on some property. Here’s where using a type constraint is useful. Consider the following criteria.

     

    <Criteria xmlns="http://Microsoft.EnterpriseManagement.Core.Criteria/">

      <Reference Id="System.Library" Version="1.0.0.0" Alias="System" />

      <Expression>

        <And>

          <Expression>

            <SimpleExpression>

              <ValueExpressionLeft>

                <Property>$Context/Path[Relationship='System.WorkItemRelatesToConfigItem' TypeConstraint='System!System.Computer']/Property[Type='System!System.Computer']/DisplayName$</Property>

              </ValueExpressionLeft>

              <Operator>Equal</Operator>

              <ValueExpressionRight>

                <Value>testmachine</Value>

              </ValueExpressionRight>

            </SimpleExpression>

          </Expression>

          <Expression>

            <SimpleExpression>

              <ValueExpressionLeft>

                <GenericProperty Path="$Context/Path[Relationship='System.WorkItemAssignedToUser' SeedRole='Source']$">Id</GenericProperty>

              </ValueExpressionLeft>

              <Operator>Equal</Operator>

              <ValueExpressionRight>

                <Token>[me]</Token>

              </ValueExpressionRight>

            </SimpleExpression>

          </Expression>

        </And>

      </Expression>

    </Criteria>

     

    Note :

     

    Ø  The typeconstraint restricts the filter to just config items of type System.Computer.

    Ø  The type “System.Computer” is referred with an alias “System!” since it’s defined in a different management pack. The reference is added to the existing MP.

    Ø  The criteria combines two different expression in an “AND” condition and hence queries for all workitems that are assigned to me (current user) and that also have a relationship with class “System.Computer”.

    Ø  The token [me] instead of the value to be compared with resolves to the current user. Other tokens that are allowed are :

     

    [me] – denotes current user

    [mygroups] – denotes current user’s group

    [now] – denotes the current utc time.

    [now + 5s], [now – 2h] – denotes some variations using s(seconds), d (days), m(months), h(hour).

     

    Example 4: Using sdk criteria to create customizable views and usage of Containment criteria.

     

    Say you have a containment relationship that you want to query over. Consider a containment relationship where you want to group all incidents related to hardware under a group “Hardware Incidents”.

     

    Currently the SM console, does not offer a way for doing the same.


     


     

     

    However, we can achieve the same by adding SDK criteria and customize the view.

     

    Step 1) In the SM console, go to WorkItems -> IncidentManagement.

    Step 2) Under Tasks choose “Create View”

    Step 3) Edit the General and Display tabs, do not add any criteria under the Criteria tab. Enter view name as “Hardware Related Incidents”.

    Step 4) Save the view. The view will show up under incident management and get saved in the unsealed management pack “Service Manager Incident Management Configuration Library”.

    Step 5) Export this MP.

    Step 6) Add the following class, relationship and type projection to the MP.

     

    <ClassType ID="IncidentHardwareGroup" Base="WorkItem!System.WorkItemGroup" Abstract="false" Accessibility="Public" Hosted="false">

      <Property ID="GroupId" Type="string" Key="true"  />

    </ClassType>

     

    <RelationshipType ID="IncidentGroupContainsIncidents" Base="System!System.Containment" Abstract="false" Accessibility="Public">

      <Source ID="incidentGrp" Type=" IncidentHardwareGroup " />

      <Target ID="incident" Type="CoreIncident!System.WorkItem.Incident" />

    </RelationshipType>

     

    <TypeProjection ID="IncidentGrpProj" Accessibility="Public" Type="CoreIncident!System.WorkItem.Incident">

      <Component Path="$Target/Path[Relationship=' IncidentGroupContainsIncidents ' SeedRole='Target']$" Alias="group" />

    </TypeProjection>

     

    Step 7) Under the definition generated for this view you will notice an empty criteria tag (<Criteria/>), since we did not pick a criteria while creating this view.

    Step 8) Replace <Criteria/> with the criteria below.

     

    <Criteria>

      <QueryCriteria Adapter="omsdk://Adapters/Criteria" xmlns="http://tempuri.org/Criteria.xsd">

        <Criteria>

          <FreeformCriteria>

            <Freeform>

              <Criteria xmlns="http://Microsoft.EnterpriseManagement.Core.Criteria/">

                <Expression>

                      <Contained>

                          <Class>$MPElement[Name='System.IncidentCriticalGroup']$</Class>

                  </Contained>

                   </Expression>

                   </Criteria>

            </Freeform>

          </FreeformCriteria>

        </Criteria>

      </QueryCriteria>

    </Criteria>

     

    The highlighted portion is the actual sdk criteria, rest of the format is needed for UI purposes. If you wanted to have some criteria on the group as well, you could have a criteria such as:

     

    <Criteria xmlns="http://Microsoft.EnterpriseManagement.Core.Criteria/">

       <Expression>

         <Contained>

          <Class>$MPElement[Name='System.IncidentCriticalGroup']$</Class>

           <Expression>

             <SimpleExpression>

               <ValueExpressionLeft>

                <Property>$Context/Property[Type='System.IncidentCriticalGroup']/GroupId$</Property>

              </ValueExpressionLeft>

              <Operator>Equal</Operator>

               <ValueExpressionRight>

                <Value>HardwareRelated</Value>

              </ValueExpressionRight>

            </SimpleExpression>

          </Expression>

        </Contained>

      </Expression>

    </Criteria>

     

     

    Step 9) Change the name of the type projection in the view to “IncidentGrpProj”.

    Step 10) Import the MP.

    Step 11) The view will get updated and show the filtered instances as below.

     


     

     

    I have covered the important concepts in SDK criteria and examples on querying over instances using the same. Using SDK criteria for querying over types will be covered in a different post. Hope you find this useful.

  • System Center Platform Team

    Troubleshooting the Data Warehouse: Reports aren’t deployed after registering the data warehouse

    • 3 Comments

    This is the latest in “Troubleshooting the Data Warehouse” series. The goal of this series is to not only help you troubleshoot specific issues you may be experiencing but also teach you more about the data warehouse. For a complete list of posts in the series, check out Troubleshooting the Data Warehouse: An overview.

     

    The symptom

     

    When you open the console, a dialog box appears indicating that the Reporting Service is unavailable. Or if the Report wunderbar appears, there are no reports in the Reporting workspace and if you look on the Reporting Services server, no reports have been deployed.

    All other aspects of the data warehouse deployment seem to have gone smoothly. For example, when you click on the Data Warehouse wunderbar then Data Warehouse Jobs, you see 2 extract jobs, a transform job, a load job and an MPSync job.

     

    The troubleshooting steps

     

    To troubleshoot the above issue, please verify things in the following orders

     

    1. Check the deployment status of your management packs

    In the Service Manager Console

    ·         Click on the Data Warehouse wunderbar

    ·         Click on Management Packs and in the search Filter field type report. This will filter to the report related management packs

    ·         Check the deployment status (last column) of the following MPs…none should have Failed

    o   ServiceManager.ActivityManagement.Report.Library

    o   ServiceManager.ProblemManagement.Report.Library

    o   ServiceManager.IncidentManagement.Report.Library

    o   ServiceManager.ConfigurationManagement.Report.Library

    o   ServiceManager.ChangeManagement.Report.Library

     

    2. Check the event log for error messages about the assembly Microsoft.EnterpriseManagement.Reporting.Code.dll

    If any of the above 5 management packs failed deployment

    ·         Go to the Data Warehouse Management Server

    ·         Open the Operations Manager Event log

    ·         Filter the events with Event Sources as Deployment and Event Level as Error

    ·         If there are error messages in the event log which indicate “cannot load Assembly Microsoft.EnterpriseManagement.Reporting.Code.dll”

    o   Your SSRS installation may be on a different server than the Data Warehouse Management Server. If so, please follow the deployment guide and copy this assembly to the SSRS server then add access permission to it.

    For detail instruction please follow the deployment guide.

    o   If your SSRS installation is on the same server as the Data Warehouse Management Server, restart the SSRS service.

    ·         To restart SSRS

    o   Log on to the server where SSRS is installed

    o   Click Start > All Programs > Microsoft SQL Server 2008 > Configuration Tools > Reporting Services Configuration Manager

    o   In the pop up window, click Connect, then click Stop, after that click Start.

     

    3. Redeploy the failed report management packs

     

    In the Service Manager Console:

    ·         Click on the Data Warehouse wunderbar

    ·         Click on  Management Packs and in the search filter type report

    ·         For each of the MPs listed in step 1, click Restart Deployment in the right hand Task Pane. Note that if the deployment status of a management pack is completed, click on this management pack, the Restart Deployment is grey out in the Task Pane

    Once the deployment status of the report mps turned from Failed to Completed, open the Console again, reports should display in the Reporting workspace. You may need to restart the console to see the reports as the console caches the list of reports.

  • System Center Platform Team

    Troubleshooting the Data Warehouse: Management packs are stuck in “pending association”

    • 3 Comments

     

    This is the latest in “Troubleshooting the Data Warehouse” series. The goal of this series is to not only help you troubleshoot specific issues you may be experiencing but also teach you more about the data warehouse. For a complete list of posts in the series, check out Troubleshooting the Data Warehouse: An overview.

     

     

    The symptom

     

    After registering Service Manager to the Data Warehouse and several (4+) hours have passed some management packs are still stuck in “pending association” state. You can check this by opening the Console, clicking Data Warehouse > Data Warehouse Jobs > MPSync Job, and then click Details from the Tasks Pane

     

    The troubleshooting steps

     

    To troubleshoot this issue, investigate the following things in order.

    1. Review each batch ID for the “stuck” MP via the MPSyncJob dialog box

    In the MP Sync Job dialog, click on the Management Pack column name to sort the list according to Management Pack name.  Find those management packs with Pending Association status, check if in the later batch, the Pending Association management pack turns to Associated.

    For example in the following figure I’ve highlighted:

    ·         In batch 136, Management Pack Microsoft.SystemCenter.ConfigurationManager is in Pending Association

    ·         In batch 207, Management Pack Microsoft.SystemCenter.ConfigurationManager is in Associated

    This means the management pack is associated properly in batch 207 even though it ran into an error in batch 136. Because it recovered in batch 207, the management pack is correctly associated and the sync completed successfully. So even if a management packs is Pending Association in one batch, if in a later batch it is successfully Associated, then the management pack is correctly and completely deployed and synchronized.

    If in the MP Sync Job dialog, the Pending Association status for a management pack repeats for every batch, you’ll need to dig a bit deeper to figure out what causes this management pack to fail to associate.

     

    2. Look for deployment failures in management packs which this management pack depends on

    Click on the Data Warehouse > Management Packs and then click the Deployment Status column header in the main pane.  If you see any management pack whose deployment status is Failed and some are Not Started this is usually due to a management pack dependency. Because management packs can depend on others, any failure can have a domino effect and cause other management packs not able to deploy. Any impacted management pack will have a status of Not Started (it’s not Failed, but because another did Fail it hasn’t yet started).

     

    3. Find the deployment failures in the event log

     Go to the Data Warehouse Management Server, open the Operations Manager Event log, filter the event log to the events whose Event Source is Deployment and Event Level is Warning or Error. 

    If there is error message similar to the below highlighted text you’ll need to unregister the Data Warehouse from Service Manager, reinstall the Data Warehouse and then re-register Service Manager to Data warehouse.

                  Deployment Execution Infrastructure has retried the maximum number of times and is giving up on this execution step. 

           MP Element ID: DerivedManagementPack.SystemDerivedMp.ServiceManager.ActivityManagement.Library.Datawarehouse

           MP name: ServiceManager.ActivityManagement.Library.Datawarehouse

           MP version: 7.0.5826.0

           Operation: Install

                    Error message: Cannot find resource with ID TransformActivityStatusResource,

     

  • System Center Platform Team

    Troubleshooting the Data Warehouse: Job(s) fail after importing a custom management pack

    • 1 Comments

    This is the latest in “Troubleshooting the Data Warehouse” series. The goal of this series is to not only help you troubleshoot specific issues you may be experiencing but also teach you more about the data warehouse. For a complete list of posts in the series, check out Troubleshooting the Data Warehouse: An overview.

     

    The symptom

     

    One or more data warehouse jobs start failing after importing a custom management pack to Service Manager and synchronizing it to the Data Warehouse

    The troubleshooting steps

     

    To troubleshoot the above issue, please verify things in the following orders

    1. Check the event log to ensure the root cause is the custom management pack

    ·         In Data Warehouse Management Server, open Operations Manager event log.

    ·         Find the event which is related to this job failure.

    ·         Determine the failure is related to the custom management pack you just imported.

    2. Remove the custom management pack and recover from the failure

    If the failure is related to the custom management pack, you should remove it and let the rest of the data warehouse operate as usual. You can fix the MP and reimport it later.

    ·         Uninstall the custom management pack via the Service Manager console

    ·         Run MP Sync job

    ·         Verify that this custom management pack is not in the Data Warehouse > Management Packs

    ·         After MP Sync job is completed, resume the failed job either from UI or cmdlet

    ·         You should be good to go

    3. Fix and reimport the custom management pack

    ·         First you must remove the custom management pack and recover from the failure (step 2 above)

    ·         Fix the custom management pack

    ·         Import the custom management pack with fix into Service Manager, run MP Sync job to sync it to Data Warehouse

     

     

  • System Center Platform Team

    A Deep Dive on Creating Outriggers and Dimensions in the Data Warehouse

    • 1 Comments

    This blog post is intended for partners and customers who are interested in customizing the data warehouse and reporting capabilities in System Center Service Manager. An introduction and definition of key data warehouse MP elements will be given and examples will demonstrate how to define your own elements. 

    Service Manager provides a configuration driven data warehouse and reporting platform that can be extended by partners and end users. If this is the first post you are reading you might want to get a high level overview of the DW & Reporting Infra from the following posts:

    The System Center Platform in Service Manager Part 6: The Data Warehouse
    Data Warehouse and Reporting Overview
    Anatomy of Management Pack Synchronization
    Reporting Deployment
    Anatomy of Extract, Transform, Load (ETL)

    The primary source of data for the warehouse is Service Manager CMDB. As we know the data in CMDB is mainly driven by the model i.e. classes, relationships and enumerations.  This data is mapped into DW via a model of its own which is again driven by the same Management pack schema as that of CMDB, and is made up of Dimensions, Facts and Outriggers which loosely correlates to classes, relationships and enumerations and more.  

    We'll now take a closer look at the data warehouse elements that users can define.  This post will introduce Outriggers and Dimensions, and a follow-up post will introduce Relationship Facts.  These elements map very closely the industry standard commonly referred to as the Dimensional model.  In Service Manager they derive their identity and structure from the CMDB model and hence the data can be correlated back and forth between CMDB and DW.

    Outriggers

    An outrigger is a glorified “list” that can logically group together a set of values. Two examples are shown below which display a logical grouping of values for concept “priority” and for the Honda line-up of automobiles.

    PRIORITY

    Low
    Medium
    High

     

    Honda Automobiles
    Accord
    Civic
    CR-V
    Element
    Fit
    Odyssey

     

    For a user, the usefulness of an outrigger is demonstrated in 2 ways:

    1) The set of discrete values from an outrigger can be used as a drop down menu for a report parameter when creating and viewing reports in the SM Console.

    2) Outrigger values can be used to group data in reports for more advanced analysis.

    Outriggers in the data warehouse can target one or more class properties and consolidate them into a single set of discrete values.  These properties can only be of data type String or ManagementPackEnumeration.   When based on an enumeration, outriggers also preserve the hierarchy. An outrigger defined on any other data type other than above is not supported. 

    While the benefit of defining an outrigger on an enumeration is more obvious, the advantage of defining an outrigger on a string column is that the infrastructure will coalesce all the distinct values of a property from the instance space into a small list which can then be used in an easy to use drop down list in a report.  A good example of a string based outrigger is the Manufacturer property on the Computer class, which is modeled as a string in CMDB. By defining an Outrigger on that property we can now provide an ability to pick a value from the drop down rather than search on possible list of Manufacturers that one procured the computers from.

    Below is an example of how an outrigger is used in a report, in the parameter header. Here you see the ActivityStatus outrigger in Figure 1.  (Click to enlarge)

     

    clip_image001

    Figure 1: Activity Status Outrigger Parameter in the Activity Distribution report

     

    Here is how this outrigger was modeled:

    Consider the class System.WorkItem.Activity defined in the MP System.Workitem.Activity.Library:

            <ClassType ID="System.WorkItem.Activity" Accessibility="Public" Base="WorkItem!System.WorkItem" Hosted="false" Abstract="true">
    <
    Property ID="SequenceId" Type="int" />  
    <
    Property ID="Notes" Type="richtext" MaxLength="4000" />
     
    <
    Property ID="Status" Type="enum" EnumType="ActivityStatusEnum" />
     
    <
    Property ID="Priority" Type="enum" EnumType="ActivityPriorityEnum" />
     
    <
    Property ID="Area" Type="enum" EnumType="ActivityAreaEnum" />
     
    <
    Property ID="Stage" Type="enum" EnumType="ActivityStageEnum" />
                      
    </
    ClassType>

    Now we want an outrigger based on the enumeration property Status. Here is how you define it in a MP of your choice.

    <Outrigger ID="ActivityStatus" Accessibility="Public">
    <
    Attribute ID="Status" PropertyPath="$Context/Property[Type='CoreActivity!System.WorkItem.Activity']/Status$" />  
    </
    Outrigger>

    As mentioned above the MP author can define an outrigger on one or more class properties. Each class property will be modeled by a corresponding attribute in the outrigger.

    Now let us walk through an example of enumeration based outrigger viz. Activity Status based on ActivityStatusEnum.  Below is how the ActivityStatusEnum looks like. 

    <EnumerationTypes>
    <
    EnumerationValue ID="ActivityStatusEnum" Accessibility="Public" />
    <
    EnumerationValue ID="ActivityStatusEnum.Ready" Parent="ActivityStatusEnum" Accessibility="Public" Ordinal="5.0" />
    <
    EnumerationValue ID="ActivityStatusEnum.Active" Parent="ActivityStatusEnum" Accessibility="Public" Ordinal="10.0" /> 
    <
    EnumerationValue ID="ActivityStatusEnum.OnHold" Parent="ActivityStatusEnum" Accessibility="Public" Ordinal="15.0" />
    <
    EnumerationValue ID="ActivityStatusEnum.Completed" Parent="ActivityStatusEnum" Accessibility="Public" Ordinal="20.0" />
    <
    EnumerationValue ID="ActivityStatusEnum.Failed" Parent="ActivityStatusEnum" Accessibility="Public" Ordinal="25.0" />
    <EnumerationValue ID="ActivityStatusEnum.Cancelled" Parent="ActivityStatusEnum" Accessibility="Public" Ordinal="30.0" />
    <
    EnumerationValue ID="ActivityStatusEnum.Rerun" Parent="ActivityStatusEnum" Accessibility="Public" Ordinal="35.0" />
     
    ...

    </
    EnumerationTypes>  

    Each of these values will now be included in the outrigger set of discrete values.  The following screenshot displays the column ID and ActivityStatusValue from the ActivityStatus outrigger which contains all the enumeration values from ActivityStatusEnum.

     

    clip_image003

    Figure 2:  ActivityStatus Outrigger containing all the enumeration values of ActivityStatusEnum

     

    Looking at the figure above, the ID column from the outrigger will contain all the EnumerationValue ID’s from the ActivityStatus enumeration type.  The ActivityStatusValue is the actual user-friendly display name that is displayed in the report drop-down menus.

    Now that we have a basic understanding of how outriggers relate to enumerations and strings, let’s go into further detail on how we construct and model an outrigger.  We again use the outrigger ActivityStatus as an example:


    <
    Outrigger ID="ActivityStatus" Accessibility="Public">
    <
    Attribute ID="Status" PropertyPath="$Context/Property[Type='CoreActivity!System.WorkItem.Activity']/Status$" />  
    </
    Outrigger>

    First, let’s look at the <Outrigger> parent tag.  The following table describes the attributes for this tag. 

    <Outrigger> Tag Attributes

    Attribute Description
    ID A unique identifier for the outrigger element.  This will also be the table name of the outrigger in the data warehouse and data mart.
    Accessibility This element should always be set to “Public”.

    Each <Outrigger> parent tag will contain one or more <Attribute> sub-element tags.  The following table describes the attributes for this tag.

    <Attribute> Tag Attributes

    Attribute Description
    ID A unique identifier for each outrigger attribute
    PropertyPath PropertyPath syntax which must uniquely identify the class and attribute that the outrigger attribute is targeting.

     

    Dimensions

    Suppose a user wants a report in SM to display some information about the attributes for the computers in a particular domain.  For example, the user may want to know the IP address, number of logical processors, and DNS name for each computer.    Using dimensions, the user would be able to bring this data over from SM to the DW where reports can query and display this data for each computer.

    A dimension in the SM data warehouse is roughly analogous to a MP class.  Each MP class has a list of properties, while each dimension contains a list of attributes.  Each dimension attribute will map to one property in a class. 

    In the SM DW, a dimension always targets a single class. The dimension attributes will then map to the target class’s properties.  So from the example above, in order to get the information about the attributes from a computer, we would have a computer dimension targeted at the Microsoft.Windows.Computers class. 

    In certain cases that will be described in further detail below, a dimension may also map to the properties of a target class’s base and derived classes.  So while a dimension may be roughly analogous to a MP class, it can also contain properties which are within that MP class’s hierarchy.

    Below is an example of how a dimension is used in an Activity Distribution report.  A dialog box is used to search for dimension instances in the ConfigItemDim dimension, where you can filter on the Display Name property.  The results shown below filter on the keyword “Computer” in Figure 3. (Click to Enlarge)

     

    clip_image005

    Figure 3: ConfigItemDim report parameter filtering on the keyword “Computer” in the Display Name property

     

    After choosing “All Windows Computer” as the dimension object, the report header is updated with the selected filter value: (Click to Enlarge)

     

    image

    Figure 4: Activity Distribution report parameter header with "All Windows Computers” selected as a filter

     

    When the report is rendered, only activities that affect the selected configuration item “All Windows Computers” are displayed.

    Let us now take a look at how the dimension was modeled.

    First, consider the class System.Entity and System.ConfigItem classes defined in the MP System.Library:

    <ClassType ID="System.Entity" Accessibility="Public" Hosted="false" Abstract="true" Singleton="false">  
    <
    Property ID="DisplayName" Type="string" MinLength="0" Key="false" CaseSensitive="false" MaxLength="4000" /> 
     
    </
    ClassType>

    <ClassType ID="System.ConfigItem" Base="System.Entity" Accessibility="Public" Hosted="false" Abstract="true"> 
    <
    Property ID="ObjectStatus" Type="enum" EnumType="System.ConfigItem.ObjectStatusEnum" DefaultValue="System.ConfigItem.ObjectStatusEnum.Active" />
     
    <
    Property ID="AssetStatus" Type="enum" EnumType="System.ConfigItem.AssetStatusEnum" />
     
    <
    Property ID="Notes" Type="richtext" MaxLength="4000" />
     
    </
    ClassType>

    We want the configuration item dimension to point at the ObjectStatus and AssetStatus properties of System.ConfigItem as well as the DisplayName property of the base class System.Library.  Here is how you define the dimension with only these 3 properties as attributes:

    <Dimension ID="ConfigItemDim" Accessibility="Public" Target="System!System.ConfigItem" InferredDimension="true" HierarchySupport="Exact" Reconcile="true">  
    <
    InclusionAttribute ID="DisplayName" PropertyPath="$Context/Property[Type='System!System.Entity']/DisplayName$" SlowlyChangingAttribute="false" />
     
    <
    InclusionAttribute ID="ObjectStatus" PropertyPath="$Context/Property[Type='System!System.ConfigItem']/ObjectStatus$" SlowlyChangingAttribute="false" /> 
     
    <
    InclusionAttribute ID="AssetStatus" PropertyPath="$Context/Property[Type='System!System.ConfigItem']/AssetStatus$" SlowlyChangingAttribute="false" />
     
    </
    Dimension>

    Let’s go into further detail on how we construct and model a dimension by examining the xml schema elements and attributes for a <Dimension>:

    <Dimension> Tag Attributes

    Attribute

    Description

    ID

    A unique identifier for the dimension element.  This will also be the table name of the dimension in the data warehouse and data mart.

    Accessibility

    This element should always be set to “Public”

    Target

    The MP class name which the dimension is targeting

    InferredDimension

    For v1, this value is always to true.

    HierarchySupport

    The hierarchy of classes which will help define the properties that will be included in the dimension.  There are 3 possible values:
    1)Exact
    2)IncludeDerivedClassProperties
    3)IncludeExtendedClassProperties
    Please see below for more details.

    Extends

    Optional boolean flag to indicate whether the dimension is a base dimension or is extending another dimension. After a dimension has been defined, the SM DW allows you to “extend” the dimension and add more attributes at a later point in time.

    If the Extends flag is set to “true”, the HierarchySupport must be set to “Exact” and all the extension attributes must be listed.  By default, this flag will be set to false.

    Reconcile

    Optional boolean flag that to indicate whether two instances which are otherwise identical and only differ from which source the data originated should be coalesced into one single row of data. By default, this flag will be set to false. 

    Configuration Item related dimensions should have this flag set to true, and Work Item related dimensions will have this flag set to false.  

     

    <HierarchySupport>

    The HierarchySupport attribute determines which classes are processed and the specific attributes that are included in the dimension. We’ll now examine each possible value in detail:

    Exact – When the HierarchySupport attribute is Exact, the user must manually define each attribute that should be included within the dimension using the <InclusionAttribute> tag.  These attributes can be either from the target class or any of the target class’s base and derived classes. 

    Each inclusion attribute will correspond to one class property.  Let’s look at each of the attributes within the <InclusionAttribute> tag:

    <InclusionAttribute> Tag Attributes

    Attribute

    Description

    ID

    A unique identifier for the attribute element. 

    PropertyPath

    PropertyPath syntax which must uniquely identify the class and attribute that the dimension attribute is targeting.

    SlowlyChangingAttribute

    In v1, this attribute should always be false

     

    The ConfigItemDim dimension example above had a HierarchySupport value of Exact.  Therefore, only the listed inclusion attributes (DisplayName, ObjectStatus, AssetStatus) will be processed in the transform and included in the dimension table in the DW repository and Datamart.

    The Exact HierarchySupport value requires the user to manually list each attribute they want in the dimension.  Users, however, will often want all the attributes for a class as well as attributes from its base and/or derived classes to be included within the dimension.  In these cases, it takes a lot of effort to list each attribute explicitly.  Therefore, we have introduced two other HierarchySupport values that will automatically handle these cases for the user.

    IncludeExtendedClassProperties – For a dimension with a HierarchySupport of IncludeExtendedClassProperties, all the attributes of the target class and ALL of its base classes will be included in the dimension table and transform. 

    Let’s take a look at the following example of CarDimension, which targets the class Car and has a HierarchySupport of IncludeExtendedClassProperties

    clip_image009

    Figure 5: CarDimension with HierarchySupport of IncludeExtendedClassProperties targeting the Car Class

     

    Since CarDimension targets the Car class and has a HierarchySupport value of IncludeExtendedClassProperties, it will process both the Car class and its base class, Vehicle.  The resulting table and transform will contain the following attributes:

    CarDimension Attributes

    Color
    Make
    Model
    NumDoors
    NumCupHolders
    Horsepower
    CargoSpace

    IncludeDerivedClassProperties  – For a dimension with a HierarchySupport of IncludeDerivedClassProperties, all the attributes of the target class, its base classes, and its derived classes will be included in the dimension table and its associated transform. 

    Slightly modifying the previous example, CarDimension now has a HierarchySupport of IncludeDerivedClassProperties below.  Since it will process both base and derived classes of the target class, the dimension will now process the attributes of 3 classes – Vehicle, Car, and Sportscar as shown below.

    clip_image011

    Figure 6: CarDimension with HierarchySupport of IncludeDerivedClassProperties targeting the Car Class

     

    The CarDimension dimension table and transform will contain the following attributes:

    CarDimension Attributes
    Color
    Make
    Model
    NumDoors
    NumCupHolders
    Horsepower
    CargoSpace
    TopSpeed

    Finally, here are some miscellaneous tips and tricks:

    Tips and Tricks

    • For Service Manager Data Warehouse v1, all user defined data warehouse elements must be in sealed management packs.

    • Dimensions with a HierarchySupport value of IncludeDerivedClassProperties cannot target abstract classes

    • For a non-exact dimension, inclusion attributes cannot be defined. The processing of that dimension will automatically determine which attributes are included in the dimension table schema and transform.

    • Users can also define exclusion attributes so that unneeded data is not transformed.  In some cases, it may be easier for the user to define a dimension with a non-exact HierarchySupport, and then define a few exclusion attributes to preclude any irrelevant data.

    • Exclusion attributes cannot be defined for a Dimension that has a HierarchySupport value of Exact, unless the Dimension is an extension dimension.

    • All extension dimensions must have a HierarchySupport value of Exact

    More information about dimension extensibility will be added in a future post, where we will explain both how our data warehouse will be able to automatically extend dimensions without any user input to correlate with class extensions, and how users can also manually extend the dimension schema themselves.

  • System Center Platform Team

    A Deep Dive on Creating Relationship Facts in the Data Warehouse

    • 1 Comments

    This blog post is intended for partners and customers who are interested in customizing the data warehouse and reporting capabilities in System Center Service Manager. An introduction and definition of key data warehouse MP elements will be given and examples will demonstrate how to define your own elements. 

    This blog post is a continuation from our previous post which introduced outriggers and dimensions.  In this blog post, we will take a look at how the user can define relationship facts in the data warehouse.

    Relationship Facts

    A relationship fact in the SM DW is analogous to a relationship in SM. A relationship fact can be used to help answer queries such as:

    • Which work items are currently assigned to the user “Joe Smith” so we can check their status?
    • I need the list of all the computers in our domain which currently have Windows 7 installed so I can update them to SP1.
    • I need to find all Review Activities that lists Sarah Smith as a reviewer so they can be reassigned since she is on vacation.

    In each of these scenarios, there is a source instance and a target instance that is joined together by some relationship.  Without a relationship fact, it would be difficult to determine these associations between instances.

    Consider the relationship Microsoft.Windows.ComputerHostsOperatingSystem in the MP Microsoft.Windows.Library:

    <RelationshipType ID="Microsoft.Windows.ComputerHostsOperatingSystem" Accessibility="Public" Base="System!System.Hosting">  
    <
    Source ID="Computer" Type="Microsoft.Windows.Computer" />  
     
    <
    Target ID="OperatingSystem" Type="Microsoft.Windows.OperatingSystem" MaxCardinality="1" />
     
    </
    RelationshipType>

    In a SM relationship, the source and target are always modeled by a MP class. In this relationship, the class Microsoft.Windows.Computer is the source and the class Microsoft.Windows.OperatingSystem is the target.

    Here’s how you define the corresponding RelationshipFact based on the Microsoft.Windows.ComputerHostsOperatingSystem relationship:

    <RelationshipFact ID="ComputerHostsOperatingSystemFact" Accessibility="Public" Domain="Domain.ConfigurationManagement" TimeGrain="Daily" SourceType="Windows!Microsoft.Windows.Computer" SourceDimension="ComputerDim">  
    <
    Relationships RelationshipType="Windows!Microsoft.Windows.ComputerHostsOperatingSystem" TargetDimension="OperatingSystemDim" />
     
    </
    RelationshipFact>

    Notice how the relationship fact defines a source dimension and a target dimension.  If you look closely, you will notice that the source and target dimension target the source and target classes from the original relationship that the relationship fact is modeled upon.

    In the previous section, we learned about dimensions.  Relationship facts really come into use by associating two dimensions together, allowing reports to take advantage of this association to display important information from each dimension in relation to each other.   For example, the relationship WorkItemAssignedToUser can be used to display information about incidents or change requests for a specific user in the report.  This allows the customer to drill down into the data to find information specific to their needs.  This is just one example how relationship facts are useful to create specialized views of data in reports.

    Let’s now take a deeper look at the attributes and sub-element tags required to model a relationship fact in a user-defined MP.  The first table describes all the attributes for the <RelationshipFact> tag

    <RelationshipFact> Tag Attributes

    Attribute

    Description

    ID

    A unique identifier for the relationship fact element.  This will also be the table name of the relationship fact in the data warehouse and data mart.

    Accessibility

    This element should always be set to “Public”, as the deployment process will create system derived MP’s which will refer to this outrigger during the generation of the automated transforms. 

    Domain

    The scope of the relationship fact.  Possible values include:
    Instance Management
    Activity Management
    Incident Management
    Change Management
    Problem Management

    The value for this attribute must be an enumeration that is a child of the parent “Domain” enumeration defined in the MP Microsoft.SystemCenter.Datawarehouse.Base

    TimeGrain

    The granularity of the relationship fact.  Value must be one of the following:
    Hourly
    Daily
    Weekly
    Monthly

    SourceType

    The management pack class for the source of the relationship.

    SourceDimension

    The dimension which targets the source class.  This is an optional field. If no SourceDimension is specified, we will automatically find the dimension which directly targets the source class itself, or the closest parent class of the source class in the class hierarchy.

    Each relationship fact will contain one or more <Relationships> sub-tags.  The following table describes the attributes within the <Relationships> tag..

    <Relationships> Tag Attributes

    Attribute

    Description

    RelationshipType

    The relationship between the source and target.  A list of relationships can be found in the inbound.relationships table in the data warehouse.

    TargetDimension

    The dimension which targets the target class.

    In a multiple relationship fact, the source dimension will always remain the same, while the target dimension can change, depending on the specific relationship.  Every relationship type attribute in a multiple relationship fact must be unique. The following is an example of the relationship fact WorkItemAssignedToAndCreatedByUser:

    <RelationshipFact ID="WorkItemAssignedToAndCreatedUserFact" Accessibility="Public" Domain="Domain.InstanceManagement" TimeGrain="Daily" SourceType="WorkItem!System.WorkItem" SourceDimension="WorkItemDim">
    <
    Relationships RelationshipType="WorkItem!System.WorkItemAssignedToUser" TargetDimension="UserDim" />
    <
    Relationships RelationshipType="WorkItem!System.WorkItemCreatedByUser" TargetDimension="UserDim" />
    </
    RelationshipFact>

    In the example above, we see that although the target dimension is identical for both relationships, the relationships themselves are unique so the relationship fact is valid.

    For more examples of outriggers, dimensions, and relationship facts, you are encouraged to look at any of the data warehouse MP’s that are shipped with Service Manager.  A great place to start is the base data warehouse management pack : Microsoft.SystemCenter.Datawarehouse.Base

  • System Center Platform Team

    Overview of Relationship Types in the System Center Platform

    • 1 Comments
     

    The System Center Platform is model based, where classes and their relationships are defined in a management pack. In this blog post I will cover an overview of the four basic relationship types available in the platform.

    The inheritance hierarchy of the four relationships is as follows:

    Reference Relationship

     

    This is the base class of all the relationship types and is the most simple relationship type.  It is used as follows in the MP:

            <RelationshipType ID="WorkItemRelatedToConfigItem" Base="System.Reference" Abstract="false" Accessibility="Public">

              <Source ID="WI" Type="WorkItem" />

              <Target ID="CI" Type="System.ConfigItem"/>

            </RelationshipType>

    There are no specific rules to be followed while committing instances of this relationship type.

    Containment Relationship

     

    This relationship is derived from reference relationship. It is used as follows in the MP:

            <RelationshipType ID="QueueContainsWorkItem" Base="System.Containment" Abstract="false" Accessibility="Public">

              <Source ID="Queue" Type="Queue"/>

              <Target ID="WI" Type=" WorkItem"/>

            </RelationshipType>

    Properties:

    1.       Security flows through the contained object. For example, if there are four work items contained in Queue1 and User x has been granted permissions on this queue, then the same user will automatically get permissions to all the work items in that queue.

    2.       Target class can be used as endpoint of any other relationships types. For example in the example mentioned above, the WorkItem class which is target of the relationship can also be used in other reference or membership relationships too.

    There are no specific rules to be followed while committing instances of this relationship type.

    Membership Relationship

     

    This relationship is derived from containment relationship, so it has the properties of the two relationship types mentioned above. It is used as follows in the MP:

            <RelationshipType ID="WIToTask" Base="System.Membership" Abstract="false" Accessibility="Public">

              <Source ID="WI" Type=" Scenario "/>

              <Target ID="Task" Type="WorkItem"/>

            </RelationshipType>

    Properties:

    1.       Security flows through the contained object (same as containment)

    2.       Target class can be used as endpoint of other relationships types. (same as containment)

    3.       The target of this relationship type has a lifetime dependency with the source. In other words, once a membership relationship is established between a source and target, the target’s existence in the system is bound to the source, such that a deletion of the source results in a deletion of the target (but not vice versa).

    While committing a membership relationship object you would need to make sure that the target object and the relationship object are committed in the same transaction. Let’s look at an example here

    //Create the source object and commit it

          ManagementPackClass classWorkItem = ManagementGroup.EntityTypes.GetClass(“WorkItem”, testMP)

          CreatableEnterpriseManagementObject workItemObject = new CreatableEnterpriseManagementObject(ManagementGroup,classWorkItem);

          workItemObject [classWorkItem, "Id"].Value = Guid.NewGuid().ToString();

          workItemObject [classWorkItem, "Title"].Value = “Hello World”;

          workItemObject.Commit();

     

          //Create the target object, but since u want to relate this object to the source object and the relationship type is membership do not commit the object

          ManagementPackClass classTask = ManagementGroup.EntityTypes.GetClass(“Task”, testMP)

          CreatableEnterpriseManagementObject taskObject = new CreatableEnterpriseManagementObject(ManagementGroup, classTask);

          taskObject [classTask, "Id"].Value = Guid.NewGuid().ToString();

          taskObject [classTask, "Title"].Value = “Hello World”;

         

          //now create the the relationship object

         CreatableEnterpriseManagementRelationshipObject relationshipObject = new CreatableEnterpriseManagementRelationshipObject(ManagementGroup,         

         ManagementGroup.EntityTypes.GetRelationshipClass(“WorkItemToTask”, testMP));

         relationshipObject.SetSource(workItemObject); 

         relationshipObject.SetTarget(taskObject);

          

         //since this is a membership relationship type, you would need to commit the target and the relationship object in the same transaction,like follows            

         IncrementalDiscoveryData dd = new IncrementalDiscoveryData();

         dd.Add(taskObject); //target object

         dd.Add(relationshipObject); //relationship object

         dd.Commit();

     

    If the membership relationship object is committed without the target object being in the same transaction, then SDK will throw the following exception while committing the relationship

     

    DiscoveryDataLifetimeDependencyException - A discovery data item was rejected because the item is already bound to another Membership relationship.

     

    You could insert the target object independent of the relationship object and no sdk error will be thrown. However, if you try to bind this object to a membership relationship later, sdk will throw the same exception as above, since the object was already inserted before the relationship. So for membership relationship it’s very important to have both the target object and relationship object being inserted in the same transaction.

     

    Hosting Relationship

     

    This relationship type is derived from the membership relationship type. For using this relationship in the MP, it is required that the target class of this relationship is marked as Hosted=true as shown below:

           <ClassType ID="SQLServerInstance" Accessibility="Public" Base="System.Entity" Hosted="true" Abstract="false">

                <Property ID="SQLId" Type="string" Key="true" />

                <Property ID="Name" Type="string" />

                <Property ID="Comment" Type="string" />

            </ClassType>

     

     

            <RelationshipType ID="ComputerToSQLInstance" Base="System.Hosting" Abstract="false" Accessibility="Public">

              <Source ID="Comp" Type="Computer"/>

              <Target ID="SQL" Type=" SQLServerInstance"/>

            </RelationshipType>

    Here if you don’t attribute the target class as hosted true, then your MP will fail to import.

     

    Properties:

    1.       Security flows through the contained object (same as containment)

    2.       The target of this relationship has lifetime dependency with the source (same as membership)

    3.       Source class of this relationship has to be a non-abstract class

    4.       Target class shares (partly) the identity of the parent class and hence cannot exist independent of the parent class

    5.       Target class can belong to one and only one hosted relationship

    Hosting relationship is implicitly created when a hosted object is committed; there is no need to explicitly commit a hosting relationship object. For example, if you want to create a SQLServerInstance object related to Computer_XYZ object. Then you would do the following:

     

          ManagementPackClass classComputer = mgmtGrp.EntityTypes.GetClass(“Computer”, sdkTestMP);

    ManagementPackClass classSQLServerInstance = mgmtGrp.EntityTypes.GetClass(“SQLServerInstance”, sdkTestMP);

    CreatableEnterpriseManagementObject sqlServerObject= new CreatableEnterpriseManagementObject(ManagementGroup, classComputer);

    sqlServerObject [classComputer, "Id"].Value = classBObjectId;  // here we are setting the key properties of the parent class

    sqlServerObject [classSQLServerInstance, " SQLId"].Value = Guid.NewGuid().ToString(); //here we are setting the key properties of hosted class

    sqlServerObject.Commit(); //on this commit, the object is commited and implicitly the hosting relationship is also set between the 2 objects

     

    As seen above, you don’t need to explicitly commit a hosting relationship object, as you would need to do for other relationships types.

     

    SDK will throw the following exception, if commit is called on a hosted object without the parent key properties filled.

     

    System.InvalidOperationException: Either the form is missing some values or the class that created it no longer exists. Make sure all key values are populated and submit again.

     

  • System Center Platform Team

    Troubleshooting the Data Warehouse: Data Warehouse isn’t getting new data or jobs seem to run forever

    • 1 Comments

    This is the latest in “Troubleshooting the Data Warehouse” series. The goal of this series is to not only help you troubleshoot specific issues you may be experiencing but also teach you more about the data warehouse. For a complete list of posts in the series, check out Troubleshooting the Data Warehouse: An overview.

     

    The symptom

     

    I don’t see data or new data in my reports or ETL Jobs look like running forever and never in Not Started status.

     

    The troubleshooting steps

     

    To troubleshoot the above issue, please verify things in the following orders

    1. Use the Get-SCDWJob PowerShell cmdlet to determine if all ETL jobs are enabled

    Start PowerShell and type Get-SCDWJob

    ·         If the ETL jobs are disabled and either MPSyncJob or DWMaintenance job is running, you’ll need to wait a little bit to get the job status again (because these two background jobs disable the ETL jobs).  However, if these two jobs are Not Started and the ETL jobs are disabled, you can use the Enable-SCDWJob cmdlet to enable each of them. For example

    Enable-SCDWJob -JobName Transform.Common

     

    ·         If ETL jobs, MPSync job and DWMaintenance job are all enabled and running, but their individual batch id has not changed for a long time or use Get-SCDWJobModule for specific job, we don’t see any module actually is running, check the event log and see if there is any error message. Sometimes, the error message might be many days old, we need to dig deep in the old day events.

     

    ·         Check if three services (System Center Data Access Service, System Center Management, System Center management Configuration) in Data warehouse Management server are actually running.

    In Data Warehouse Management Server, click Start > Run .., enter Services.msc, in the Services window, verify if the following services are running or not

    o   System Center Data Access Service

    o   System Center Management

    o   System Center management Configuration

    If any of them are not running, restart all three services. In addition, if all services are actually running, events from Event Source Data Warehouse and OpsMgr SDK Service are outputted to Operations Manager event log. We can use this information as another source to verify if all services are running or not. If you don’t see events from Event Source Data Warehouse and OpsMgr SDK Service for a while, restart all three services regardless.

     

  • System Center Platform Team

    Service Manager 2010 SDK Documentation Released!

    • 1 Comments

    The Service Manager 2010 Software Development Kit (SDK) documentation is now released on MSDN.  There are some good code samples and most importantly all the reference documentation for the API layer.

    MSDN also now supports community contributions.  Please feel free to add your own comments, examples, code samples, etc!

    http://msdn.microsoft.com/en-us/library/gg469708.aspx

  • System Center Platform Team

    Using ObjectQueryOptions – Configuring Object retrieval from the CMDB

    • 0 Comments

    What are some of the things you would want to think about before you query and retrieve objects from the CMDB? Here are some of them,

    a.      How many objects are you expecting in the database for your criteria? How will the count affect your read performance?

    b.      Do you want your results sorted?

    c.      Do you want your results filtered by the source that created it?

    d.      Do you want all your objects returned or only the TOP N results?

    e.      Do you want all object properties returned or only some of them?

    All Service Manager V1 APIs used for retrieving objects from the database take a parameter of type Microsoft.EnterpriseManagement.Common.ObjectQueryOptions. This class exposes several APIs that can be used to configure all of the above and more.

    This is the first in a series of posts that I will be writing to go over each of the points above and see how we can use ObjectQueryOptions to achieve what we want.

     

     

    Scenario 1: Large number of instances in the database; how to improve read performance

    Suppose you have a large number of objects(1000+) in the CMDB that match your criteria; you may not want to or more importantly need to read the objects from the CMDB all at once, but instead retrieve only the first N objects and have the remaining objects load in the background (load on demand).

    ObjectQueryOptions exposes a public property called ObjectRetrievalMode which is an enum that you can set. This property indicates whether you want the objects returned all at once (in a single batch) or in several small sized batches (aka pages)

    /// <summary>

    /// specifies whether the object retrieval from

    /// server should be buffered or not

    /// </summary>

    public enum ObjectRetrievalOptions

    {

        /// <summary>

        /// does not buffer the data

        /// </summary>

        NonBuffered,

     

        /// <summary>

        /// buffers the data

        /// </summary>

        Buffered

    }

     

    NonBuffered – you get all your objects returned in one batch.

    Buffered – you’ll get your objects returned in batches ( you can specify the size of the batches as well). The default batch size is 250

     

    Lets take a look at an example of how we can actually get back objects from the CMDB in batches. In the example below I am querying for  all of the objects of class Microsoft.Windows.Computer but accessing  only the first object.

     

    Code snippet:

    EnterpriseManagementGroup emg = new EnterpriseManagementGroup("sm-server");

    ManagementPack mp = emg.ManagementPacks.GetManagementPack("Microsoft.Windows.Library", keyToken, new Version("7.5.1049.1");

    ManagementPackClass computer = emg.EntityTypes.GetClass("Microsoft.Windows.Computer", mp);

    ObjectQueryOptions queryOpt = new ObjectQueryOptions();

    queryOpt.ObjectRetrievalMode = ObjectRetrievalOptions.Buffered;

     

    IObjectReader<EnterpriseManagementObject> instanceReader = emg.EntityObjects.GetObjectReader<EnterpriseManagementObject>( computer, queryOpt);

     

    // get the first object that matches the criteria

    instanceReader.GetData(0);

     

     

    NOTE: In the above example, even though I am accessing only the first object, the reader internally retrieves a the first page of instances from the CMDB.i.e the reader always retrieves enough instances from the CMDB to fulfill a client request, but at a minimum it gets back a page of instances from the CMDB.

     

    In SMv1, the buffered/ non-buffered mode of retrieval can be used for retrieving objects and projections. The APIs return either IObjectReader<EnterpriseManagementObject> or IObjectProjectionReader<EnterpriseManagementObject> depending on whether instances or projection were queried for.

     

    The batch or page size is the number of objects retrieved from the database at a given time. It can be set using the property on IObjectReader<T> or IObjectProjectionReader<T>.  The default page size is set to 250. It can be set to any acceptable value for an unsigned integer. If it is set to number greater than the actual number of instances matching a query, then only the available matching instances are brought back.

     

    int PageSize

    {

     

     get;

     set;

     

    }

     

     

    The instance and projection readers have many more powerful features that can be used to do a wide range of things to query for  instances from the CMDB. I will cover all the remaining features in a separate blogpost.

     

     

  • System Center Platform Team

    Using Under Operator for List Queries

    • 0 Comments

    Suppose you want to query all hardware related incidents using the following criteria: 

    <Criteria xmlns="http://Microsoft.EnterpriseManagement.Core.Criteria/">
      <Expression>
        <SimpleExpression>
          <ValueExpressionLeft>
            <Property>$Context/Property[Type='System.WorkItem.Incident']/Classification$</Property>
          </ValueExpressionLeft>
          <Operator>Equal</Operator>
          <ValueExpressionRight>
            <Value>{9C46171F-64A2-B33A-FBD3-D3ED9881CDF6}</Value>
          </ValueExpressionRight>
        </SimpleExpression>
      </Expression>
    </Criteria>
     
     

    In the above, "9C46171F-64A2-B33A-FBD3-D3ED9881CDF6" is the ID of list "IncidentClassificationEnum.Hardware". This criteria works as expected as long as "IncidentClassificationEnum.Hardware" does not have any child lists appended.  However, you may want to customize this list by adding more specific information, such as specifying what kind of hardware is related to incidents. Suppose this list hierarchy is changed as follows:

    IncidentClassficationEnum.Hardware
    |- IncidentClassficationEnum.Hardware.Disk
       |- IncidentClassficationEnum.Hardware.Disk.HardDisk
       |- IncidentClassficationEnum.Hardware.Disk.CD
    |- IncidentClassficationEnum.Hardware.Memory
    |- IncidentClassficationEnum.Hardware.CPU

    Then you can create incidents and set the classification property as one of the above child lists.  In this case, in order to get all hardware related incidents, you need to adapt your query criteria.  In Service Manager 2010, your option was to use the 'In' operator:

    <Criteria xmlns="http://Microsoft.EnterpriseManagement.Core.Criteria/">
      <Expression>
        <In>
          <Property>$Context/Property[Type='System.WorkItem.Incident']/Classification$</Property>
          <Values>
            <Value>9C46171F-64A2-B33A-FBD3-D3ED9881CDF6</Value>
            <Value>3D5430DC-EA64-C457-70D8-11D8257E0B58</Value>
            <Value>3C272B6C-A330-9135-E836-4E787EDE8C7C</Value>
            <Value>6185BF42-FBBA-4339-9AA9-DA0FDE48D881</Value>
            <Value>E1ECB59B-A9F0-9ED4-B675-B4C1C8CE97FD</Value>
            <Value>BCA04F7A-41E6-B053-6E40-EF4B02FDCCFD</Value>
          </Values>
        </In>
      </Expression>
    </Criteria>
     

    In the above query, you have to collect all child list Ids as well as the parent list Id. Further, whenever you add more child lists, you need to change the criteria correspondingly.  In Service Manager 2010 SP1, we introduce a new operator "Under" in which you only need to specify the root list ID. The criteria keeps working if any child item is added, deleted or updated from the corresponding hierarchy.

    <Criteria xmlns="http://Microsoft.EnterpriseManagement.Core.Criteria/">
      <Expression>
        <SimpleExpression>
          <ValueExpressionLeft>
            <Property>$Context/Property[Type='System.WorkItem.Incident']/Classification$</Property>
          </ValueExpressionLeft>
          <Operator>Under</Operator>
          <ValueExpressionRight>
            <Value>{9C46171F-64A2-B33A-FBD3-D3ED9881CDF6}</Value>
          </ValueExpressionRight>
        </SimpleExpression>
      </Expression>
    </Criteria>

    The under operator won't filter out the root list by default.  The results of using the above criteria include all incidents whose classification is equal to IncidentClassificationEnum.Hardware. If you only want to get incidents related to specific hardware, then you have to exclude the root list explicitly: 

     <Criteria xmlns="http://Microsoft.EnterpriseManagement.Core.Criteria/">
      <Expression>
        <And>
          <Expression>
            <SimpleExpression>
              <ValueExpressionLeft>
                <Property>$Context/Property[Type='System.WorkItem.Incident']/Classification$</Property>
              </ValueExpressionLeft>
              <Operator>NotEqual</Operator>
              <ValueExpressionRight>
                <Value>{9C46171F-64A2-B33A-FBD3-D3ED9881CDF6}</Value>
              </ValueExpressionRight>
            </SimpleExpression>
          </Expression>
          <Expression>
            <SimpleExpression>
              <ValueExpressionLeft>
                <Property>$Context/Property[Type='System.WorkItem.Incident']/Classification$</Property>
              </ValueExpressionLeft>
              <Operator>Under</Operator>
              <ValueExpressionRight>
                <Value>{9C46171F-64A2-B33A-FBD3-D3ED9881CDF6}</Value>
              </ValueExpressionRight>
            </SimpleExpression>
          </Expression>
        </And>
      </Expression>
    </Criteria>
     

    This new operator has not been supported in UI criteria builder.  In order to use it in SM console, you need to do the changes manually.  Suppose you create a view to list all hardware incidents as shown below:

     

    Notice that you cannot choose the 'Under' operator here.  Instead, select the 'equals' operator for now. Next, export the management pack where you saved this view and edit the view criteria to change 'Equal' operator to 'Under'.  After importing the updated management pack, click the above view and you are able to see all hardware related incidents there.

    It is important to note that if you open the "edit view" dialog for the changed view, it still shows the 'Equals' operator in the criteria area.  This means that if you make any subsequent changes to the criteria of the view through the console, it will overwrite the 'Under' operator and you will have to manually edit the criteria again.

  • System Center Platform Team

    Using ObjectQueryOptions – Configuring Object Retrieval from the CMDB, Part II

    • 0 Comments

    Using ObjectQueryOptions – Configuring Object Retrieval from the CMDB Part II

     

    This is a continuation of my previous post on Microsoft.EnterpriseManagement.Common.ObjectQueryOptions. The previous post introduced the basics of object retrieval from the CMDB. In this post, I want to explain how you to get back results from the CMDB sorted in some order using ObjectQueryOptions.

     

    Objects of a class or projection can be retrieved from the CMDB sorted by properties of the object.

    There are two kinds of properties in every object. Those that are defined on a class in a management pack and those that are applicable to all objects for e.g LastModified, TimeAdded etc..

     

    The properties applicable to all objects are called generic properties. The entire list of generic properties is below:

     

    namespace: Microsoft.EnterpriseManagement.Configuration

    public enum.EnterpriseManagementObjectGenericPropertyName

    {

            Id = 0,

            Name = 1,

            Path = 2,

            FullName = 3,

            DisplayName = 4,

            LastModified = 5,

            TimeAdded = 6,

            LastModifiedBy = 7

    }

     

     

    Case 1: Sorting objects of a class

     

    The relevant API on ObjectQueryOptions  for sorting class objects

     

    /// <summary>

    /// Used for sorting isntances by property

    /// </summary>

    /// <param name="property">Property to sort on</param>

    /// <param name="sortOrder">Sort Order</param>

    public void AddSortProperty(ISortableProperty property, SortingOrder sortOrder)

     

    The above API is used to specify the sorting on objects of a class retrieved from the CMDB.

    The property parameter can of type

     Microsoft.EnterpriseManagement.Configuration. ManagementPackProperty

     

    Or

     

    Microsoft.EnterpriseManagement.Configuration.EnterpriseManagementObjectGenericProperty

     

    Let’s look at an example of retrieving all available Microsoft.Windows.Computer from the CMDB sorted by ascending order of property NetworkName (defined on class Microsoft.Windows.Computer) and ascending order of generic property TimeAdded

    EnterpriseManagementGroup emg = new EnterpriseManagementGroup("sm-server");

    ManagementPackClass mp = emg.ManagementPacks.GetManagementPack("Microsoft.Windows.Library", keyToken, new Version("7.5.1049.1");

    ManagementPackClass computer = emg.EntityTypes.GetClass("Microsoft.Windows.Computer", mp);

    ObjectQueryOptions queryOptions = new ObjectQueryOptions(ObjectPropertyRetrievalBehavior.All);

     

    // sort based on NetworkName

    ManagementPackProperty networkName = computer["NetworkName"];

    queryOptions.AddSortProperty(networkName, SortingOrder.Ascending;

     

    // sort based on LastModified

    EnterpriseManagementObjectGenericProperty genericProperty =

    new EnterpriseManagementObjectGenericProperty(EnterpriseManagementObjectGenericPropertyName.TimeAdded);

    queryOptions.AddSortProperty(genericProperty, SortingOrder.Ascending);

     

    IObjectReader<EnterpriseManagementObject> instanceReader = emg.EntityObjects.GetObjectReader<EnterpriseManagementObject>(computer, queryOptions);

     

     

    Case 2: Sorting objects of projection types.

     

    The relevant APIs on ObjectQueryOptions  for sorting projection objects

     

    // <summary>

    /// Projection sorting based on generic and non-generic properties

    /// </summary>

    /// <param name="sortXml">properties to sort on, specified in xml</param>

    /// <param name="projection">the projection type</param>

    /// <param name="managementPack">the management pack to be used for resolving the properties specified in the sort xml</param>

    /// <param name="managementGroup">the management group</param>

    public void AddSortProperty(string sortXml, ManagementPackTypeProjection projection, ManagementPack managementPack, EnterpriseManagementGroup managementGroup)

     

     

    // <summary>

    /// Projection sorting based on generic and non-generic properties

    /// </summary>

    /// <param name="sortXml">properties to sort on, specified in xml</param>

    /// <param name="projection">the projection type</param>

    /// <param name="managementGroup">the management group</param>

    public void AddSortProperty(string sortXml, ManagementPackTypeProjection typeProjection, EnterpriseManagementGroup managementGroup)

     

    Below is an example of sorting objects of type System.WorkItem.Incident.ProjectionType.

    Let’s retrieve all incidents assigned to user “Peter”, by descending order of priority (property defined on  class System.WorkItem.TroubleTicket) and ascending order of AffectedUser (defined on class  System.Domain.User )name in each incident.

     

    Here, is the criteria I used for getting incidents assigned to “Peter”


    <Criteria xmlns="http://Microsoft.EnterpriseManagement.Core.Criteria/">

      <Reference Id="System.WorkItem.Library" Version="7.5.1083.0" Alias="WorkItem" PublicKeyToken="9396306c2be7fcc4"/>

      <Reference Id="System.Library" Version="7.5.1083.0" Alias="System" PublicKeyToken="9396306c2be7fcc4"/>

      <Expression>

        <SimpleExpression>

          <ValueExpressionLeft>

           <Property>$Context/Path[Relationship='WorkItem!System.WorkItemAssignedToUser' TypeConstraint='System!System.Domain.User']/Property[Type='System!System.Domain.User']/UserName$</Property>

          </ValueExpressionLeft>

          <Operator>Equal</Operator>

          <ValueExpressionRight>

            <Value>Peter</Value>

          </ValueExpressionRight>

        </SimpleExpression>

      </Expression>

    </Criteria>

    Here is the sort xml I used:

    <Sorting xmlns="http://Microsoft.EnterpriseManagement.Core.Sorting">

      <Reference Id="System.WorkItem.Library" Version="7.5.1083.0" Alias="WorkItem" PublicKeyToken="9396306c2be7fcc4"/>

      <Reference Id="System.Library" Version="7.5.1083.0" PublicKeyToken="9396306c2be7fcc4" Alias="System" />

      <SortProperty SortOrder="Descending">$Context/Property[Type='WorkItem!System.WorkItem.TroubleTicket']/Priority$</SortProperty>

      <SortProperty SortOrder="Ascending">$Context/Path[Relationship='WorkItem!System.WorkItemAffectedUser' TypeConstraint='System!System.Domain.User']/Property[Type='System!System.Domain.User']/UserName$</SortProperty>

    </Sorting>

    EnterpriseManagementGroup emg = new EnterpriseManagementGroup("sm-server");

    ManagementPackClass mp = emg.ManagementPacks.GetManagementPack("ServiceManager.IncidentManagement.Library", keyToken, new Version("7.5.1083.0"));

    ManagementPackTypeProjection typeProjection = emg.EntityTypes.GetTypeProjection("System.WorkItem.Incident.ProjectionType",mp);

     

    ObjectProjectionCriteria criteria = new ObjectProjectionCriteria(Criteria.incident,typeProjection, emg);

    ObjectQueryOptions opt = new ObjectQueryOptions();

    opt.AddSortProperty(Criteria.sort,typeProjection, emg);

    IObjectProjectionReader<EnterpriseManagementObject> p = emg.EntityObjects.GetObjectProjectionReader<EnterpriseManagementObject>(criteria, opt);

     

     

  • System Center Platform Team

    Using SDK Criteria for Querying Types

    • 0 Comments

    In the previous post, we talked about using criteria to query on instances. This post talks about using criteria to query types in the management pack such as classes, relationships, tasks, views etc. These are queries such as:

    ·         Query for all classes that match a given class name.

    ·         Query for all relationships with a particular target class.

    ·         Query for all tasks with a particular category.

    ·         Query for all management packs of a particular version and so on.

    There is a criteria class associated with each of these types. Example, the class  ManagementPackClassCriteria is used to query for management pack classes.

     

    namespace Microsoft.EnterpriseManagement.Configuration

    public class ManagementPackClassCriteria : QueryCriteria<ManagementPackClassCriteria>

        {

           

        }

    The following table indicates all the properties of a management pack class that can be queried for. Example – All classes with a specific management pack id, classes that are abstract, classes that are sealed etc.

    Id

    Name

    Base

    ManagementPackId

    Hosted

    Abstract

    Extension

    Accessibility

    Singleton

    Sealed

    TimeAdded

    LastModified

    DisplayName

    Description

    Category

     

    The constructor for these classes accepts a criteria xml as shown below:

    public ManagementPackCriteria(string criteria)

    {

    }                             

    All type related criteria classes provide a similar constructor. The criteria parameter follows the same schema as discussed in this post except for the following differences:

    ·         Only <GenericProperty> tags are used and not <Property> for indicating properties to query. As a result the syntax followed is <GenericProperty>propertyName</GenericProperty>

     

    ·         ContainmentCriteriaType and FullTextCriteriaType cannot be used.

     

    ·         Guid values for types need to be used instead of type names as there is no managementpack parameter exposed and hence the type name cannot be resolved.

     

    Here’s an example of a criteria xml that queries for all classes in a given management pack.

    <Criteria xmlns="http://Microsoft.EnterpriseManagement.Core.Criteria/">

      <Expression>

        <SimpleExpression>

          <ValueExpressionLeft>

            <GenericProperty>ManagementPackId</GenericProperty>

          </ValueExpressionLeft>

          <Operator>Equal</Operator>

          <ValueExpressionRight>

            <Value>dbb151b7-53e0-4de0-967e-a1ade70f5c3c</Value>

          </ValueExpressionRight>

        </SimpleExpression>

      </Expression>

    </Criteria>

     

    The code for the same looks as below:

    ManagementPackClassCriteria criteria = new ManagementPackClassCriteria(criteria);

    IList<ManagementPackClass> mpClasses = managementGroup.EntityTypes.GetClasses(criteria);

     

    In this post, we talked about querying for all incidents that were related to config items. Consider querying for all relationships that exist in the system that include config item as a source or target. The criteria xml for the same would look like:

     

     

    <Criteria xmlns="http://Microsoft.EnterpriseManagement.Core.Criteria/">

      <Expression>

        <Or>

          <Expression>

            <SimpleExpression>

              <ValueExpressionLeft>

                <GenericProperty>SourceType</GenericProperty>

              </ValueExpressionLeft>

              <Operator>Equal</Operator>

              <ValueExpressionRight>

                <Value>replace with guid value for config item type</Value>

              </ValueExpressionRight>

            </SimpleExpression>

          </Expression>

          <Expression>

            <SimpleExpression>

              <ValueExpressionLeft>

                <GenericProperty>TargetType</GenericProperty>

              </ValueExpressionLeft>

              <Operator>Equal</Operator>

              <ValueExpressionRight>

                <Value>replace with guid value for config item type</Value>

              </ValueExpressionRight>

            </SimpleExpression>

          </Expression>

        </Or>

      </Expression>

    </Criteria>

     

    The following table indicates all the properties on which a ManagementPackRelationship can be queried for

     

     

    Name

    Base

    SourceType

    TargetType

    ManagementPackId

    Abstract

    Accessibility

    Sealed

    TimeAdded

    LastModified

    DisplayName

    Description

    Category

     

     

    Note: For operations that involve querying for classes or relationships frequently, it will be better to use regular APIs without criteria and compare results on the client for performance reasons.

  • System Center Platform Team

    Introductions

    • 0 Comments

    The System Center Service Manager platform team has decided to spin up our own blog discussing platform focused topics separate of the Service Manager team blog. The target audience of this blog is the developer audience and hence we put it on MSDN. We debated about whether or not we need another blog or to use the existing team blog, but decided that the target audience is not the same (even though there is definitely overlap) and, in addition, the platform we deliver is also consumed by our sister System Center products, Operations Manager and Essentials.

    The features of the infrastructure we plan to focus on include, but are not limited to, the CMDB, general data access, data warehousing, extensibility, performance and scale, reporting and security and we’ll have posts from various people (developers, program managers and testers) from all these areas so you will get not only a “straight from the horse’s mouth” perspective, but also direct access to the people working on the features you read about.

    Our goal is to enable customers and partners to deliver solutions on top of our platform that cannot be developed just by using the console (Analyst or Authoring). We also hope to give those interested a deeper look at our platform and direct access to our platform team to answer any questions you might have and to field requests for platform changes or blog posts.

  • System Center Platform Team

    Troubleshooting the Data Warehouse: Custom data warehouse extensions don’t appear in the data warehouse

    • 0 Comments

    This is the latest in “Troubleshooting the Data Warehouse” series. The goal of this series is to not only help you troubleshoot specific issues you may be experiencing but also teach you more about the data warehouse. For a complete list of posts in the series, check out Troubleshooting the Data Warehouse: An overview.

     

    The symptom

     

    After importing your management pack which defines some dimensions and/or fact tables to Service Manager, MPSync job has run several times but you still don’t see your dimension and/or fact tables in the DataMart.

     

    The troubleshooting steps

     

    To troubleshoot this issue, investigate the following things in order.

    1. Ensure your management pack is sealed

     

    MPSync Job can only bring sealed management pack from Service Manager to Data Warehouse. If you haven’t sealed your management pack, the simple solution is to seal it and then import it to Service Manager.

    2. Ensure your management pack has synced to Data Warehouse

     

    ·         Open the Console

    ·         Click on the Data Warehouse wunderbar

    ·         Click on Management Packs then find your management pack in the list of management packs. To do this, use the search feature (type your management pack name in the search box).

    If you don’t see your management pack,

    ·         It might have failed to import to Data Warehouse Management Server. Go to the Data Warehouse Management Server, open the Operations Manager event log and then filter the events with Event Sources as OpsMgr SDK Service.

    ·         The MPSync job may not have run yet. It runs on a recurring schedule, out of the box it runs every hour. You can modify the schedule with Powershell. In order to speed up the management pack synchronization, after import your management pack, you can manually resume MPSync job either from UI or powershell.

     

    3. Check the deployment status of your management pack

     

    If the management pack is sealed and has synced to Data Warehouse

    ·         Open the Console

    ·         Click on the Data Warehouse wunderbar

    ·         Click on Management Packs then find your management pack in the list of management packs. To do this, use the search feature (type your management pack name in the search box).

    ·         Check the deployment status (last column) of your management pack.

     

    4. If the deployment status is Failed

     

    Go to the Data Warehouse Management Server, open the Operations Manager event log and then filter the events with Event Sources as Deployment.

    If there is error message, usually the error message will indicate what went wrong. If after correct the management pack, the error still occurs. In order not to affect the overall system, you can uninstall this management pack from Service manager Console, Administration > Management Packs. After MPSync job run, the management pack will be uninstalled from Data Warehouse Management Server.

     

  • System Center Platform Team

    Troubleshooting the Data Warehouse: An overview

    • 0 Comments

    I know that troubleshooting the data warehouse can be a bit intimidating. Sometimes even figuring out where to begin to look can be difficult. In this series of blog posts, I am going to help you figure out how to troubleshoot the data warehouse. I’ll use this post as an index with links to all of the subsequent topics.

     

    Where to begin?

     

    Depending on what kinds of issues you see, your start point might be different. Here I am going to talk about the general approach. 

    If you observe a failure or something not too right, generally, there are a few places we can start our debugging.

     

    1.      Start with the Operations Manager event log on the Data Warehouse server

     

    This is the absolutely most important place to begin! Almost all of the errors from the Data Warehouse are output to this event log. Events in this log will have two different sources: Deployment and Data Warehouse.

    Events with a source of Deployment are usually from management pack deployment which includes report deployment or building out the data warehouse (like creating outriggers, dimensions, fact tables, etc).  If you see an error in the event log, it usually has instructions on how to recover from it. For example, it may indicate you need to restart the services. In the Data Warehouse Management Server, there are three services just like on the Service Manager Management Group server:

    ·         System Center Data Access Service,

    ·         System Center Management,

    ·         System Center management Configuration

    It’s usually best to recycle all of them at the same time.

    Once your data warehouse is deployed, events are more likely to have a source of Data Warehouse. These events are written by jobs within the normal course of operations like the ETL jobs, the MPSync job and the DWMaintenance job. However, these all still write their events to the Operations Manager event log.

     

    2. Data Warehouse Workspace in the Service Manager console

     

    When you click on the Data Warehouse Jobs in the Data Warehouse workspace, you should see the ETL jobs and MPSync Job status. If your deployment was successful and your data warehouse is correctly registered to at least one Service Manager management group, you’ll see at least 5 jobs. Every job should either be in a status of “Running” or “Not Started”. 

     

    If you see a job is in Failed status, you can click on this job and then click on Modules from the Tasks pane to find out which job module has failed. However to find out why it failed, the best place to get information from is the Operations Manager event log on the Data Warehouse server. You can also get more details from Powershell cmdlets which I will talk about in the next step. 

     

    In the Data Warehouse workspace, you can also click on the Management Packs link in the left pane. Here’s where you can see all the management packs in the Data Warehouse and the status of their deployment. When you import a management pack to Service Manager, the MPSync job will synchronize it to the Data Warehouse (hence the name…MPSync means “management pack synchronization”). When you get the list of management packs in the data warehouse, you can find out if your management pack has been deployed successfully or not. 

    If your management pack has defined data warehouse specific elements such as outriggers, dimensions, fact tables or reports, this Management Pack must be successfully deployed before the new tables and reports will be ready to use.

     

    3. Powershell

     

    The Powershell cmdlets provide much more detailed information about the Data Warehouse jobs than the console does.  This will likely change in a future release, but for now I really recommend you learn how to use the following useful cmdlets:

    ·         Get-SCDWMgmtGroup

    This command tells us the sources which are registered with DW currently. We expect to see at least two different DataSourceName.

     

    ·         Get-SCDWJob

    This command tells us all DW jobs status in the current batch.

    In the above screenshot, you can check whether the jobs are enabled or not (see the highlight), which jobs are running and when they started. 

    When MPSync Job or DWMaintenance job start, they both disable all of the ETL jobs so you will see the IsEnabled column set to false for each of the ETL jobs. This means that even if the ETL job status shows it is running, it actually isn’t. When MPSync Job or DWMaintenance job complete the ETL jobs will automatically get enabled and resume processing.

    Jobs are normally in the “Not Started” status and if it is you can assume the previous batch has completed. If you’d like, you can use the following command to view the latest few batches of a specific job.

    Get-SCDWJob –JobName <Specific job name>  -NumberOfBatches <number>

    In the above screenshot, you can see the latest MPSyncJob completed, when it started and when it ended. If you’d like you can calculate how long it ran, and what the next batch id and status is.  The job batch id is always incremental.

    ·         Get-SCDWJobModule

    This command provides detailed information about the specific modules within the job.  This is very useful when you see job failed and want to find out what caused the failure.

    List of common issues and how to troubleshoot them

    This list is not exhaustive, but it does cover most of the common issues customers have run into. This list will get added to over time so feel free to check back if you run into a new issue:

    ·         Reports aren’t deployed after registering the data warehouse

    ·         Job(s) fail after importing a custom management pack

    ·         Data warehouse isn’t getting new data or jobs seem to run forever

    ·         Custom data warehouse extensions don’t appear in the data warehouse

    ·         Management packs are stuck in “pending association” after registering the data warehouse

    ·         ETL jobs fail due to login credentials issues

     

  • System Center Platform Team

    Troubleshooting the Data Warehouse: ETL failures due to login issues

    • 0 Comments

    This is the latest in “Troubleshooting the Data Warehouse” series. The goal of this series is to not only help you troubleshoot specific issues you may be experiencing but also teach you more about the data warehouse. For a complete list of posts in the series, check out Troubleshooting the Data Warehouse: An overview.

     

    The symptom

     

    Some or all ETL jobs have failed. The Operations Manager event log in the Data Warehouse Management Server indicates the ETL job failure is related to a login user failure.

     

    The troubleshooting steps

     

    To troubleshoot this issue, investigate the following things in order.

    1. Check if the password for each run as accounts has changed or expired.

     

    If it is, you need to update it. To do this:

    ·         Click on Data Warehouse > Security > Run As Account, click on the related run as account, then click the Properties from the Tasks pane, update the Password field in the window and then click OK

    ·         If this run as account is Operational System Account, you also need to update the services that are running under this account.

    o   In Data Warehouse Management Server, click Start > Run .., type Services.msc

    o   In the Services window, update the passwords for the services that run under this account, for example,  System Center Data Access Service and System Center management Configuration

    o   Restart the services

     

    Note that MP Sync job and Extract job for Service Manager can use a different run as account other than Operational System Account. This run as account is created while Service Manager is registered to Data Warehouse. See the following Data Warehouse Registration Wizard, the New … button

     

    It is easy to update the password if it is expired. However it is complicated to update the system if you change the run as account. We don’t recommend that you change run as accounts.

     

    2. If the job failure is not related to password

     

    You should make sure the run as account for the failed job can be used to connect to the target database. For example, ensure the Extract job run as account can be used to connect to the Service Manager database. If not, make sure the SQL service hosting the database is running.

     

     

     

     

Page 1 of 1 (17 items)