Thoughts on ALM, Azure, and Agile by Walter Myers III

Walkthrough to Configure System Center Management Pack for Windows Azure Fabric Preview for SCOM 2012 SP1 (with a MetricsHub Bonus)

Walter Myers III — Sat, 13 Apr 2013 18:44:58 GMT

The wait is finally over. This is a huge update to the Azure Management Pack over the one that was released a couple of years ago, which not only monitors Azure applications but storage and virtual machines as well. You can download the new management pack from http://www.microsoft.com/en-us/download/details.aspx?id=38414. Remember that the management pack is currently a preview, and also that it is compatible only with System Center Operations Manager (SCOM) SP1. Below is the functionality of the management pack from the documentation:

Discovers Windows Azure Cloud services.
Provides status of each role instance.
Collects and monitors performance information per role instance.
Collects and monitors Windows events per role instance
Collects and monitors the .NET Framework trace messages from each role instance.
Grooms performance, event, and the .NET Framework trace data from Windows Azure storage.
Changes the number of role instances.
Discovers Windows Azure Virtual Machines
Provides status of each role instance of the Virtual Machine
Discovers Windows Azure Storage
Monitors availability and size of each Storage and optionally alerts
Discover relationships between discovered Azure resources, to see which other resources a particular Azure resource uses. This information is then displayed in a topology dashboard
Monitors management and cloud service certificates and alerts if the certificates are about to expire
Includes a new Distributed Application template that lets you create distributed applications that span Azure as well as on-premise resources, for Hybrid monitoring scenarios.
Includes a set of dashboards for the Hybrid monitoring scenarios

In this post, I will provide a quick walkthrough of my experience getting the new management pack up and running. It is impressive, and the product group has promised improvements going forward. So let’s get started.

After you download the management pack install, which is a .msi file, you will have to run it so it can extract the two management packs, and write them to your file system. Note that the extensions are .mpb and not .mp, and thus are only compatible with SCOM 2012. Recall that management packs have historically been single XML files either in an .xml format or in a binary format with a .mp extension. The new format, which can consist of multiple XML files has a .mpb extension which means “management pack bundle.” Note the path to the management packs which you will need when you import the management packs into SCOM 2012, as seen below.

Launch the SCOM 2012 Operations Console and select the Administration view. Navigate to the Management Packs node and select the Import Management Packs item from the popup menu, as seen below.

You will now see the Import Management Packs dialog. Select the Add button, which will drop down two selections. Choose the Add from disk… option, as seen below. When you are prompted to search the online catalog, then select the No button.

Locate the management pack bundles and select both, as seen below.

Now go ahead and click the Install button that will start the import process, as seen below.

Once the import is done for the two management packs, click the Close button.

In the Search box at the top of the management pack list, enter “azure” and select the Find Now button. It may take a couple of minutes but you should eventually see two management packs:

Microsoft.SystemCenter.WindowsAzure.Dashboard
Windows Azure

Verify that the management packs are there, as seen below.

In the old management pack, you head straight to the monitoring wizard, but the new and improved pack works on a subscription boundary so it can discover all of the Azure resources that exist in the subscription. Still, you must select from the available resources the specific resources that you want to monitor. Let’s navigate to the bottom of the Administration explorer view and locate the Windows Azure node. Select the Add subscription link, as seen below.

You will now be presented with a wizard to configure your Azure subscription. Supply the Subscription GUID, the private key of the management certificate that you have uploaded to the management portal, and the certificate password. I’m presuming you have already installed the private key in your Personal and Trusted Root Certification Authorities certificate stores. If not, please see the instructions in my previous post for SCOM 2007 here. The instructions are pretty much identical, and you will also need to follow the procedure to setup your Run As accounts.

In the next page, you will choose the Proxy Agent, which will be this SCOM server. Select the Browse… button, which will launch the Search dialog.

In the Search dialog, select the Search button, and then select your server in the Available items results list. Click the OK button when you’re done.

Now back at the Proxy Agent page, select the Add subscription button.

You will see a warning dialog, as seen below. Allow Operations Manager to distribute the accounts by selecting the Yes button. In this case, the accounts will be distributed to the SCOM server.

You’re now done and can select the Finish button.

Now you’re back at the Windows Azure configuration page, as seen below. We’re all set now to see what SCOM has discovered in your subscription.

Select the Monitoring view in the lower left-hand corner, and then navigate to the Discovered Cloud Services node in the Azure Resource Inventory folder. Your discovered cloud services view should look similar to below. Note that you will have to wait a few minutes for discovery to take place for your subscription. All items in the list will have a “Not monitored” state, which is what we should expect since we have to specifically select what we want to be monitored amongst the resources that have been discovered by SCOM. That is a very nice touch with the new management pack because from the beginning it is conserving resources, allowing you to select what you desire to actively monitor.

Now before we can select what we will monitor, we need to create our own management pack to store information on the resources we decide to monitor. Return to the Administration view, navigate to the Management Packs node and right-click on the node. Select the Create Management Pack item from the popup menu, as seen below.

In the Create a Management Pack wizard, specify the name of your management pack, and provide a version number if so desired, as seen below. Select the Next button.

On the next page of the wizard, select the Create button. Your management pack will now be created.

After a couple of minutes, verify your management pack has been added to the list, as seen below.

Now we can begin monitoring specific resources in your subscription. Select the Authoring view, and navigate to the Windows Azure Monitoring node. Right-click on the node and select the Add Monitoring Wizard… item from the popup menu, as seen below.

In the Add Monitoring Wizard, select Windows Azure Monitoring as the monitoring type (yes, it is redundant because that is the very reason you right-clicked on the explorer node previously), and select the Next button.

On the General Properties page, note that this is different from the SCOM 2007 R2 management pack. This does not map to a specific cloud service, but can be a combination of things you want to monitor with this particular management pack object. It could be any collection of cloud services, virtual machines, or storage, and can be in any combination of staging or production slots. So provide a name appropriate to what you would like to monitor in this monitoring object. Also, select the management pack you created since you will be putting all of your management objects in your own management pack and not the sealed Windows Azure management pack. When you’re done, select the Next button.

On the next Subscription page, select your subscription you previously configured, as seen below. Select the Next button.

On the Cloud Services page, select the Add button to add your desired cloud services that you want to monitor in this subscription. You will be presented with a Search page.

Run the search based on the desired criteria. From the available services list, add each to the lower selected services list, as seen below. Select the OK button.

You’re now back to the Cloud Services page, where you will see the cloud services you selected. Select the Next button.

On the Virtual machines page, you can now select the IaaS Virtual Machines that you have created in your subscription. In the same manner as you selected the cloud services, select your virtual machines.

When you’re done, you will be back to the Virtual machines page. Select the Next button to continue.

You will now be on the Storage page. In the same manner as you selected cloud services, select the Azure Storage that you would like to monitor. When you’re done, select the Next button.

Finally, you are brought to the Summary page where you can review the objects you have selected for monitoring in your subscription, as seen below. Select the Create button to proceed.

You will subsequently see your new monitoring object has been created, as seen below.

Let’s now return to the Monitoring view. Navigate to the Cloud Service State node under the Monitored Azure Resources folder. You will see here the cloud services you have selected for monitoring. Note that we came here instead of the Discovered Cloud Services node because this node is in the folder that only presents the monitored Azure resources we previously selected. The Discovered Cloud Services will simply display everything that has been discovered.

If we head back to the Discovered Cloud Services node, we will see all of the discovered cloud services. Note that just as above, I have three cloud services that are now being monitored, but two have a critical state. Let’s investigate the cause of the critical state.

Right-click on one of the critical state items, select Open in the popup menu, and then select Health Explorer for <cloud_service_name> in the secondary popup menu.

You will now see the Health Explorer, which can provide specific details regarding where you may have problems with a given object you are monitoring. By default, the view is scoped to unhealthy child monitors (note the scoping in the yellow section immediately above the Health Explorer nodes), and we can see at the bottom of the explorer that it has found my expired certificate which caused it to go into a critical state soon after the object was first monitored.

If I go ahead and delete the scope to unhealthy child monitors by selecting the “x” next to “Scope is only unhealthy child monitors,” we find even more information on the cloud service, as seen below. Let’s go ahead and close the Health Explorer and go back to the Operations Console (this is a separate window, so closing it won’t close the Operations Console).

Another way to find out the problem is to check the alerts. Let’s navigate to the Active Alerts node under the Windows Azure folder, as seen below. Notice that the alert tells us the service certificate for this hosted (cloud) service will expire soon. This is a great new feature of the SCOM 2012 SP1 Azure management pack! All I have to do is update the service certificate and my cloud service state will turn back to green. Impressive!

Let’s now navigate to the Storage State node under the Monitored Azure Resources folder. Note that we have a healthy storage account.

Finally, let’s navigate to the Virtual Machine State node under the Monitored Azure Resources folder. Note that we are monitoring healthy virtual machines. One thing to note here is that the management pack does not look at any resources running on the virtual machines such as SQL Server or IIS. This is purely for the overall health of the server itself, and not any installed features or server programs running on the server. If you want that functionality, you will have add these servers to a Virtual Network so that you have IP-level connectivity and can monitor them just as you would an on-premises server. That would also mean adding the appropriate service packs for programs you would like to monitor such as SQL Server 2008/2012 or IIS.

Let’s now go to the Health Explorer for one of the virtual machines, and clear the scope for only unhealthy child monitors. When we expand the Performance node, notice that there are two performance counter monitors. Unfortunately, this is a bug and these monitors should not have been enabled. I know, what a disappointment and I have no idea what it actually is monitoring but I’m trying to find out and will report back when I do. What is working here is the Availability monitor, even though the Performance monitors are not.

It’s not like this can’t be done because MetricsHub (an Azure-based Azure application monitoring and cost-optimization solution recently acquired by Microsoft) does it by providing an agent that you have to manually install on the virtual machine. The agent provides a secure, encrypted public service endpoint that allows MetricsHub to read the performance counters, displaying CPU, Network (Received/Sent), Memory, and Disk (Free Space/Read/Write) performance counters by default in its portal. SCOM can already distribute its own agent to on-premises servers, and to virtual machines that participate in an Azure virtual network and have a VPN connection established. The VPN connection established from on-premises to a virtual network makes IP-level connectivity to servers in the virtual network seamless. So there is no reason SCOM can’t adopt the same model as MetricsHub for virtual machines that don’t participate in a virtual network coupled with VPN. Until SCOM provides this functionality by providing an agent and updating the management pack to call the agent, you may want to take a look at MetricsHub to augment your monitoring of Azure applications and virtual machines. Out of the box, MetricsHub can display the following counters (the bold ones map to those mentioned above):

LogicalDisk\% Free Space
LogicalDisk\Disk Read Bytes/Sec
LogicalDisk\Disk Write Bytes/Sec
Memory\% Physical Memory
NetworkInterface\Bytes Received/sec
NetworkInterface\Bytes Sent/sec
Process\% Processor Time
Process\IO Read Bytes/sec
Process\IO Write Bytes/sec
Process\Working Set – Private
Processor\% Processor Time

I won’t walk through the setup of MetricsHub because it is pretty easy to setup. You will have to go the Windows Azure Store in the management portal to add it, and the best thing is it is currently free. Below in the MetricsHub portal you see a virtual machine that has not yet been configured for monitoring by MetricsHub. You can select the link to download the Agent that must be installed on the virtual machine, as seen below.

After you click the button to install the Agent, you will be provided with instructions to download the MetricsHub Agent and get it installed on your virtual machine, as seen below.

About 30 minutes after installing the Agent, voila! You now have actual, working performance counters for your virtual machine. Some of these performance counters are the same as those presented in the Azure management portal, but you get those and even more from MetricsHub.

If you want to add additional counters to the display, select the Create Chart… button on the same page and you will be presented with a dialog that allows you to select the other available out-of-box performance counters.

If you want to add additional counters, there is a configuration file named MetricsHub.Diagnostics.Monitor.exe.config provided with the agent that has all of the out-of-box counters, so you can use these as templates to add any valid performance counters you wish. Below is a screen shot of section in the configuration file for setting up performance counters.

That ends our walkthrough for now, but we will be looking into finding more information on the new management pack. Stay tuned for much more to come on SCOM 2012 SP1 and Azure. We’re just getting started!

Setting Up an Alert for Hosted Service State Changes in SCOM 2007 R2

Walter Myers III — Tue, 19 Feb 2013 21:45:49 GMT

I’m posting this by request from a customer who desires to setup an alert when a hosted service changes its state. The process is fairly straightforward, provided you know where to look. In the screenshot below, we have a hosted service (in my case, eponymously named walterm), that is currently healthy. What we want is to generate an alert if the health state of this service changes.

By default, there is already an “out of the box” monitor at the Windows Azure role level which you will find in the Monitors view of the Authoring pane. Navigate to Windows Azure Role | Entity Health | Availability, as seen below. There you will find the monitor titled Role Instance Availability Dependency. Right-click on this item and select the Properties page.

On the Properties page, choose the Alerting tab, where you can verify that by default the alerts are being generated for this monitor, as seen below.

Now if you want to create an individual monitor at the role instance level (whether for all role instances or for individual hosted services) then you can override the out of the box monitor already at that level, which doesn’t generate alerts by default. For our example, we will navigate to the general Windows Azure Role Instance for a specific hosted service, as seen below.

Right-click on the Windows Azure Role Instance Status Monitor item and select the Properties page. If you don’t see it, scroll down to the Generates Alert parameter name, as seen below. Change the Override Value to “True.”

Next, if you don’t see the Enforced column, scroll horizontally until you see it and select Enforced for this monitor, as seen below. When you’re done, select the Apply button, and then select the OK button.

Let’s now right-click on the monitor you just modified, and from the flyout menu select Overrides | Override the Monitor | For all objects of class: Windows Azure Role Instance (your_hosted_service_name).

You will now see the override you just created, as seen below. Go ahead and select the Close button.

We will now go to the Windows Azure management portal and stop the service we desire to test, as seen below.

When we return to the SCOM console and select the Hosted Service State node for our management pack, after a few minutes the service state will change to critical, as seen below.

Shortly after the state change, alerts will show up in the Active Alerts view, as seen below.

If we now select an alert with the name Windows Azure role is not running, we can see that this alert was generated at the level of Windows Azure role, as seen below.

If our role instance level override was setup properly, we should also see a similar alert with the name Microsoft.SystemCenter.Azure.RoleInstance.Status.Monitor. This is the alert we previously (and successfully) configured, as seen below.

Creating Unit Monitors for Windows Azure Performance Counters in SCOM 2007 R2

Walter Myers III — Wed, 23 May 2012 23:15:00 GMT

Monitors are powerful management pack objects in SCOM. In the Operations Manager documentation, a monitor is described in the following manner: “monitors can be used to assess various conditions that can occur in monitored objects. For example, a monitor can be used to assess the values of a performance counter, the existence of an event, the occurrence of data in a log file, the status of a Windows Service, or the occurrence of a Simple Network Management Protocol (SNMP) trap. The result of this assessment determines the health state of a target and the alerts that are generated.” So basically, we use monitors to watch for certain events on a monitored object, assess the health state of that object, and then generate an alert if required based on that assessment.

In this post, we will create a simple unit monitor based on a performance counter to determine if any role instance of a Windows Azure application has CPU utilization above a specified threshold. If so, it will generate an alert. The idea is to simply respond to specific conditions met by a performance metric using a Unit Monitor. Since SCOM has no knowledge of Windows Azure data sources, we will have to leverage the Authoring Console so we can properly set our data source and then create the monitor. In this post, we will create our own unit monitor that will generate an alert if we have three consecutive samples of processor utilization above 80%. Alternatively, we could simply modify the existing Total CPU Utilization Percentage monitor by overriding the existing processor utilization threshold and adding a diagnostic or recovery task. But for learning purposes this exercise shows you how to do it yourself.

Below you will see that we have the Authoring Console running, and have loaded our management pack previously exported from the Operations Console. For your reference, here are some probable locations of the references the Authoring Console will most likely ask you to resolve before it loads your management pack:

Microsoft.SqlServer.SqlAzure: C:\Program Files\System Center Operations Manager 2007
Microsoft.SystemCenter.WebApplication.Library: C:\Program Files (x86)\System Center 2007 R2 Hotfix Utility\KB2495674\ManagementPacks
Microsoft.SystemCenter.Azure: C:\Program Files (x86)\System Center Management Packs\Monitoring Pack For Windows Azure Applications

In the Health Model view, select the Monitors node, as seen below.

Expand the Windows Azure Role Instance node in the Monitors pane, and then the Performance node, which will have the two out of box monitors listed, as seen below (you will also see two below that I have already added). The out of box monitors are disabled by default (and thus you will see the icons are grayed), but you can enable them by using an override in the Operations Console. Go ahead and right-click on the Performance node, and then select New | Custom Unit Monitor… from the popup menu.

Now you will be asked to give the monitor a unique identifier, as seen below.

You will next see the configuration dialog with the General tab presented. It is already setup with the ID, Target, and Parent Monitor. So just provide a name and description as seen below.

Select the Configuration tab, as seen below. Before you can proceed, you must select a unit monitor type. Select the Browse for a type… link as seen below.

You will now see a dialog that allows you to select your unit monitor type, but you must first narrow your choices down to the unit monitor types in the Windows Azure management pack. So just type “azure” in the Look for: textbox. Your choices for unit monitor types are:

Windows Azure Event Log Simple Monitor Type
Windows Azure Performance Counter Simple Monitor Type
Windows Azure Role Instance Status Monitor Type

Select the second of the three listed above, as seen below. Then click the OK button.

You will now see the desired configuration, where you will need to enter the appropriate values. Enter the values below, but don’t hit the OK button yet because we need to add additional values manually.

Interval Seconds: 300
Timeout Seconds: 120
CounterName: % Processor Time
ObjectName: Processor
Threshold: 80
Direction: greater
NumSamples: 3

Now you need to select the Edit… button so we can add additional important configuration parameters, namely InstanceName and AllInstances. Be precise as below, or the editor won’t like it when you try to save the configuration. Save the configuration from the File menu and close the editor.

Now we have our final configuration, as seen below. Let’s next select the Health tab.

On the Health tab, enter your desired text in the Operational State fields. Select Critical for the condition True health state, and select Healthy for the condition False health state, as seen below.

Next, we want to setup our alert. Select the “Generate alerts for this monitor” checkbox. Choose “The monitor is in a critical health state” from the dropdown box below. Leave the “Automatically resolve the alert when the monitor returns to a healthy state” checkbox checked. In the Alert name textbox, enter text indicative of the alert. Leave the Priority and Severity dropdowns as they are. Now, select the ellipsis next to the Alert description textbox, which will display the Alert Description dialog.

In the Alert Description dialog, clear out the existing text. From the Data button, select the Value item in the flyout menu. This will then insert the $Data/Context/Value$ parameter in the text box. This value will provide the number of samples, but you also want the sample value which represents the processor percentage. That actually is $Data/Context/SampleValue$, but it isn’t in the flyout menu. What I did was to just cheat and grab the text from the out of box CPU Utilization Percentage monitor, which is as follows:

The threshold for the Processor\% Processor Time\_Total performance counter has been exceeded for $Data/Context/Value$ samples with last value that exceeded the threshold being $Data/Context/SampleValue$.

For a great list of variables, please see Kevin Holman’s blog here, where he indicates the difference between the Value and SampleValue variables we are working with. Another great place to check out for variables is Ben Oostdam’s blog here. When you’re done, select the OK button.

Your dialog should now look similar to below.

Finally, select the Options tab so this will be properly categorized when you setup notifications. Select the PerformanceHealth radio button. When you’re done, select the OK button.

You should now see your unit monitor as below, and you’re ready to export the management pack to your management group. We have covered this in other blogs, so I won’t show how to do it here.

You can see how to setup notifications in the Operations Console based on alerts in my SCOM 2007 R2 Event Log Alerting and Monitoring for Azure Applications blog post. Look for the Setting up Notifications in SCOM section.

Viewing the Operations Manager Event Log on Your SCOM 2007 R2 or SCOM 2012 Management Server from the Operations Console

Walter Myers III — Tue, 22 May 2012 14:05:00 GMT

In corporate environments, sometimes a SCOM administrator is given access to the Operations Console only through installation on a client machine or through a Citrix session. So they can’t just launch the Event Viewer in order to take a look at the Operations Manager event log. The great news is this doesn’t mean you won’t be able to gain access to the event log on the SCOM management server, which is essential when debugging SCOM issues. So here’s a quick post so you will know how to gain access to the event log directly from the Operations Console. Just select the Windows Computer node in the Monitoring view, as seen below. Locate your SCOM management server in the right-hand pane, and right-click on it. Select Open | Event View from the popup menu.

Next you will see the Operations Manager event log, which will be the event log on the server you select, whether it be a SCOM management server or an agent-managed computer. This can be seen below.

Hope this helps, as this is often overlooked by those who are not SCOM experts.

Notes on Monitoring Windows Event Logs and Trace Logs in SCOM 2007 R2

Walter Myers III — Fri, 11 May 2012 18:42:00 GMT

In my previous post, I showed you how to visualize Windows Azure Event logs in SCOM 2007 R2. I would like to extend this post to add on some of my additional learning of dealing with Event logs, and Trace logs as well. First, I want to show how to split out event log errors from warnings. Second, I want to show how to visualize Trace logs as well as setup views that will show them individually from Event logs. Third, I want to show how doing all of this applies to notifications. These are not necessarily in any order.

Event Log and Trace Log Setup

In the screenshot below, notice that I have four rules oriented around event logs and trace logs:

In the first two rules highlighted above for event logs, I have added a filter for Errors and Warnings, respectively. This means log items of others will be filtered out, so the reason for creating these is primarily for notification purposes. Recall in the previous post in the Authoring Console we created a Condition Detection filter that showed how to filter on certain event log channels, so I won’t go through all of the steps again since you can refer to that post. Below, I show how you can alternatively setup a filter for an Error, Warning, or any other type of event. It will translate these into the correct values, so don’t worry if the ordinal doesn’t match up with the event level ordinals you may have seen on various MSDN or TechNet pages.

One other thing that you need to be sure to do when setting up an Event log rule for Windows Azure is to go to the Options tab of your rule editor and select the EventCollection category. The reason we have to do this is because when we create a rule for Windows Azure, we always create a custom rule, so we have to categorize this particular type in order to be able to visualize in the Event View, as seen below. If you don’t, then when you attempt to setup an Event View to show items generated by specific rules, your rule won’t show up.

In order to verify that you have the proper rule category setup in the Operations Console, just open your rule and verify in the editor as seen below. If the rule category still has “Custom", then go ahead and change the rule category to Event Collection here.

In the third and fourth rules mentioned above, these will respectively bring in all Event log and Trace log items unfiltered for visualizing in SCOM Event Views. One point I want to make here is that in the rule editor there is a different data source for Event logs than for Trace Logs. Notice your choices as seen below, after narrowing down the list by typing “azure” in the Look for: textbox. You will see separate items as follows:

Windows Azure Role Instance .NET Trace Collection Simple Data Source
Windows Azure Role Instance Event Log Collection Simple Data Source
Windows Azure Role Instance .Performance Counter Collection Simple Data Source

So all you have to do is choose the right data source, assure that you have selected the right rule category, and you will have the full event and trace logs at your disposal when creating an Event View for your Windows Azure applications and also when you create a new notification subscription.

Notification Subscriptions

If your event and trace logs are setup properly above, then setting up notifications are straightforward, provided you have also properly setup your Channels and Subscriptions as described in my blog post SCOM 2007 R2 Event Log Alerting and Monitoring for Azure Applications, in the Setting up Notifications in SCOM section. In the screenshot below, I am setting up a notification subscription in the Administration view under the Subscriptions node.

When you click the Next button, you would select the created by specific rules or monitors (e.g., sources) checkbox, and then select the specific rules that will generate your notifications. So you would click on the specific link as seen below.

Once you select the specific link, you would then select your management pack in the Management pack: dropdown, scroll down to your rule, and then double-click it or select the Add button to add it to the Selected rules and monitors: list box as seen below.

So there. You should be all set now with respect to Event and Trace logs in terms of displaying them and setting up notifications, given the proper setup we discussed. Please feel free to post any comments or questions!

Visualizing Windows Azure Event Logs in SCOM 2007 R2

Walter Myers III — Sat, 18 Feb 2012 07:35:27 GMT

In this blog, I will demonstrate how to visualize event logs in SCOM 2007 R2. Even though the Windows Azure Management Pack documentation says that the management pack “collects and monitors Windows events,” it doesn’t simply collect them on its own. It will only collect them if you create an event collection rule to collect them. In order to create any rule for Windows Azure, as we have seen in previous posts, you must either modify your management pack manually, or leverage the trusty Authoring Console, which ships with the System Center Operations Manager 2007 R2 Authoring Resource Kit which can be downloaded here. I will assume you have read my previous blog on configuring performance counter rules that shows in the beginning how to export your management pack and open it in the Authoring Console (there is a gotcha I pointed out regarding CU4 that I believe has been fixed in CU5, but haven’t checked it out yet).

So presuming you now have your management pack loaded in the Authoring Console, and you are in the Health Model view, and your Actions pane is visible, in the right-hand corner you will need to select the New link and Custom Rule… from the fly out menu as seen below.

You will now see a dialogue asking you to complete the unique identifier for your rule. Tack on something like “EventLogRule” and then press OK, as seen below.

You should now see the dialog below that will allow you to configure your event collection rule. But there is a decision point here. You can either select a specific Windows Azure Role Instance from the Target dropdown, as seen below…

Or you can select the Browse all classes… item and then select the generic Windows Azure Role Instance, as seen below…

If you select an individual role instance, then you will have to create a rule for each of your applications. If you select the generic Windows Azure Role Instance, then this will be inherited by all of your role instances. Which path you decide on depends on your situation, but what you need to consider is if you require conditional filters that will apply to all of your role instances, or if you will vary the conditional filters on a role basis. Even though when you create event views in the Operations Console you can filter on criteria such as Source, Event Level, and Event Number, you can only filter on the name of the event log (e.g., Application or System) within the event rule filter itself. But if you don’t create a filter in the Authoring Console, you won’t be able to create one in the Operations Console and will have to go back to the Authoring Console to add the filter. Also, when you create a filter, it constrains what is actually collected, and whenever you make changes the changes only apply moving forward and not retroactively. I know, what a pain.

What I decided to do for this exercise was to use the generic role instance, and to go ahead and add a criteria filter in the Authoring Console to capture both System and Application event logs. That way, I can modify in the Operations Console at a later time. Be forewarned that if you make a change to the event rule filter in the Operations Console, even if from an individual role instance, then your change will apply to the base as well as all other instances. So it may work better for you to create these rules on an individual role instance level. Now that we have that established, you can see below on the General tab where I selected the base role instance so this would propagate to all role instances.

Now click on the Modules tab, as seen below, and let’s create our Windows Azure data source by selecting the Create… button below.

Type “azure” in the Look for: textbox to narrow down the search, select the Windows Azure Role Instance Event Log Collection Simple Data Source item in the list box, and type in “AzureDS” as your Module ID. Select OK when you’re done.

Your screen should now be as seen below. Next select the Create… button in the Actions area at the bottom.

You will now be in the Choose module type dialog. Type “event” in the Look for: textbox in order to narrow your search. Choose Event Data Collection Write Action, which is the action that will write to the SCOM operational store. Then type “WriteToDB” in the Module ID textbox. Click the OK button when done.

Now click the Create… button again so we can setup up collection to the SCOM data warehouse. This time, select the Event data publisher item and type “WriteToDW” in the Module ID textbox as seen below.

Your screen should now look like below. This is the basic setup, and if you don’t wish to have any filter criteria, then you would accept the dialog from here. Otherwise, you would select the Create… button in the Condition Detection area. Since we will be adding criteria, we will select the button.

In the Condition Detection dialog, type “filter” into the Look for: textbox to narrow your search. Select the System.ExpressionFilter item, and then type in “FilterID” as the Module ID at the bottom. Select the OK button.

So we look like below now. We’re not done with the filter, so we must now select the Edit… button to configure the filter.

You will now see the filter configuration dialog. Select the Configure… button, which will make your job much easier, unless you really enjoy writing XPath queries.

In our case, we will add an OR group so we can filter the rule to only accept the System and Application logs.

Add your criteria to the OR group. Typical parameters that you can use are below. These and others are documented in the Management Pack Authoring Guide as part of the System Center documentation.

Channel – name of the event log such as Application or System
Event Number – select specific event numbers you would like to show (you can filter this in the Event view)
Publisher Name – source of the event (you can filter this in the Event view)

So now go ahead and add your first filter condition.

Using the Expression button will add another condition.

Now fill in your remaining conditions, if you have any. I have my filter set to only pull from the System and Application event logs. Select OK when you’re done.

Now you see the final expression. Select OK.

Last, but not least, we need to set the category for the rule to EventCollection in the Options tab. The primary reason for this is if we want to be able to filter on the log name in an Event view (such as if you want Application and System events in one view and Trace logs in another view). If you don’t do this, then your rule won’t show up in the “generated by specific rules” condition in the properties page for your Event view, and thus you won’t be able to select it. So do set the category as seen below.

And now you have your final event rule ready to go.

We’re done creating the rule in the Authoring Console. Let’s now go ahead and export the management pack back to the management group (from Tools | Export MP to Management Group…).

Now let’s head back to the Operations Console. Once there, go to the Authoring view and select the Rules node. Locate your event collection rule in the Windows Azure Role Instance type.

Click the Edit… button in the middle of the dialog and verify the filter criteria. Click OK to leave the dialog, and then close the underlying rule dialog.

You can next scroll down to any individual role instance and see the event collection rule is there as well.

Let’s now go to the Monitoring view. Right-click on the management pack where you added the rule. Select New | Event View, as seen below.

Type in a name for your event log, and then select the ellipsis (…) after the Show data related to: label with Entity selected by default. From the Select Items to Target dialogue, find your role instance, select it, and then select OK. From the original dialog, select OK.

If you don’t immediately see the events pouring in, please give SCOM time. Patience is a virtue with SCOM when it comes to data collection. You should use a tool such as the Cerebrata Azure Diagnostics Manager to verify that your events are actually being written to Windows Azure storage for this application that you are monitoring.

If you want to add additional columns, you can just get properties on your event view, as seen below.

Select the Display tab and add the columns you want. I always add the log name.

As you can see, we’re capturing both Application and System logs.

Now let’s take note of where the Event view gets the event logs from. Based on our target, it looks to see if the target is collecting event logs, and thus picks up the logs from the available rule. I haven’t tried writing multiple event rules to see if they show up also, but I suspect if you have multiple rules then you have conditions that collect different event logs or different variations of actual events.

If you want to further filter on certain columns without affecting the rule itself, you can just go into the properties page of the view and select what you would like to filter on.

As seen below, I have pointed out a couple of possible selections for filtering. As I wrote earlier, if you want to filter by the actual log name (e.g., Application or System), you can go back and change the filter of your individual role instance that inherited from the base role instance event collection rule.

In order to change the filter you would go back to Rules in the Authoring view, open the event rule, and select the Edit… link in the Condition section as seen below.

From the property page, you would just modify the condition as desired. As noted above, this change will be propagated to the base role instance event rule as well as all other role instances. So again, you may find this is not desirable for you and want to create the event rules on an individual application (role instance) basis.

That’s it. Good luck and let me know if you have any questions or issues. Next time I will blog on setting up Service Level Tracking, and I will eventually get around to SCOM 2012 also!

Windows Azure Data Security (Cleansing and Leakage)

Walter Myers III — Wed, 01 Feb 2012 17:36:45 GMT

Disclaimer: The procedures explained in this document are current as of January 2012, and are subject to change without notice.

Of utmost concern to enterprise customers (indeed all customers) who are considering deploying their applications to Windows Azure is the security of their underlying data. A sometimes overlooked aspect of data protection is assuring that when disk space is freed and reallocated to other customers that those new owners cannot read the data that was there when the space was released. An extreme form of this is when the drives themselves are removed from the data center for disposal or repurposing for other tasks. The most straightforward way to make such a guarantee would be to overwrite the freed space with zeros or some other pattern before freeing it. Such overwrites can significantly impact performance, so Azure (like most systems) uses more complex but more efficient mechanisms.

In this post, we will find practices implemented by the Windows Azure and SQL Azure software to prevent the possibility of data leakage or exposure of one customer’s data to another customer upon deletion of a Windows Azure virtual machine instance, Windows Azure virtual machine drives, Windows Azure drives, Windows Azure storage, SQL Azure data, or a SQL Azure instance itself. The mechanisms vary in detail but are all conceptually similar: no user is ever allowed to read a place on the disk that he has not previously written.

The details in this post were provided by Charlie Kaufman, Principal Software Engineer and Security Architect extraordinaire for Windows Azure. You can find some of Charlie’s great work here and here. Thank you, Charlie!

Concepts on Data Protection

In practice, disks are allocated sparsely. This means that when a virtual disk is created, disk space is not allocated for its entire capacity. Instead, a table is created that maps addresses on the virtual disk to areas on the physical disk and that table is initially empty. The first time a customer writes data on the virtual disk, space on the physical disk is allocated and a pointer to it is placed in the table. We can see this conceptually in the progression of diagrams below:

Figure 1: Data Blocks Allocated to Users

In Figure 1 above, two users each have two data blocks that have been allocated to them on disk based on their respective write requests.

Figure 2: User Frees Up Data Block

In Figure 2 above, one user “deletes” data that frees up a data block. The data block is marked is free, but is otherwise unaffected.

Figure 3: User Allocated Recently Freed Data Block

In Figure 3 above, upon a write request, a new user is allocated a recently freed data block as well as a data block that has not been previously allocated. The previously freed data block is still unaffected. Essentially, the process is that when a user makes a write request to disk it must be determined whether there is already space on an existing data block allocated to that user that can store the new data. If so, then the new data overwrites the data in the existing block. If not, then a new data block is allocated and data is written to the new block. The logic can be seen in the figure below.

Figure 4: User Requests to Write Data to Disk

Now there is the question regarding the possibility of one customer reading the deleted data of another customer, or an Azure administrator reading a customer’s deleted data. If anyone tries to read a region on virtual disk that they have not yet written to, physical space will not have been allocated for that region and thus only zeroes would be returned. So we can see this logic in the figure below, and the result. Only an Azure administrator could read blocks marked as free, and there are no utilities that would help the administrator figure out who the previous owner of the block was.

Figure 5: User Makes Read Request

Conceptually, this applies regardless of the software that keeps track of reads and writes. In the case of SQL Azure, it is the SQL software that does this enforcement. In the case of Azure Storage, it is the Azure Storage software. In the case of non-durable drives of a VM, it is the VHD handling code of the host OS. Since customer software only addresses virtual disks (the mapping from virtual to physical address takes place outside of the customer VM), there is no way to express a request to read from or write to a physical address that is allocated to a different customer or a physical address that is free.

Note: in some cases, the logic for writes (see Figure 4) is modified in that if a block is written for a second time that data is not overwritten on disk. Instead, a new block is allocated and the data is written there. The old block is then marked as free. This approach is often referred to as a log-based file system. It may sound inefficient, but it allows most data writes to be to consecutive locations on the physical disk, and this minimizes seek times and yields better performance. This detail is transparent to the customer, but it is relevant because it means that even if a customer were to explicitly overwrite every block on the virtual disk with zeros before freeing it, it would not assure that the customer’s data was not still present on the physical disk.

Windows Azure Virtual Machine (VM)

When a VM is deleted, the space on disk that held the contents of its local virtual disk is marked as free, but is not zeroed. The space will eventually be used to hold data for some other VM, but there is no upper bound on the amount of time the obsolete contents may stay there. The virtualization mechanism, however, is designed to ensure that those spots on the disk cannot be read by another customer (or the same customer for that matter) until data is written again, thus ensuring there is no threat of data leakage. When a new virtual disk is created for a VM, it will appear to the VM to be zeroed, but that illusion is created by explicitly zeroing the buffers when a portion of the virtual disk is read before it is written. If a VM instance is reinitialized in place, it’s the same as if it had been moved to new hardware.

Windows Azure VM Drives and Windows Azure Drives (“X-Drives”)

There are two kinds of virtual drives that might be accessible to a VM instance in Windows Azure. The C:, D:, and E: drives that exist for Web and Worker roles are backed by disks that are local to the compute node. The data on them is not stored redundantly and must be considered ephemeral. In the event of a hardware failure, the VM instance is moved to a different node and the virtual disk contents are reset to their initial values. If a VM instance is reinitialized in place, the C:, D:, and E: drives revert to their initial states, the same as if it had been moved to new hardware.

Windows Azure Drives (aka “X-Drives”) are implemented as blobs in Windows Azure storage. X-Drives are persistent and are not reset unless the customer takes some explicit action to replace them. This data is stored redundantly and survives hardware failures. Deleting a VM instance does not cause data to be deleted in an associated X-Drive. An X-Drive is deleted by deleting the blob itself (or by deleting the storage account containing the blob). Please see the next section that explains how deletion of data is handled for Windows Azure storage.

Windows Azure Storage (Table, Blob, Queue)

The Windows Azure's Storage subsystem makes customer data unavailable once delete operations are called. All storage operations including delete are designed to be instantly consistent. Successful execution of a delete operation removes all references to the associated data item and it cannot be accessed via the storage APIs. All copies of the deleted data item are eventually garbage collected. The physical bits are overwritten (i.e., reinitialized) when the associated storage block is reused for storing other data, as is typical with standard computer hard drives.

SQL Azure

With SQL Azure, deleted data is marked for deletion, but it is not zeroed. If an entire database is deleted, it is the equivalent of deleting its entire contents. In any event, the SQL Azure implementation is designed to ensure used data is never leaked by disallowing all access to the underlying storage except via the SQL Azure API. That API allows users to read, write, and delete data, but does not have any way to express the idea of reading data that the user has not previously written.

Automatic Backup and Forensics

In the common case, customers want assurance that there is no unauthorized access to their data. In some cases, however, they would like assurance that there is not even any authorized access to deleted data. While there is no way through interfaces exposed to customers to retrieve data once it has been deleted or changed, that data may remain on disks for an extended period and it would theoretically be possible to recover it with internal forensic tools (though the likelihood of the deleted data being present would decline over time). Ultimately, any physical disk is either completely erased or destroyed after being removed from the production environment.

We are contemplating possible future features that would allow deleted data to be recovered (and changed data restored) for a limited period of time without customers having to make explicit backups. Using these tools, a customer could not be assured that the data would be for practical purposes inaccessible by authorized parties immediately after it is deleted. Any such tools will be designed to make deleted data retrievable only for a limited period of time less than 30 days unless the customer opted into some longer backup period. As of this writing, there are unexposed tools that may permit recovery of data deleted from SQL Azure databases for 14-21 days. There are no such tools for Azure Storage or Azure Compute ephemeral disks.

Resolving Windows Azure Diagnostics Connection String Issues with SCOM 2007 R2

Walter Myers III — Mon, 16 Jan 2012 22:31:26 GMT

In earlier versions of the Windows Azure SDK, the default Windows Azure Diagnostics string provided by the Visual Studio template in the service configuration file was named DiagnosticsConnectionString. Later Windows Azure SDKs have changed this name to Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString. By default, however, the SCOM 2007 R2 Windows Azure Management Pack continues to reference the DiagnosticsConnectionString setting in order to find the storage account from which to collect diagnostics. This has obviously caused some chagrin for those who have deployed new applications and setup SCOM to monitor them only to see no results in SCOM, particularly with performance counters. The Azure Management Pack documentation does document the fact that if a different name is used in the service configuration file, you must override the Diagnostics Storage Account Setting Name parameter for Windows Azure Hosted Service Total Discovery, but is not immediately obvious how to do so. This blog post will walk you through the process of resolving the problem. A special shout out goes out to my Microsoft colleague Bruno Saille who first alerted me to the solution for this issue.

Below is our starting point in Visual Studio 2010, where I’m using the 1.5 version of the Windows Azure Tools for Visual Studio. You can always just change the value in Visual Studio to the default setting name in SCOM before you deploy your application (this change to the connection name would be applied in the service configuration file and also the DiagnosticMonitor.Start call that references the connection name in your role OnStart method). If the application has already been deployed, you can modify the value in the configuration file from the management portal. If that’s too much trouble (as you are probably the SCOM admin), it’s easy to change the setting in SCOM to point to the correct diagnostics connection string, which is what we will show you in this post.

Presuming you want to make the change in SCOM, we will start in the Authoring view of the Management Console. Select the Object Discoveries node under Management Pack Objects, as seen below. I have scoped the Object Discoveries pane down to Windows Azure Role Instances to make my deployment easy to find.

Find your desired target whose name should look something like Windows Azure Hosted Service Total Discovery (<your hosted service name>) as seen below. Right-click on the item and select Properties from the popup menu.

You will now see the Properties dialog, as seen below. Select the Override… button, which will present a fly out menu. In the fly out menu, select the For all objects of class:… menu selection.

You will now see the Override Properties dialog, as seen below. Click on the Override check box for the Diagnostics Storage Account Setting Name parameter.

That row you selected now becomes enabled. Your dialog should now look as seen below, where we will override the existing DiagnosticsConnectionString value.

Change the value to look like below, by entering Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString into the Override Value field (or whatever other name you may have specified in the service configuration file).

Now, use the scroll bar to display the columns not currently in view to the right, if you can’t see all of them. Your next step is to select the Enforced check button for this parameter, as seen below. Choose the Apply button.

Your view should now look similar to the screen below. Verify that the Effective Value field has your new value, and that the Enforced box is checked. Next click the OK button. This will take you back to the original Properties dialog.

From the Properties dialog, you can select the View Summary button in order to verify your override, as seen below. When you’re done, close the summary dialog and then select the OK button on the Properties dialog to dismiss it.

It may take some time for the updated discovery to take, so you can hurry it along a bit by going to the Services administrative tool and restarting the System Center Management service, as seen below. Please be careful not to bother any other System Center services.

Return now to the Management Console, select the Monitoring view, and then select the Hosted Service State node. Select the desired hosted service. After some time you will eventually see the Diagnostics Storage Account Setting Name update to your overridden value. You may have to refresh this in order to see the updated value, but please be patient as this can take a while. In my case it took about 10 minutes after recycling the service.

You should be all set now, and I would suggest that you next go to a performance view and wait for your counters to show up. Of course, be sure that you are looking at a performance view with counters you are actually writing out with Azure diagnostics in your role (two common counters are % Processor Time Total and Memory Available Megabytes). Please be patient as it may take up to 10-15 minutes before you begin to see results flowing into your performance views.

That’s all for now, and I hope this helps.

Testing Windows Azure Web Apps with Synthetic Transactions in SCOM 2007 R2

Walter Myers III — Wed, 28 Dec 2011 20:53:29 GMT

Synthetic transactions with SCOM are one way you can automate a set of web requests to test various aspects of your Windows Azure web applications. The SCOM 2007 R2 Operations User Guide defines synthetic transactions in the following manner (I mashed up the text below a bit to read better):

“In Operations Manager 2007, synthetic transactions are actions, run in real time, that are performed on monitored objects. You can use synthetic transactions to perform HTTP requests to check availability and to measure performance of a Web page, Web site, or Web application.

For example, for a Web site, you can create a synthetic transaction that performs the actions of a customer connecting to the site and browsing through its pages. For databases, you can create transactions that connect to the database. You can then schedule these actions to occur at regular intervals to see how the database or Web site reacts and to see whether your monitoring settings, such as alerts and notifications, also react as expected.”

This is a powerful feature that should not be overlooked when monitoring your Windows Azure applications with SCOM. In this article, we will walk through a simple scenario of creating a synthetic transaction that records a few web requests. We will then verify operation by looking at the generated events in the Cerebrata™ Azure Diagnostics Manager tool, as well as looking at performance counters in the Operations Console along a short timeline.

To create a synthetic transaction, launch the Operations Console and select the Authoring pane. Select the Web Application node, as seen below. Right-click the node and select the Add Monitoring Wizard… option from the popup menu, as seen below.

You will now see the Add Monitoring Wizard dialog, where you will now select Web Application in the list, and select the Next button.

In the General Properties wizard page, enter a friendly name for your test, a description, and be sure to select your management pack at the bottom. Select the Next button.

Next, you will come to the Web Address wizard page. Here, enter the address of your Windows Azure web application, sans the http(s) protocol suffix. Select the Test button to verify access. The test results will appear immediately below. Everything looks good here, so we will proceed.

In the Choose Watcher Nodes wizard page, you will select the agent managed computers that will access your web application. By default, I just selected the current server as I have a single managed agent computer, but notice that you can have multiple managed agent computers sending load to your web application, which is a very nice feature. Next, I set the synthetic transactions to run every 10 minutes, as seen below. Select the Next button.

In the Summary wizard page, you will see a summary, and will be given the option to record your browser session so you can test desired functionality.

Once you select the Create button, you will be taken to the Web Application Editor. The Web Application Editor allows you to capture navigation through your website, interactively recording a sequence of actions that you take. When you get here, go ahead and delete the web request that it populates, and then select the Start Capture button to capture your sequence of requests.

You will now see the Web Recorder has been launched and the Record button is pressed in the left-hand pane. Go ahead and enter the URL of your web application in the Address box, as SCOM doesn’t do this for you. In the right-hand pane you will then see your Windows Azure web application. The application I’m testing is a variant of the logging sample from Mike Kelly’s great article Take Control of Logging and Tracing in Windows Azure. You can my variant of the sample code at my blog here.

Basically, my actions are to switch to the Event Log page, and then send five events to the event log, as you can see below in the recording. Once done with the recording, I press the Stop button, which will close the recorder and return us to the Web Application Editor.

Now that we’re back at the Web Application Editor, as seen below, we can set flexible criteria that will generate an error or warning health state for each web request. When done setting your criteria for any given request in the list, use the Verify button to verify your criteria first, and if everything checks out, select the Apply button, and then close the editor.

Now you’re back at the Operations Console with your synthetic transaction setup. It may be up to 15-20 minutes before you will be able to see the results of your synthetic transaction.

If you want to make any changes, such as your criteria or the frequency of the transitions, double-click on the synthetic transaction you just created to launch the Web Application Editor again. You can make changes to criteria directly, or, for example, you can launch the Web Application Properties dialog by selecting the Configure settings button in order to modify the frequency of your web requests (as seen below). In this dialog, you can specify performance criteria by transaction response time, and you can even set performance counters to be collected against your web application when requests are sent.

In order to verify that your synthetic transaction is working, you have a number of options. First, you can view the synthetic transactions that monitor Web applications listed in the Web Applications folder of the Monitoring pane, as seen below.

You can use any number of third-party tools that will allow you to view the logs from Windows Azure diagnostics. Below is a screen shot from the Cerebrata™ Azure Diagnostics Manager tool, verifying that I am producing the five events every ten minutes as specified previously in the Web Application Editor.

Next, we will return to the Operations Console to verify that our test application is experiencing the load we are placing on it with the synthetic transaction we created. I selected the ASP.NET Requests/Sec performance counter, which verifies that I have performance “spikes” every ten minutes. This opens up interesting scenarios for load testing and generating alerts based on performance counters.

Hopefully you can see the power in SCOM 2007 R2 to test functionality of your Windows Azure web applications, and measure performance as well with the ability to leverage multiple agent managed computers. One thing I didn’t show is the ability to run the tests manually in the Web Application Editor and see full results of the web requests in a highly detailed manner. This article only scratches the surface, so be sure to dig in and see all of the great functionality that SCOM provides.

Grooming Collected Diagnostics Data from Windows Azure Storage in SCOM 2007 R2

Walter Myers III — Wed, 28 Dec 2011 20:11:20 GMT

Recently, I had a colleague ask me how to groom diagnostic data collected in Windows Azure storage from Windows Azure applications. Since he is using SCOM, there is a ready answer for configuring regular grooming of Windows Azure diagnostic data, thus saving on storage consumption costs and preventing massive quantities of data from accumulating over time. Below I will provide a quick walkthrough to configure SCOM 2007 R2 for Azure diagnostics storage data grooming. Please note that this article assumes you have a working installation of SCOM, and that you have the Windows Azure Management Pack installed. If you need instructions on doing so, please see my previous blogs.

From the Operations Console, select the Authoring view. From here, select the Rules node under the Management Pack Objects tree. Regardless of how your view in the right-hand pane might look, the first thing we want to do is to “scope” the view down to make it more manageable for our purposes. If it is un-scoped, which is the default, your view will look similar to below. There are many, many object targets with thousands of rules included in SCOM. Go ahead and select the Scope button on the toolbar.

Upon selecting the Scope button, you will be presented with a dialog box. Type “azure” in the Look for: textbox, and select the View all targets radio button, as seen below. From here, select the following targets:

Windows Azure Hosted Service
Windows Azure Role
Windows Azure Role Instance

Your dialog should look similar to below. Select the OK button.

You will now see the three targets you selected with their out of box rules listed beneath the target heading. Notice that for Windows Azure hosted services, there are several rules enabled by default, each of which generates alerts if the conditions of the rule are met. For each Windows Azure role instance, there are default performance counters (which I covered in a previous blog). Now with respect to a Windows Azure role, we have three grooming rules, as follows:

.NET Trace Grooming
Performance Counter Grooming
Event Log Grooming

Note that these are the only diagnostics data sources groomed because these are the only sources the SCOM Azure Management Pack will collect diagnostics data from currently.

Go ahead and collapse both the Windows Azure Hosted Service and Windows Azure Role Instance targets, so your view will look as below.

Now right-click on the Windows Azure Role .NET Trace Grooming item under the Windows Azure Role target. In the flyout menu, s elect Overrides | Override the Rule | For all objects of class: Windows Azure Role.

You should now see the Override Properties dialog, as seen below. Select the Override checkbox for the Enabled parameter, and change the Override Value field to True. Next, select the Override checkbox for the Grace Period Hours parameter, and change Override Value field to the desired number in hours. I chose 168 hours, which means SCOM will delete collected trace logs that are over one week old. Finally, select your management pack as the destination management pack at the bottom of the dialog. You can then select the Apply button, and then eventually the OK button to dismiss the dialog.

Do likewise for Performance Counter grooming and Event Log grooming.

Next, let’s verify the work we have done. Right-click on the .NET Trace Grooming item, and select Overrides Summary from the popup menu.

You should now see the summary, as below. Verify that grooming is enabled and that you have the desired grooming period in hours.

You can also right click on the Windows Azure Role target and select the Overrides Summary popup menu item to see all of the overrides for that target, as seen below.

That is all for now. Please feel free to let me know if any questions.