Microsoft Monitoring Agent, System Center Operations Manager and Visual Studio Application Insights

Was this helpful? Share It!

Microsoft Monitoring Agent, System Center Operations Manager and Visual Studio Application Insights

Rate This
  • Comments 4

Since the release of System Center 2012 R2 Preview (and more after GA was announced) a lot of people asked me why did we rename the Operations Manager agent to "Microsoft Monitoring Agent"? Some information that went out together with the GA of SC2012R2 can be found at the following link: http://technet.microsoft.com/en-us/library/dn465154.aspx    
And here is a post from Marnix, System Center MVP http://thoughtsonopsmgr.blogspot.com/2013/09/scom-2012-r2-hello-mma-microsoft.html    
Essentially, Microsoft Monitoring Agent is not *only* the SCOM agent  anymore - the agent is now licensed with System Center OR with Visual Studio. When it was first released, it could already be used when reporting to SCOM (for monitoring), and it could also be used for standalone IntelliTrace collection (diagnostics scenario, more geared towards Dev/App owners). Read more in these other blog posts by Larry: Introducing Microsoft Monitoring Agent and Performance Details in IntelliTrace.

Enter ‘Application Insights’

With Microsoft Monitoring Agent 2013 Update Rollup 1 (at the time of this writing available as a preview), Microsoft Monitoring Agent can now also be used to report APM data to the brand new Application Insights Preview feature in Visual Studio Online that was announced a couple of weeks ago. Application Insights is an Azure-backed SaaS solution allowing teams to “[…] get insights from monitoring and going back and make the app better. Some people call it DevOps [...] but it's a sort of holistic view of the application: 360 degrees telemetry data like usage, performance, exception data, crash data, all that you need to have in live site to know how well your application is doing.[…]” (see the complete interview to Soma here).

You can read more also on
http://blogs.msdn.com/b/somasegar/archive/2013/11/13/visual-studio-2013-launch-announcing-visual-studio-online.aspx
http://blogs.msdn.com/b/visualstudioalm/archive/2013/11/13/announcing-application-insights-preview.aspx    
Application Insights 360 Dashboard

So what powers some (but not all) of the data that you have at your fingertips in Application Insights – like you might have imagined - is the APM agent within MMA: the same APM agent you can use with OpsMgr. And in Application Insights you’ll see the same familiar data you see in OpsMgr such as exceptions and performance events (which can be exported to IntelliTrace format), and performance counters, but this time in a multi-tenant SaaS solution specifically designed for DevOps teams.

APM Events in Application Insights

MMA 2013 UR1 Preview is available as a standalone download from Microsoft Download Center (as well as from the Application Insights Preview itself, within the Visual Studio Online portal) and it is the first version of the agent that allows the agent to connect to both on-premises System Center OpsMgr systems as well as to the SaaS service.
http://www.microsoft.com/en-us/download/details.aspx?id=41143
http://msdn.microsoft.com/en-us/library/dn481094.aspx
Microsoft Monitoring Agent - Select Connect to Application Insights

NOTE: Keep in mind that at the time of this writing, this is a CTP (“Preview”) release of the agent. It is not supported by CSS for non-Visual Studio Online-related scenarios. Even though we are not currently aware of any major compatibility issues between this CTP and SCOM (or when multi-homing between Application Insights and SCOM), only very limited testing was done for this agent working together with SCOM at this stage. We encourage SCOM customers NOT to use it in their production environments and wait for the final Microsoft Monitoring Agent 2013 Update 1 release.

In the future, anyhow, dual homing could be used to let your agent differentiate what data to send to which solution: i.e. send only alerting and performance information necessary for monitoring and triaging production issues to the on-prem System Center Operations Manager system, while the detailed and much more verbose code-level information can be sent for developers to consume in our multi-tenant SaaS APM offering within Team Foundation Service online (so you don’t have to worry about managing extra storage for APM data in the SCOM database), or to SCOM (maybe only data from some application, i.e. ‘PROD’ applications), or to both systems in various combinations – based on environment, project, operational model, processes and teams/ownership… It is for example very practical to use Application Insights to conduct functional and load tests in development and test environments – without the need to stand up another OpsMgr infrastructure, or to affect the scale and performance of the one that is designed to handle ‘prod’ data – to feed rich and actionable diagnostic information into the development lifecycle, to improve those applications even before they go in production.

Maarten, one of the System Center MVP’s, has also started a series of post on Application Insights where he started sharing his perspective about the powerful hybrid monitoring scenarios that have been enabled when using Microsoft Monitoring Agent with Application Insights and with System Center 2012 R2.

APM for Azure PaaS

Added benefit - MMA, when used with Visual Studio online, can also be installed in Azure Cloud Services's instances (PaaS) - which was not a supported scenario in System Center (see this post where I mentioned this before). This is the first time we are able to offer true APM monitoring for Azure PaaS. In OpsMgr, agents are uniquely identified by their FQDN (Fully Qualified Domain Name), and everything in SCOM from connector to DB to DAL to SDK - all these components rely on agent names. Machine names in most corporate networks are well-defined pieces of information, follow a logical naming convention, and rarely change. SCOM Management Servers also rely on Kerberos/AD on premises and/or certificates (again using the FQDN) to authenticate the agents, and expect to only be talking to ‘well known’, pre-authorized, machines. But with Azure Cloud Services (PaaS) and IaaS (in certain configurations) you can have any number of cloned, identical, elastic instances of 'roles' (worker and web) deployed and running at any given time which appear and disappear as you scale them up and down. Machine names don't last and don’t matter much in Azure PaaS like they do on-prem... and it is much more natural to have Azure send APM data to Azure, not to on-prem - which would otherwise require opening inbound ports in your perimeter - remember the agent initiates connection to the infrastructure it reports to, be it a SCOM Management Group, or Application Insights in the cloud. The agent includes a brand new connector that can talk web-friendly protocols to report to the SaaS offering, which is a very different backend than an OpsMgr Management Server/Group. Application Insights uses a newly built backend running in Azure, written with Cloud-First principles. The way to authenticate to the service is thru an ‘application key’ (which represents the Application Insights tenant); the cloud service does not use the machine’s identity. You place the application key in the application’s configuration file, and Visual Studio allows you to package/bundle a script to silently install and configure the agent automatically, so that every time your PaaS roles are re-deployed, you will have the agent installed on it. Machines come and go, applications stay, and they need to be monitored – those applications and their lifecycle are what Application Insights and Visual Studio Online are all about. For infrastructure-level info you don’t need an agent, instead: from System Center Operations Manager, you can of course keep using the Azure Management Pack, which polls from the Azure Management API and does a better job to create/dispose of those ‘elastic’ objects that come and go (thru discoveries); if you are only in the cloud (=no on-prem infrastructure) you can find that type of OS-level info (CPU/Memory/Disk) in Azure Portal.

Availability Monitoring

Availability information (and other metrics such as external response time) that is tracked in Application Insights comes from synthetic tests providing an ‘Outside In’ perspective: single URL probes or Visual Studio webtests. If you are one of my OpsMgr readers, you would have probably understood this is backed by Global Service Monitor – the same service, offering ‘watcher nodes in the cloud’, that you can attach to OpsMgr.

More than just APM (as we knew it in System Center)

More explicit instrumentation can be added to apps in various ways, when reporting to Application Insights. These include:

  • Client side Usage monitoring : Client-side monitoring instrumentation in Application Insights is a completely different solution than the one in OpsMgr. First, the focus is on usage, visitors, and their experience – but more in the analytics sense, rather than with the alerting angle of the one in OpsMgr. Second, enablement is different: Application Insights provides you with a javascript snippet that can be added to any website, also if not .Net – unlike in OpsMgr where .Net server-side monitoring is used to hook up automatic injection of javascript – but the change must be done by a developer. The manual method in the end proves more compatible with many applications and browsers.
  • Server SDK's by which you can instrument logging for custom metrics in your code and have it report to the service directly
  • Client SDK for Windows Phone 8 apps by which you can instrument logging for custom metrics in your code and have it report to the service
  • Deployment information can be collected – see post from Charles – this is extremely useful to understand if changes in performance or reliability are related with deployments of new versions of the app/service
  • Beautiful Customizable Dashboards and a fresh, modern UI on top of all this data

How can I try it out?

Application Insights is currently in preview and you need an invitation to try it out. You might want to go to www.visualstudio.com and register for a VSO subscription and add yourself to the waiting list by clicking the blue “Try Application Insights” button.

Some more links

Series of videos on how to use it http://channel9.msdn.com/Series/Application-Insights-for-Visual-Studio-Online     
Forum on MSDN http://social.msdn.microsoft.com/Forums/vstudio/en-US/home?forum=ApplicationInsights
Documentation on MSDN http://msdn.microsoft.com/en-us/library/dn481095.aspx

Leave a Comment
  • Please add 8 and 8 and type the answer here:
  • Post
  • Very useful information, Daniele. The features and capabilities you discuss here are very exiting, especially for developers of cloud apps. The only concern I have is in the last bullets:

    Application Insights provides you with a javascript snippet that can be added to any website, also if not .Net – [...] – but the change must be done by a developer.

    Why is it necessary for a developer to add the js to their code, when APM is supposed to inject this "seemlessly" in the .NET app when monitoring on-prem apps?

  • It is a completely new implementation of client-side monitoring and it isn't hooked up for automatic injection. The older implementation for onprem is (when reporting to onprem). I would suggest you go to the forums and ask the owners whether there are plans to hook up automatic injection when reporting to Application Insights too. I don't directly work on that team, albeit I have worked on various aspects of it in the past.

    Besides, for many agile teams who write modern applications, especially if using Visual Studio Online with Azure with continuous integration, fixes get checked in and sites and services get redeployed as often as multiple times a day - so adding two lines of javascript really is no biggie in my opinion. Even for other sites/blogs/content management systems it is often possible to totally customize 'themes' that come in with their html/css and javascript includes where you can place the javascript.

    Also, the client-side implementation for Application Insights seems to be currently more focused to an angle of analyzing and understanding usage patterns, while the one in SCOM is event/alert driven.

  • Thanks for the additional information. I suppose, when it comes to .Net apps, I would expect the same functionality as we had with native APM modules - where we did not need to include additional code in the app in order to achieve similar monitoring capabilities.

    Are you saying that it's as simple as including the js in the main page head of an app developed in .net, kind of like google analytics or some other analytics app works, and we have all the fruitful information we need to diagnose e2e the app?

  • Yes is just one of those javascript snippets, everybody can place it in, usually... it is a common practice in the industry. This is only for CLIENT-SIDE (in the browser) information. The information reported by this is different than the one in OM: look at the videos I linked, such as this channel9.msdn.com/.../Setting-up-usage-monitoring-in-Application-Insights-for-Web-sites and you should get a sense of it.

    The server-side APM information on .NET code performance and exceptions call stacks are instead the same, and both OM and AppInsights can collect and display those. AppInsights also gets additional info from the agent, that OpsMgr doesn't use.

    It's about giving the right information to the right people. I wrote earlier in other blog posts about the balance between needing a lot of verbose information for Devs to reproduce, understand and fix code defects, while the need for Ops to have minimal and actionable alerts for when things are on fire. OpsMgr is more geared towards the latter, AppInsights more towards the former. And there is obviously some part that is in common. Makes sense?

Page 1 of 1 (4 items)