The CRM Online performance topics listed below are general ideas that should be considered when working through performance issues.  I will do my best to update this blog entry with new content when it becomes available.

Questions To Ask

Before digging in, we’ll need to get a better understanding of the performance issues that the user is experiencing.  I’ve listed some of the questions that you’ll want to consider below.  The answers to these questions will help you determine what areas you need to focus your troubleshooting efforts on.

• General
• What OS is being used?
• What is the total memory on the machine?
• Can the issue be reproduced in another CRM Online environment?
• Does the issue repro in the CRM web client, CRM Outlook client, or both (it will be important to verify this)? 
• If the issue reproduces in the CRM web client, troubleshoot there first as there are other layers of the CRM Outlook client that could slow you down to reaching the root cause.
• What version of Office is being used (only relevant if we’re dealing with a CRM Outlook client issue)?
• If the issue is occurring on the CRM Outlook client, what Update Rollup is currently on the user's machine (this can be verified by getting the version of the crmaddin.dll…located in the Microsoft Dynamics CRM\Client\bin directory or by clicking the About Microsoft Dynamics CRM button within the Outlook client?
• Magnitude of the issue
• How many users are experiencing the issue?
• Can the issue be consistently reproduced, or does it happen sporadically?
• Does the performance issue happen only in certain areas of the CRM Online application?  If so, what area(s)?
• If the issue is happening sporadically, does it seem to be occurring during a certain time of the day, or when a certain process is running?
• Performance related
• What type of internet connection does the user(s) have?
• Do multiple users experience this issue during the same period of time, or does it appear to be a one-off performance issue?
• What datacenter is your CRM Online deployment in?  This can be determined based on your deployment URL.  At the time of this post, the following information links the CRM Online URLs to their correlating datacenter:
• --https://<orgname>.crm.dynamics.com = North America
• --https://<orgname>.crm4.dynamics.com = Europe
• --https://<orgname>.crm5.dynamics.com = Asia
• What country is the user experiencing the issue accessing CRM from?

Network Issues

One of the first things that we can do to verify where the issue lies is to verify whether or not the network (mainly bandwidth or latency) seems to be part of the problem.  Low bandwidth and high latency will negatively impact the end-user CRM Online experience.  It is a good idea to test WAN performance at the beginning of the troubleshooting process.  Our CRM Online R&D team has created a CRM Online Network Performance tool that gathers client network configuration data.

• Log in to your CRM Online organization
• Navigate to the appropriate relative path for your organization
  https://<orgname>.crm.dynamics.com/tools/diagnostics/diag.aspx
• Click the Run button for All Tests.

The client latency tool provides insight into the network conditions you are working with.  It’s important to note that CRM Online is designed to work best over networks with latency under 150 milliseconds and bandwidth greater than 50 KB/s.  These numbers are provided as general performance guidelines rather than guarantees.

Browser Related

Client-side browser settings can have a significant impact on the performance of the CRM Online application.  The items listed below are a few items to consider:

• For optimal CRM Online performance, use the latest IE browser (currently IE9 at the time of this blog).  IE9 includes JavaScript acceleration functionality to help improve client-side performance.
• By default, Windows Internet Explorer 7 and earlier versions limit the number of files that you can download at one time to two.  Windows Internet Explorer 8 limits the number of files that you can download at one time to six.  This change reflects the faster connection speeds that are now typical for most users.  Note that for dial-up connections, the limits from earlier versions still apply.  If you find that you are running on a version prior to IE8, you may want to follow the steps listed within the “Optimizing and Maintaining the Performance of Microsoft Dynamics CRM Clients” to increase the MaxConnection settings.
• IE Internet Option Settings:
• Internet Options – General – Browsing History – Temporary Internet Files and History Settings – Check for newer versions of stored pages.  Verify that Automatically is selected.
• Internet Options – General – Browsing History – Temporary Internet Files and History Settings – Disk space to use.  Verify that a value from 50 to 250 MB is selected (the closer to 250 MB, the better).
• Internet Options – General – Browsing History.  Verify that “Delete browsing history on exit check box is cleared to prevent the deletion of temporary Internet files each time that the browser is closed.
• Internet Options – Advanced – Security.  Verify that the “Do not save encrypted pages to disk” check box is cleared, and then click OK.

Note:  After the client-side browser settings on a computer are modified, loading the Dynamics CRM Web client for the first time (the “cold” load) may take several moments longer than usual. Subsequent page loads (“warm” loads) will show improved performance, however, as many of the Microsoft Dynamics CRM Web pages will have been cached during the initial load.

Customizations

When optimizing the performance of Microsoft Dynamics CRM clients, keep in mind that client performance is directly affected by the:

• Number of (limit where possible):
• Columns and grids in use
• Rows returned per page
• Quick find enabled fields
• Sub-grids used on a page
• Controls on a form (Form Design)
• Controls on the ribbon
• Client-side events (OnLoad, OnSave, OnChange)
• Complexity and number of visualizations used in dashboards
• Use of Jscript and Plugins. Advanced Jscript can add a significant amount of time to, open, save, close and other events especially if calls are made to Microsoft Dynamics CRM or other system. Plug-ins can also affect performance when saving.

If the issue is general slowness, can the user reproduce the issue in a CRM Online test environment with no customizations?  It will be important to determine whether or not the issue is potentially related to customizations from the environment where the issue has been reproduced.  Can any customizations be disabled in the environment for testing purposes (ex. OnLoad events)?  If subgrids are contributing to the long form load times, it may be helpful to contract the subgrid sections as a test to see if performance improves.  It is possible to contract sections by default on the load of the form, then expand the form shortly after.

CRM Online Configurable Settings

• EnableRetrieveMultipleOptimization
• When you perform RetrieveMultiple queries on large datasets in Microsoft Dynamics CRM 2011, such as pulling together a CRM Online view, you experience slow performance.  This problem occurs because large datasets cache the Microsoft SQL query plans for the RetrieveMultiple queries.  The RetrieveMultiple optimization for the basic depth is required for the "local depth" read privilege and for the "deep depth" read privilege
• http://blogs.msdn.com/b/crminthefield/archive/2011/09/28/crm-online-performance-enableretrievemultipleoptimization.aspx
• http://support.microsoft.com/kb/2535245

CRM Outlook Client Performance

There are several factors, specific to the CRM Outlook client, that can increase the amount of time that certain actions take to complete (ex. CRM Outlook client opening, navigating to a view, etc.).  You’ll find some troubleshooting items below to test out:

• Limit the number of views per entity
• Limit the number of pinned views per entity
• Limit the frequency at which customizations are published
• Limit the number of columns within views

You may also see sluggish CRM Outlook client performance due to background polling activities that are taking place.  During these polling intervals, the CRM Outlook client is communicating with the CRM server (often times reacting to server-side changes).  These process will take place within your CRM Outlook client whether or not CRM functionality is actually being used at the time.  There are caches for various CRM features like address book, reminders, views, and metadata.  For example, one polling interval that takes place on the CRM Outlook client machine occurs when when the metadata file is being refreshed.  Metadata refreshes occur due to certain CRM Admin related activities, such as publishing customizations, creating new CRM security roles, or modifying existing CRM security roles.  To keep the metadata refresh minimal, publish your customizations at a less frequent rate.  Ultimately, the less that you publish customizations, the less you’ll see the metadata.xml file refresh.  If you have installed additional MUI packs (language packs) that you aren’t using, it will be in your best interests to remove the unused packs as those do add to the size of the metadata file.  There are also a set of registry keys that can be modified to tune the polling intervals.  For more information on these keys, please see the following KB:
http://support.microsoft.com/kb/2585157

Our CRM Online team is always attempting to improve the CRM Outlook experience.  This includes incorporating new functionality, as well as improving performance and resolving any existing CRM Outlook client issues.  Improving the CRM Outlook client in this way is done through Microsoft CRM Update Rollups.  When running into CRM Online performance issues within the CRM Outlook client, it's always a best practice to download and install the most recent Update Rollup.  Update Rollups can be found at the Microsoft Download Center.  You can find the most recent CRM 2011 Outlook client Update Rollup by doing the following:

• Navigate to:  http://www.microsoft.com/download/en/default.aspx
• Search for CRM 2011 Update Rollup
• Set the Sort by option to "Newest to Oldest"
• Once an Update Rollup is selected, you'll have two options to download for the CRM Outlook client:
• CRM2011-Client-KBxxxxxxx-ENU-i386.exe for 32 bit versions of the Outlook client.
• CRM2011-Client-KBxxxxxxx-ENU-amd64.exe for 64 bit versions of the Outlook client.

Tools

• Fiddler
• Run a Fiddler trace to see how long each section of the page opening process is taking.  When looking at the timeline in Fiddler, you should be able to identify which process is taking the longest.
• To capture CRM Online data, you’ll need to do the following:  Tools – Fiddler Options – HTTPS.  Check HTTPS Connects and Decrypt HTTPS traffic.
• For more information on Fiddler, review the following post:
• http://blogs.msdn.com/b/crminthefield/archive/2011/05/19/using-fiddler-http-debugger-for-crm-troubleshooting-and-performance-tuning.aspx
• SQL Tools
• In situations where more advanced SQL tracing is needed to determine long running queries and/or blocking, the Microsoft support team will need to be engaged.  All previously mentioned levels of troubleshooting should be completed prior to this step.

- Jon