The Microsoft Dynamics CRM Blog
News and views from the Microsoft Dynamics CRM Team

Parallelism – Using Parallel.For with the CRM SDK

Parallelism – Using Parallel.For with the CRM SDK

  • Comments 3

Our guest blogger today is CRM MVP Aaron Elder with some of his characteristic “forward looking” observations.

Architect.Microsoft .NET 4.0 is almost upon us and should be released in the next couple weeks (April 12th, 2010). There are so many cool things in .NET 4.0 framework as well as the CLR and languages like C# 4.0. I recently spent some time playing around with one of my favorite new features found in the Parallel Extensions Parallel.For / Parallel.ForEach and to also try out optional parameters.

As you may have read in my post on the topic, you unfortunately will not be able to make use of .NET 4.0 for CRM plug-ins or other code that runs in-process with Microsoft Dynamics CRM 4.0 (Since CRM is compiled against the .NET 2.0 CLR). However, you can definitely use .NET 4.0 with Microsoft Dynamics CRM 4.0 if your code is running out-of-process, such as on a separate web site, console app, Windows service or integration. Fortunately, we can all look forward to using .NET 4.0 for CRM 5.0 extensions and more.

Parallel Extensions in .NET 4.0 provide all kinds of cool stuff, but at its core these extensions allow you to very quickly and very easily add multi-threading to your applications with very little effort. While the old method of creating and managing a thread pool is still available and still needed, for many simple scenarios you can shortcut all that coding and get threading with basically a single line of code!

Parallel.For(0, 500, i =>

{

// Do something here

});

So how does this perform? The answer is “pretty darn well”. On a single CPU machine, the upside is as expected negligible to slightly negative. But on a multi processor machine, the benefit is definitely noticeable! Behold:

image

As you can see, add one additional CPU to the mix, provides an approximately 40% increase in throughput.

To get these results, I used the following code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Diagnostics;
using System.Collections;

using ConsoleApplication1.CrmSdk;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            // Prime the pump, this prevents the serialization hit from affecting our results
            // in production code, we could pre-compile this or just take the hit the first
            // time a CRM Service is created.
            CrmService service = new CrmService();

            // Start the timer to see how long this takes
            Stopwatch watch = new Stopwatch();
            watch.Start();

            // Test different Parallelism options
            ParallelOptions ops = new ParallelOptions();
            ops.MaxDegreeOfParallelism = 2;

            // This loop uses the new multi-threaded extensions
            Parallel.For(0, 500, i =>
            {

            // This loop is the standard for loop
            //for (int i = 0; i < 500; i++)
            //{
                dynamic acc2 = Program.GetDefaultAccount("
http://xrm.ascentium.com");
                acc2.name += i;

                // Tried this three ways, both creating a new CRM Service each time
                // using a single shared service and caching one service per thread
                CrmService theService = Program.GetCrmService(Thread.CurrentThread.ManagedThreadId);
                Guid id = theService.Create(acc2);

                // Write out our progress
                Console.WriteLine(id.ToString("b") + " " + i + " " + watch.ElapsedMilliseconds);

            //} // For

            }); // Parallel.For

            watch.Stop();

            // How long did all that take?
            Console.WriteLine(watch.ElapsedMilliseconds);
            Console.ReadLine();
        }

        /// <summary>
        /// Built a defaulted Account record
        /// .NET 4.0 - Play around with optional parameters
        /// </summary>
        static object GetDefaultAccount(string websiteurl = null)
        {
            account acc = new account();
            acc.name = "Test Account";
            acc.numberofemployees = new CrmNumber();
            acc.websiteurl = websiteurl;
            return acc;
        }

        static Dictionary<int, CrmService> _services = new Dictionary<int, CrmService>();

        /// <summary>
        /// Get a cached copy of a CRM Service based on a thread ID
        /// </summary>
        static CrmService GetCrmService(int id)
        {
            CrmService currentService;

            // Was a service already created for this thread ID?
            if (_services.TryGetValue(id, out currentService))
            {
                return currentService;
            }
            else
            {
                // Configure the CRM service
                currentService = new CrmService();
                currentService.UseDefaultCredentials = true;
                currentService.UnsafeAuthenticatedConnectionSharing = true; // Speed Racer
                currentService.CrmAuthenticationTokenValue = new CrmAuthenticationToken();
                currentService.CrmAuthenticationTokenValue.OrganizationName = "AscentiumCrmDev";

                // Dictionary Objects aren't thread safe
                lock (((ICollection)_services).SyncRoot)
                {
                    // Cache the instance for this thread ID
                    _services.Add(id, currentService);
                }

                return currentService;
            }
        }
    }
}

But there are many ways to approach this and I tried several. The pattern above is what seems to work best. I made use of all the best practices that I blogged about many moons ago and was sure to use the connectionManagement configuration element to make sure the application was not limited to only 2 connections with the server. I wanted to try to focus my tests on the actual performance difference of the threading. Here are the results for the various other methods I tried:

1. Create a new service each iteration – This method involves creating a new CrmService within the “For Loop” and thus giving each thread its own copy of the service.

2. Single shared service for entire loop – This method involves creating a single CrmService instance outside of my “For Loop” and using it for all requests.

3. Single shared service for entire loop (2 threads max) – Same as #2, but I used the ParallelOptions MaxDegreesOfParallelism to limit the loop to only 2 threads. I wanted to see if I could “out smart” the Parallel Extensions framework and prevent extra unneeded context switching.

4. Each thread gets its own service – In this model, I created a factory that uses the current thread context to create a unique CrmService instance per thread. This method is what is illustrated in the code sample provided. This method yielded the best performance and is similar to other multi-thread solutions we have built for CRM in the past (only with a lot less code).

And here is how the various approaches performed:

image

Why did approach #4 perform the best and can you always use this approach? The reason this approach had the best performance is probably attributed to the internal plumbing of WebClientProtocol and WebRequest. The CrmService, like other .NET SOAP objects makes use of the SoapHttpClientProtocol, which descends from HttpWebClientProtocol. MSDN has this to say on the thread safety of this object:

The properties on this class are copied into a new instance of a WebRequest object for each XML Web service method call. While you can call XML Web service methods on the same WebClientProtocol instance from different threads at the same time, there is no synchronization done to ensure that a consistent snapshot of the properties gets transferred to the WebRequest object. Therefore, if you need to modify the properties and make concurrent method calls from different threads you should use a different instance of the XML Web service proxy or provide your own synchronization.

Internally, this all comes down to the WebClientProtocol and WebRequest objects, which are making synchronous and thus blocking calls to the actual Microsoft Dynamics CRM ASMX web services. By creating a separate instance of the CrmService per thread, we are reducing the amount of contention for the same resources and since we have upped the process maximum HTTP connections, we don’t have to wait for underlying connections to be made available either.

As MSDN notes above, details from the client are copied to each WebRequest object, it is important that you not use this factory approach with UnsafeAuthenticatedConnectionSharing across multiple threads when you are dealing with multiple user contexts. Since the system will not re-authenticate the user, all requests will be assumed to be happening by the same user. This is great for data migrations, but not so great for cases where you need CRM to know about the user making the call.

If you are still interested in Parallel Extension performance and optimization, this article is definitely worth a read: http://msdn.microsoft.com/en-us/magazine/cc163340.aspx

Enjoy and happy threading!

Aaron Elder

This posting is provided "AS IS" with no warranties, and confers no rights.



  • Great posts, loved reading it, thanks!

  • All this will only work with web references, right ? Will there be CRM 4 SDK DLLs compiled against the .NET 4 runtime ?

  • Hey Aaron,

    Great post.

    Thanks.

Page 1 of 1 (3 items)
Leave a Comment
  • Please add 3 and 8 and type the answer here:
  • Post