Quick microbenchmarks in Visual Studio with Code Snippets

Quick microbenchmarks in Visual Studio with Code Snippets

Rate This
  • Comments 5

Parallelism is all about performance.  After all, in the majority of cases, introducing parallelism into code adds some level of complexity, and the primary reason we’re ok with that additional complexity is because we get great performance enhancements as a result.  As such, as we develop our parallel runtimes and libraries to help others parallelize their code, we pay a lot of attention to what things cost, ensuring that we have as little overhead as possible in our libraries so that users of our libraries get the most benefit out of their parallelization.

Given that, often while coding I’ll find myself wondering what a particular operation costs. More often than not, I’ll have multiple ways I can potentially do something, and I’m interested in understanding if there’s any significant difference between the approaches that could have a non-negligible impact on the performance of my code.  I found myself writing the same boilerplate code over and over in Visual Studio, setting up a stopwatch, writing long loops to invoke each of the two operations, performing some basic statistics at the end, etc.  Then I realized that I could save all of that time if I just created a Code Snippet for this in Visual Studio.

With my snippet, I now just start typing “perfcomp” into the Visual Studio editor:

image

and when I tab-tab to expand the snippet, I get code like the following:

var sw = new Stopwatch();

const int ITERS = 1000000;

while (true)

{

    Console.Write("A: ");

    sw.Restart();

    for (int i = 0; i < ITERS; i++)

    {

        // Operation A

    }

    var elapsedA = sw.Elapsed;

    Console.WriteLine(elapsedA);

 

    GC.Collect();

    GC.WaitForPendingFinalizers();

    GC.Collect();

 

    Console.Write("B: ");

    sw.Restart();

    for (int i = 0; i < ITERS; i++)

    {

        // Operation B

    }

    var elapsedB = sw.Elapsed;

    Console.WriteLine(elapsedB);

 

    GC.Collect();

    GC.WaitForPendingFinalizers();

    GC.Collect();

 

    Console.WriteLine("A/B     : {0}",

        elapsedA.TotalMilliseconds /

        elapsedB.TotalMilliseconds);

    Console.WriteLine("B/A     : {0}",

        elapsedB.TotalMilliseconds /

        elapsedA.TotalMilliseconds);

    Console.WriteLine("(A-B)/A : {0}",

        Math.Abs((elapsedA.TotalMilliseconds -

                  elapsedB.TotalMilliseconds) /

                 elapsedA.TotalMilliseconds));

    Console.WriteLine("(B-A)/B : {0}",

        Math.Abs((elapsedB.TotalMilliseconds -

                  elapsedA.TotalMilliseconds) /

                 elapsedB.TotalMilliseconds));

    Console.WriteLine();

 

    Console.ReadLine();

}

I can then easily substitute the two operations I want to test for “Operation A” and “Operation B”, and quickly get some basic comparisons about their performance.  The whole thing is wrapped in a loop so that I can easily run multiple comparisons, which also helps to avoid any discrepancies that may arise from first-time use.

To install this snippet, unzip the PerfComparison.snippet file from the attached file, and then add it in Visual Studio via the Code Snippets Manager available from the Tools menu.

I hope it’s helpful.

Attachment: PerfComparison.zip
Leave a Comment
  • Please add 5 and 4 and type the answer here:
  • Post
  • Ok... I've been trying, honestly. How is this better than calling a perfcomp(Action a, Action b) method?

  • Hi Marcel-

    For me, the benefit is three fold.  First, I don't need to write a perfcomp method into every project, nor add a reference to some library or add a using statement to bring the right functionality into scope: I simply type "perfcomp" into VS in whatever code file in whatever project I'm in, and the comparison is added for me.  Second, there is a cost to executing a delegate, and often times for the kinds of things I measure, the overhead of executing a delegate gets in the way of the measurement; by having access to the source, I can avoid that overhead by adding the operation directly into the loop.  Third, often times the things I want to measure don't fit nicely into such a pattern, e.g. doing some initialization before starting the stopwatch.

    All of those things make this particular functionality quite useful to me.  If you find more benefit from the appropriate of referencing a library with a perfcomp function, you should of course choose the approach that suits your needs better.

    I hope that helps.

  • Hi Stephen

    Do you have an estimate of the cost of evaluating a delegate compared to .. what ever you normally do? Would that be regular functions maybe?

    Lars

  • Answering questions like that is what this blog post was all about ;)

    To answer your question, I typed 'perfcomp', I added the following code:

       static int total = 0;

       static Action action = () => total++;

    and then for operation A I did:

       total++;

    and for operation B I did:

       action();

    I also reset the total to 0 before each phase. On my machine, operation B was about 2x the cost of operation A.

  • That is amazingly fast! I wonder if the JIT was able to inline the delegate?

Page 1 of 1 (5 items)