Parallel Programming with .NET : C++

PDC09 Parallelism Session Videos Now Available

toub — Sat, 21 Nov 2009 09:05:20 GMT

Attendees at PDC09 this past week were privy to quite a few sessions on parallel computing. Now that the videos of these sessions are online, you can view them as well from the comfort of your own home. Here are some of the key parallelism-related sessions from this past week:

Overview

The State of Parallel Programming

Managed code in Visual Studio 2010

Native code in Visual Studio 2010

HPC Server

Accelerating Applications Using Windows HPC Server 2008

Research and Incubation

Enjoy!

Debugging TPL apps in VS2010

DanielMoth — Sun, 15 Nov 2009 08:18:00 GMT

The new parallel debugger windows in Visual Studio 2010 (Parallel Tasks and Parallel Stacks) have had many fixes and updates.

I have refreshed the existing content and also added new material for Beta 2. Find links to all of it from my blog post on Parallel Debugging.

Cheers
Daniel

Debugging Support for Tasks

DanielMoth — Sat, 16 May 2009 07:55:00 GMT

Visual Studio 2010 has new debugger windows to support the Task-based programming model. Check out my blog post about the Parallel Tasks window.

Parallelism Videos Galore

toub — Tue, 18 Nov 2008 11:14:00 GMT

It's been a hectic and exciting few weeks, and we on the Parallel Computing Platform team have been having a great time talking with customers all over the world, at the PDC, at TechEd EMEA, at DevConnections, through Channel 9, and more. A lot of the resulting material is now available online for viewing, so do check it out if you're interested.

PDC 2008 videos

Parallel Programming for Managed Developers with the Next Version of Microsoft Visual Studio

Parallel Programming for C++ Developers in the Next Version of Microsoft Visual Studio

Microsoft Visual Studio: Bringing out the Best in Multicore Systems

Concurrency Runtime Deep Dive: How to Harvest Multicore Computing Resources

Parallel Symposium: Addressing the Hard Problems with Concurrency

Parallel Symposium: Application Opportunities and Architectures

Parallel Symposium: Future of Parallel Computing

Research: Concurrency Analysis Platform and Tools for Finding Concurrency Bugs

The Concurrency and Coordination Runtime and Decentralized Software Services Toolkit

Channel 9: Visual Studio 2010 and .NET Framework 4.0 Week

Using the Parallel Extensions to the .NET Framework

Native Parallelism with the Parallel Patterns Library

Debugging Parallel Applications with Visual Studio 2010

TechEd EMEA 2008

Parallel Programming for Managed developers with Visual Studio 2010 and .NET Framework 4.0

Tech chat with Microsoft's parallel guru Steve Teixeira

Why we Need the Task-Based Programming Model Introduced in .NET 4

The Inexorable Drive to Many-Core Processors

Other (OnMicrosoft, University of Washington Colloquium, ...)

Concurrent Programming on Windows

Visual Studio 2010 Part 1 of 2- Support for Parallelism

Microsoft's Parallel Computing Platform: Applied Research in a Product Setting

And if you're more interested in the written word, check out this recent tech brief in Redmond Developer News:

Tech Brief: Parallel Extensions

All of this is, of course, in addition to all of the previous material that's been posted online, such as was mentioned at The Channel 9 videos are rolling in, Webcasts on Parallelism from France, Parallel Extensions Demo Fun on Channel 9, More Channel 9 Parallel Extensions Goodness, New PLINQ video on Channel 9, Task Parallel Library on Channel 9, Burton Smith on Channel 9, Parallel Extensions on .NET Rocks, Parallelism in October 2008 MSDN Magazine, and Another Parallel Extensions screencast.

Enjoy!

Concurrent, Multi-core Programming on Windows and .NET

toub — Tue, 28 Oct 2008 08:51:00 GMT

Thanks to everyone who attended our PDC pre-conference session yesterday on parallelism and concurrency! We had a wonderful turnout at the event, and David, Joe, and I all had a terrific time. Attached to this post are the slides we presented.

(It turns out that the PDC site does allow you to submit an evaluation for a precon. If you attended, we'd love your feedback.)

Native concurrency, tools, and TM

toub — Mon, 13 Oct 2008 08:32:00 GMT

The Parallel Computing Platform team at Microsoft is working on much more than Parallel Extensions to the .NET Framework...

A few weeks back, Charles from Channel 9 spoke with us about our efforts on supporting concurrency and parallelism in native code; the video of that conversation was released this week, and you can view it at http://channel9.msdn.com/posts/Charles/The-Concurrency-Runtime-Fine-Grained-Parallelism-for-C. Niklas Gustafsson, an architect on our team, was also recently on Intel's Parallel Programming Talk podcast discussing the Concurrency Runtime; you can listen to that podcast at http://software.intel.com/en-us/blogs/2008/10/07/parallel-programing-talk-microsoft-concurrency-runtime/.

Charles also spoke with our tools team, which for Visual Studio is focusing on dramatically improving the debugging and profiling experience for when writing parallel applications; you can view that video at http://channel9.msdn.com/posts/Charles/Parallel-Computing-Platform-An-Integrated-Approach-to-Tooling/.

In other cool news, our team focused on implementing an experimental transactional memory system has just started up a blog (http://blogs.msdn.com/stmteam/):

"This blog is about transactional memory with entries written by the team at Microsoft that is implementing an experimental transactional memory system in .NET. While we work in the Developer Division’s Parallel Computing Platform product group, we are not working on a product release at this time. Instead we have been incubating, researching, and experimenting with transactional memory.

Our goal is to create a TM system that lives up to its promise. This work has lead us to what we think is some industry-leading work – but we have been so busy doing it that we haven’t had a chance to talk about it; share what we have found; and more importantly hear what the community thinks of this effort.

This blog is our first step to remedy that. Welcome to the Transactional Memory blog here at MSDN."

The first post on the blog is quite interesting and is definitely worth a read.

Announcing the "Parallel Programming in Native Code" Blog

essey — Sat, 07 Jun 2008 09:24:00 GMT

Our colleagues working on native concurrency have just launched the Parallel Programming in Native Code blog. We’re all on the same team and work really closely to ensure that the right programming models are exposed both for managed and native consumers, so you will be able to use your language of choice.

Here’s a little bit from the Welcome post explaining what they’re all about.

“…We’re looking to help overcome these barriers and improve productivity for C++ developers.

We want to make expressing concurrency easier by adding abstractions for describing opportunities for parallelism that maintain the original intent, readability and composability of the code.

We’re trying to minimize the number of new concepts we introduce to ensure that the model remains approachable and familiar to mainstream C++ developers.

We’re exploring ways for developers to overcome the challenges of shared memory by providing a means of describing applications as isolated components that communicate with a rich message passing interface.

We’re looking at providing a common and efficient Concurrency Runtime that supports a broad range of parallel abstractions and removes the need for developers to build this infrastructure.”

Check out the post to see an example of their work in action.

Mandelbrot in Parallel with C++/CLI

toub — Sat, 07 Jun 2008 01:11:44 GMT

Since the release of the December 2007 CTP of Parallel Extensions, we've received several questions about whether Parallel Extensions can be used from C++/CLI. In short, yes, it can! (It can be used with any .NET language, one of the beauties of this functionality being provided through a library.) To demonstrate, we included in the June 2008 CTP an example of doing just that, using Parallel Extensions to compute the Mandelbrot fractal. (To use Parallel Extensions in your own C++ project, you'll need to add a reference to both System.Threading.dll and System.Core.dll.)

The sample application loads and displays the Mandelbrot fractal. You can click-and-drag to move around in the fractal, and on each move the image will be redrawn. However, as soon as you move again, the previous rendering will be canceled and a new rendering will begin; this keeps the UI responsive. You can also double-click with the left mouse button to zoom in or double-click with the right mouse button to zoom out. The application starts out in sequential rendering mode, signified by the "(1x)" in the title bar of the app. If you press the 'p' key while the application is running, it will switch over into parallel mode, and the title bar will display "(2x)" on a dual-core, "(4x)" on a quad-core, and so forth. The title bar also displays the time it took to perform the last rendering, and the image will be re-rendered every time you push 'p' or 's' (for sequential), so you can quickly flip back and forth between the two modes to see the difference in rendering times.

While Parallel Extensions can be used from C++/CLI, it's admittedly not the best experience due to C++'s current lack of support for lambda expressions and anonymous methods. In C# I could write a method something like the following to render an image (this is just pseudo-code):

static Bitmap Create(MandelbrotPosition position, int width, int height)
{
    Bitmap bmp = new Bitmap(width, height);
    Parallel.For(0, height, delegate(int j)
    {
        for(int i=0; i<width; i++)
        {
            bmp.SetPixel(i, j, RenderPixel(i,j,position));
        }
    });
    return bmp;
}

This works because C# supports closures and is able to capture references to variables like width, height, position, and bmp in order to use them inside of the delegate. In C++/CLI in contrast, I need to write this more like the following:

static Bitmap^ Create(MandelbrotPosition position, int width, int height)
{
    Bitmap^ bmp = gcnew Bitmap(width, height);
    RenderImageData ^rid =
        gcnew RenderImageData(bmp, position, width, height);
    Parallel::For(0, height, gcnew Action<int>(
        rid, &RenderImageData::RenderRow));
    return bmp;
}

Not shown, I've defined RenderImageData as a class that stores the various values and that exposes a RenderRow method that implements the inner loop which renders a single row. I can call that RenderRow method in parallel by passing a delegate to it to the Parallel::For method.

Some of you who are familiar with OpenMP will note that this particular task of parallelizing a loop is OpenMP's bread-and-butter and doesn't require some of these funky gyrations to manually implement a closure, since the compiler handles it much of it for you. As it turns out, the nice folks in Visual C++ land implemented support for OpenMP 2.0 in C++/CLI, so you can actually use OpenMP from managed code. To do so, in your project you need to throw the /openmp switch, which can be done by setting the OpenMP Support value to Yes in the property pages for the project under Configuration Properties | C/C++ | Language. Under Configuration Properties | General, you'll also need to set Common Language Runtime support to Common Language Runtime support (/clr). Now, on a for loop (like the one used to render the image), you can add an OpenMP pragma like:

#pragma omp parallel for
for(int j=0; j<height; j++) { ... }

and the system will automatically parallelize the rendering of the loop (you may need to help the system to find the OpenMP runtime library VCOMP90.DLL, such as by copying that DLL into the directory containing your binary). Of course, the OpenMP approach has its downsides, too, besides all of the configuration we just went through. For one, OpenMP doesn't provide an easy mechanism to jump out of parallelized loops, such as by using a break statement. This makes it difficult to cancel the rendering of the loop, such as how we do with Parallel::For. Additionally, OpenMP doesn't currently provide good support for task parallelism as is available through the System::Threading::Tasks namespace in Parallel Extensions, or for the plethora of new coordination and synchronization constructs available in System::Threading and System::Threading::Collections. All of these Parallel Extensions constructs should be usable from C++/CLI.

For more details on this application, download and install the June 2008 CTP. The Mandelbrot sample is installed by default at %PROGRAMFILES%\Microsoft Parallel Extensions Jun08 CTP\Samples\MandelbrotFractals. We'd be very interested in hearing about your experiences using Parallel Extensions from C++/CLI, so please send any and all feedback our way, especially if you have suggestions for things we could do to improve the scenario.

Parallel Computing Platform at TechEd 2008

toub — Thu, 29 May 2008 08:16:00 GMT

Heading to TechEd 2008? Come discover some of the exciting technologies the Parallel Computing Platform team is working on.

We're presenting four sessions on parallelism at the Developer conference:

DVP205
The Microsoft Parallel Computing Initiative: Bringing Concurrency to the Masses
Tuesday, June 3 10:30 AM - 11:45 AM, S210 B

Software is headed for a fundamental change. Over the last 30 years, developers have relied on exponential growth in computing power in order to write applications that run fast. However, whereas the average PC clock speed increased more than ten-fold between 1993 and 1999, the average processor speed in the last four years hasn’t even doubled. Instead, the hardware industry is shifting to multi-core and manycore processors. The average PC on the market today is a dual-core. Next year, expect the average to be quad-core. Two years after that, eight-core. Unfortunately, most software today is single-threaded in nature and will not take advantage of multiple processors. The key to high-performance applications in the future is parallelism. In this session, we explore the need for concurrency, as well as what Microsoft is doing through its programming languages, libraries, runtimes, and tools to help developers leverage parallelism in their code.

DVP304
Parallelize Your Microsoft .NET Framework-Based Applications with Parallel Extensions
Wednesday, June 4 10:15 AM - 11:30 AM, S220 A

With the Microsoft .NET Framework today, correctly introducing concurrency into libraries and applications is difficult, time consuming, and error-prone. However, as the hardware industry shifts towards multi-core and manycore processors, the key to high-performance applications is parallelism. Parallel Extensions to the .NET Framework offers solutions to help make writing concurrent applications significantly easier. In this session, we dive into Parallel Extensions, exploring Parallel LINQ-to-Objects (PLINQ), the Task Parallel Library (TPL), the underlying concurrency runtime, and more, in order to provide an in-depth look at the next generation of parallel programming with .NET.

DVP316
Simplify Parallel Application Debugging and Profiling with Concurrency Tools in Microsoft Visual Studio
Thursday, June 5 8:30 AM - 9:45 AM, S330 C

Developing software doesn’t begin and end with writing code. Tools are incredibly important in the life cycle of applications, especially when it comes to debugging and improving performance. This is never more true than when writing parallel applications, as concurrency introduces a whole host of issues that can be very difficult to find and fix. In this talk, we explore difficulties with debugging and profiling concurrent applications, and we dive into powerful new tools being introduced in Visual Studio to improve their development.

TLA327
Parallelize Your Microsoft Visual C++ Applications with the Concurrency Runtime
Friday, June 6 8:30 AM - 9:45 AM, S310 A

Introducing concurrency into native Visual C++ applications has long been the domain of true experts and gurus. Yet, as the hardware industry shifts toward multi-core and manycore processors, all developers will need to be able to write robust and scalable parallel applications. As part of its work on Visual C++ and Visual Studio, the Parallel Computing Platform team is building a key set of technologies that will enable the development of such applications. In this talk, we explore libraries for expressing concurrency, a set of messaging APIs that allow developers to consistently build parallel applications that are robust and resilient, and a shared user-mode runtime for scheduling and for coordinating system resources. Come learn about these exciting new technologies that will help bring concurrency to the masses.

Heading to the IT Professional conference? No problem: we're presenting there, too!

APP253
The Manycore Shift: Bringing Concurrency to the Masses
Tuesday, June 10 3:00 PM - 4:15 PM, N220 A

Recent advances in microelectronic technology and the advent of multi-core and manycore processors are a signal that a large industry change is on the horizon. The computational power of manycore processors, new programming models and platforms, and advanced research in usability promises to change the way people interact with computers. As a result, software is headed for a fundamental change. Over the last 30 years, developers have relied on exponential growth in computing power in order to write applications that run fast. However, whereas the average PC clock speed increased more than 10x between 1993 and 1999, the average processor speed in the last four years hasn’t even doubled. Instead, the hardware industry is shifting to multi-core and manycore processors. The average PC on the market today is a dual-core. Next year, expect the average to be quad-core. Two years after that, eight-core. Unfortunately, most software today will not take advantage of multiple processors. The key to high-performance applications in the future is parallelism, and IT professionals need to be intimately aware of this coming shift so that the applications and scripts they develop can fully utilize all available processing power. In this session, we explore the need for concurrency as well as what Microsoft is doing through its programming languages, libraries, runtimes, and tools to help developers and IT professionals leverage parallelism in their code.

We're looking forward to seeing you there!