Parallel Programming in Native Code : C++0x

Code Samples for the Concurrency Runtime, Agents Library and Parallel Pattern Library updated for Beta2

rickmolloy — Thu, 12 Nov 2009 14:25:09 GMT

We’ve posted an update to our sample pack at http://code.msdn.com/concrtextras for Visual Studio 2010 Beta2 . The newest thing is this drop of the sample pack are the three new header files in the ConcRTExtras folder. Here’s what these files contain:

ppl_extras.h contains additional stl style parallel algorithms like parallel_accumulate, parallel_partial_sum, parallel_transform, parallel_all_of, parallel_any_of, parallel_none_of and also parallel_for_fixed, parallel_accumulate_fixed, parallel_partial_sum_fixed.

agents_extras.h contains additional useful message blocks like spriority_buffer, bounded_buffer, alternator, join_transform and a recalculate_buffer.

concrt_extras.h contains useful wrapper classes and functions from the runtime itself like concrt_suballocator a std::allocator built on Concurrency::Alloc, task_scheduler, schedule_group and schedule_task which are simple wrapper classes around Scheduler and ScheduleGroup that offer functor support like PPL for scheduling tasks.

These are located in the Concurrency::samples namespace and the team will be blogging about many of these over the coming weeks, but please feel free to ask any questions here or in the forums.

-Rick

Synchronization with the Concurrency Runtime - Part 3

vinodsu — Thu, 04 Jun 2009 23:51:00 GMT

In my previous posts, I addressed the motivation behind using concurrency runtime’s synchronization primitives and also introduced Critical Section and reader writer lock. In this blog, I will cover concurrency runtime’s event.

Event

This is a bi-state type class which, unlike Critical Section or Reader Writer Lock, does not protect access to shared data. Events synchronize flow of execution and use concurrency runtime’s facilities to enable cooperative schedule of work. They behave similar to Win32 manual-reset event. The main difference between the concurrency runtime’s event and Win32 event is that the concurrency runtime’s event are designed to cooperatively yield to other cooperative tasks in the runtime when blocked in addition to preempting whereas Win32 events are, by design purely pre-emptive in nature.

Example:

This example enables the scheduler to create two threads and then calls DemoEvent function that takes event class and a Win32 manual-reset event class as template parameters. The Demo function first creates several tasks that simulate some work and then wait for a shared event to become signaled.

Special thanks to Rick Molloy for sharing this example.

// event.cpp : Defines the entry point for the console application.

// compile with: /EHsc

#include <windows.h>

#include <concrt.h>

#include <concrtrm.h>

#include <ppl.h>

using namespace Concurrency;

using namespace std;

class WindowsEvent

{

HANDLE m_event;

public:

WindowsEvent()

:m_event(CreateEvent(NULL,TRUE,FALSE,TEXT("WindowsEvent")))

{

}

~WindowsEvent()

{

CloseHandle(m_event);

}

void set()

{

SetEvent(m_event);

}

void wait(int count = INFINITE)

{

WaitForSingleObject(m_event,count);

}

};

template<class EventClass>

void DemoEvent()

{

EventClass e;

LONG volatile taskCtr = 0;

//create a taskgroup and schedule multiple copies of the task

task_group tg;

for(int i = 0;i < 8; ++i)

tg.run([&e, &taskCtr]{

//Simulate some work

Sleep(100);

printf_s("\tTask %d waiting for the event\n", InterlockedIncrement(&taskCtr));

e.wait();

});

Sleep(1000);

printf_s(" Setting the event\n");

//Set the event

e.set();

//wait for the tasks

tg.wait();

}

void main ()

{

// Create a scheduler that uses two threads.

CurrentScheduler::Create(SchedulerPolicy(2, MinConcurrency, 2, MaxConcurrency, 2));

printf_s("Cooperative Event\n");

DemoEvent<event>();

printf_s("Windows Event\n");

DemoEvent<WindowsEvent>();

}

Sample Output:

Cooperative Event

Task 1 waiting for the event

Task 2 waiting for the event

Task 3 waiting for the event

Task 4 waiting for the event

Task 5 waiting for the event

Setting the event

Windows Event

Task 1 waiting for the event

Task 2 waiting for the event

Setting the event

Task 3 waiting for the event

Task 4 waiting for the event

Task 5 waiting for the event

We observe that when using cooperative event, we execute all 5 tasks, each task when waiting on the event that is not set, cooperatively yields so that another task can be scheduled and run in the thread’s quantum. When using windows event, we observe that the 2 tasks scheduled block the thread until the event is set.

Auction Simulation written in the Asynchronous Agents Library

rickmolloy — Sat, 30 May 2009 10:11:29 GMT

One of our architects, Niklas Gustafsson has just posted an actor based auction simulation written in the Asynchronous Agents Library. If you follow the blog trail back, you'll see that this was implemented originally in scala and that Matthew Podwysocki has implemented this in F#. Niklas also alluded to, but hasn't shared yet an implementation built in Axum. I find it fascinating to compare and contrast the implementations for similarities and differences.

It’s also worth stressing his concluding remarks here which really highlight why we’re providing an asynchronous message passing and actor based programming model:

As a parting thought, I want to draw your attention to one simple fact: we just wrote an application with 17 agents working together on a problem in parallel, interacting in non-trivial patterns updating state and arriving at a shared understanding of “their world” without a single lock, mutex, or critical section.

The ability to write such complex and concurrent code without dealing with the headaches of managing synchronization primitives is incredibly exciting (at least to me it is), so go check this out.

-Rick

What's new in the Concurrency Runtime and the Parallel Patterns and Asynchronous Agents Libraries

rickmolloy — Thu, 21 May 2009 23:23:00 GMT

Visual Studio 2010 Beta1 has been released, and it is a full install version. The team has been busy, busy busy since the CTP release last fall to deliver most of the APIs and objects we’ve blogged about here into Beta1. So I wanted to take a moment to summarize what’s new in the Beta for the Parallel Pattern Library, the Asynchronous Agents Library and the Concurrency Runtime.

You can find the official reference documentation and walkthroughs in the MSDN Library here.

What’s new in the Parallel Patterns Library

The Parallel Patterns Library (PPL) provides an imperative programming model that promotes scalability and ease-of-use for developing concurrent applications. In general the library’s surface area remains largely unchanged, with a few high impact additions. You will also find it more stable, more robust, faster, and more scalable – lots of goodness!

Added support for exception handling in task_group.
The new is_current_task_group_canceling API helps query cancellation status.
The combinable<T> object has been added for helping avoid races on reductions and accumulations.
We’ve made significant performance improvements to our parallel loops: parallel_for, parallel_for_each and parallel_invoke
critical_section and reader_writer, the cooperative synchronization primitives that we blogged about recently, are now in.
We’ve also included a manual reset event called Concurrency::event.

What’s new in the Asynchronous Agents Library

The Asynchronous Agents Library (or just Agents Library) provides a programming model that enables you to increase the robustness of concurrency-enabled application development. The Agents Library is a C++ template library that promotes an actor-based programming model and in-process message passing for fine-grained dataflow and pipelining tasks. The Agents Library builds upon the scheduling and resource management components of the Concurrency Runtime.

Here’s a summary of what’s we’ve added and revised in the Agents library:

Added the choice, join and multitype_join message blocks to allow users to wait on a set of messages; choice waits for any message, join waits for all messages of a single type, and multitype_join waits for all messages and allows for multiple different types.
Renamed the transform message block to transformer (we didn’t want Concurrency::transformer to conflict with std::transform)
Added functor support to the pipeline message blocks call and transformer
Building custom message blocks is now easier due to some refactoring.

What’s new and improved in the Concurrency Runtime itself

The underlying Concurrency Runtime has undergone significant feature improvements and performance enhancements, many of which we have been discussing already on this blog. These changes will have improvements to everything built on top of the runtime, including the Parallel Patterns Library and the Agents Library.

Added support for User Mode Scheduled Threads on Windows 7 RC (see our blog posts on the Concurrency Runtime’s support and go deep on Win7 UMS)
Revised and simplified scheduler policies and APIs for constructing schedulers.
Added dynamic resource management for load balancing between multiple scheduler instances.
Further improved load balancing by applying the hill climbing technique to thread injection.
Added the concurrent sub-allocator for high performance allocations and frees on multiple threads

Call to Action

Download and install Microsoft Visual Studio 2010 Beta 1 today. Impress your friends and coworkers by being among the first to learn and use our libraries. Provide early and critical feedback on our forum that will shape the way that these libraries are ultimately released.

See this blog post on the value of your feedback: On Achieving Perfection –or– Why We Love Your Feedback (and Why You Can Love Giving It).

Happy coding!

Synchronization with the Concurrency Runtime - Part 2

vinodsu — Thu, 14 May 2009 21:46:00 GMT

In my previous post, I addressed the motivation behind using concurrency runtime’s synchronization primitives and also introduced Critical Section. In this blog, I will cover concurrency runtime’s reader writer lock.

Reader Writer Lock

This class enables multiple threads to read from a shared resource at the same time but only allows one thread to write to it at a time. They share many characteristics with concurrency runtime’s critical section, reader writer locks are non-reentrant and block cooperatively. The reader writer lock resembles the Win32 Slim reader/writer locks (SRWLock). Reader writer lock performs better than critical section in read-mostly environments.

Similarity to Win32 Slim reader/writer locks:

- Can be used only by threads of a single process.

- The reader writer lock can be owned by multiple reader-threads or only one writer thread at any time.

- Non-reentrant.

- Do not support upgrades or downgrades.

Differences with Win32 Slim reader/writer locks:

- The concurrency runtime’s reader writer lock object guarantees that the order of exclusive (writer) lock-ownership is on a first-come, first-serve basis.

- There is no need to explicitly call Initialization of resources before use of the concurrency runtime’s reader writer lock and release after the use of the reader writer lock.

- Cannot specify spin count for the concurrency runtime’s reader writer lock object.

- The concurrency runtime’s reader writer lock enforces cooperative blocking where they yield to other cooperative tasks in the runtime when blocked.

- The concurrency runtime’s reader writer locks give writer preference over readers; i.e. if there are readers and writer(s) simultaneously waiting for the lock, the lock would be handed over to the first writer in queue.

- Exceptions are thrown by the concurrency runtime’s reader writer lock object; on recursive calls, or if unlock is called when the lock is not held, or if a lock is destroyed when being held.

Example:

Given below is a code sample illustrating reader_writer_lock using 4 readers and 1 writer. The readers output the value of the shared data and the writer updates the value of the shared data and outputs it.

// reader_writer_lock.cpp

// compile with: /EHsc

#include <ppl.h>

#include <stdio.h>

#include <windows.h>

using namespace std;

using namespace Concurrency;

//number of iterations each thread performs

static const int NUM_ITERATIONS = 2;

//the shared data that needs protection from race/tearing

static unsigned int sharedData = 0;

//Demonstrates the use of the reader lock

void Reader(reader_writer_lock* pRWLock)

{

for( int i = 0; i < NUM_ITERATIONS; ++i)

{

//use the reader lock

pRWLock->lock_read();

printf_s("Reading %d\n", sharedData);

//Sleep for some time, this is to simulate potential work done while holding the lock

Sleep(100);

//release the lock

pRWLock->unlock();

}

//Demonstrates the use of the writer lock

void Writer(reader_writer_lock* pRWLock)

{

for( int i = 0; i < NUM_ITERATIONS; ++i)

{

//use the writer lock

pRWLock->lock();

printf_s("\tWriting %d\n", ++sharedData);

//Sleep for some time, this is to simulate potential work done while holding the lock

Sleep(100);

//release the lock

pRWLock->unlock();

}

int main()

{

reader_writer_lock rwlock;

//performs reader writer operations in parallel

parallel_invoke(

[&] { Reader(&rwlock); },

[&] { Writer(&rwlock); }

);

return 0;

}

Sample output:

Reading 0

Writing 1

Reading 1

Writing 2

Reading 2

An interesting point to note here is that even though we use 4 readers, we notice that only 3 readers output their value and the lock is taken by the writer. This happens because the lock prefers writers. We do not guarantee the order of task execution but if such a guarantee is required, you could consider using events, which I will cover in the next blog post.

Synchronization with the Concurrency Runtime - Part 1

vinodsu — Wed, 22 Apr 2009 23:21:00 GMT

In a concurrent world multiple entities work together to achieve a common goal. A common way to interact and coordinate is to use shared data. However, shared data must be accessed carefully. This can be achieved through synchronization, primarily using:

i) Blocking methods such as locks and mutexes

ii) Non-blocking methods such as lock-free programming techniques.

I will talk about the synchronization using blocking methods within a process, using constructs provided as part of the concurrency runtime and exposed through the Parallel Pattern Library (PPL). In this blog, I will address the concurrency runtime’s critical section and will cover reader writer lock and events in subsequent blog posts.

For a general picture of the native concurrency runtime, and high level roles of each of its components please refer to this post.

Motivation

Goals of the concurrency runtime’s synchronization primitives:

1. Simple APIs

Unlike their Win32 equivalent, concurrency runtime’s synchronization primitives don’t have C-style initialization and release/destroy type of resource management calls. The exposed interfaces are simple and conform to the C++0x standards and the synchronization objects throw meaningful exceptions on certain illegal operations.

2. Block in a cooperative manner

The synchronization objects are cooperative in nature, in that they yield to other cooperative tasks in the runtime in addition to preempting. For an illustration of this scenario, refer to this post.

Critical Section

This represents a non-reentrant, cooperative mutual exclusion object that uses concurrency runtime’s facilities to enable cooperative scheduling of work when blocked. This class satisfies all Mutex requirements specified in C++0x standards. The concurrency runtime’s critical section provides a C++ façade as compared to its C-styled Win32 equivalent: Windows CRITICAL_SECTION

Similarity to the Win32 CRITICAL_SECTION:

- Can be used only by threads of a single process.

- The critical section object can only be owned by one thread at a time.

Differences with Win32 CRITICAL_SECTION:

- The concurrency runtime’s critical sections are non-recursive. Exceptions are thrown upon recursive calls.

- The concurrency runtime’s critical section object guarantees that threads waiting on a critical section acquire it on a first-come, first-serve basis.

- There is no need to explicitly call Initialization/allocation of resources before use of the concurrency runtime’s critical section and release resources after the use of the critical section.

- Cannot specify spin count for the concurrency runtime’s critical section object.

- The concurrency runtime’s critical section enforces cooperative blocking where they yield to other cooperative tasks in the runtime when blocked.

- Exceptions are thrown by the concurrency runtime’s critical section object; on unlock calls when the lock is not held, or if a lock is destroyed when being held.

Example:

The sample below alternates between printing to standard output from FunctionA and FunctionB.

// critical_section.cpp

// compile with: /EHsc

#include <ppl.h>

#include <stdio.h>

#include <windows.h>

using namespace std;

using namespace Concurrency;

//number of iterations each thread performs

static const int NUM_ITERATIONS = 5;

//Demonstrates use of critical section

void FunctionA(critical_section* pMutex)

{

for( int i = 0; i < NUM_ITERATIONS; ++i)

{

//use exclusive lock

pMutex->lock();

printf_s("A %d\n", i);

//Sleep for some time, this is to simulate potential work done while holding the lock

Sleep(100);

//release exclusive lock

pMutex->unlock();

}

//Demonstrates use of critical section

void FunctionB(critical_section* pMutex)

{

for( int i = 0; i < NUM_ITERATIONS; ++i)

{

//use exclusive lock

pMutex->lock();

printf_s("\tB %d\n", i);

//Sleep for some time, this is to simulate potential work done while holding the lock

Sleep(100);

//release exclusive lock

pMutex->unlock();

}

int main()

{

critical_section mutex;

//call FunctionA and FunctionB in parallel

parallel_invoke(

[&] { FunctionA(&mutex); },

[&] { FunctionB(&mutex); }

);

return 0;

}

Sample output:

Note: There is a possibility that the order may be swapped, where B gets the lock before A; the locks are handed out on a first-come, first-serve basis, it’s a race to try and get the lock at the beginning. One way of guaranteeing consistency is using events.

On Channel9: a chat with members from the Visual C++ and Concurrency Runtime teams

rickmolloy — Wed, 14 Jan 2009 04:36:00 GMT

Last week, Don McCrady, Damien Watkins from the C++ team and I sat down with Charles from Channel9 and chatted about the Concurrency Runtime, new C++ work in Visual Studio 2010 and how our teams are working together.

As Charles pointed out, we also touched briefly upon new features in the C++0x standard that haven’t been discussed before.

Here's the link:

http://channel9.msdn.com/shows/Going+Deep/Parallel-Computing-in-Native-Code-New-Trends-and-Old-Friends/

Enjoy and as always your feedback is very welcome,

Rick