Research and technology are constantly evolving. Dryad and DryadLINQ are not immune. We know this has caused some confusion to users of our external release. For example, the Dryad paper presented at EuroSys’07 reports results obtained on compute nodes that were fitted with Windows Server 2003. This, however, does not mean that our current external release works in this environment. As Michael has explained in a previous post, the Dryad execution engine can run on multiple cluster-management systems. Our publically available research software requires Windows HPC Server 2008 SP1, which most people in academia can get for free (up to a number of nodes, additional nodes are steeply discounted) via the MSDN Academic Alliance.
Another source of confusion has come from changes in the DryadLINQ APIs. “Some sample programs written in DryadLINQ” is a collection of programs that illustrate the use of the DryadLINQ parallel computation framework. For each program, the guide provides the important source code and explains the key ideas. The guide was first published in May 2008 when DryadLINQ used DryadDataContext and DryadTable objects to represent distributed collections of strongly-typed .NET objects. The code to load a partitioned table and to copy it looked like this:
DryadDataContext ddc = new DryadDataContext(“file://\\machine\directory”);
DryadTable<LineRecord> table = ddc.GetPartitionedTable<LineRecord>(“input.pt”);
The above code no longer compiles. All DryadLINQ tables are now PartitionedTable objects. The new API uses a single URI scheme to identify the type of the data, its location and additional attributes. So the above code is now written as:
string uri = “file://\\machine\directory\input.pt”;
PartitionedTable<LineRecord> table = PartitionedTable.Get<LineRecord>(uri);
And if you wanted to specify the compression scheme for the table being written out:
To reflect those changes, we have published an updated version of “Some sample programs written in DryadLINQ”. The updated document is downloadable from the DryadLINQ project page at MSR or by following this link. We hope you will find this useful as you learn about DryadLINQ.