High Performance Computing
I've been pretty busy over the past couple of weeks working with some HPC (High Performance Computing) bits and pieces.
Windows Compute Cluster Server (http://www.microsoft.com/hpc) is approaching, and it's been interesting to see the capability and types of environments in which CCS can be used.
CCS uses a job scheduler to conduct operations such as sequential lists of tasks, but also parametric sweep jobs, and parallel jobs using MPI. Options include number of processes to use, maximum processor time and the like. As you can imagine there is command-line access to the scheduler, but it also has an API and a XSD for job submission and scheduler control. This article is a great overview.
In terms of network topology there are a number of options depending on what you want out of the cluster (e.g. Remote Installation Services). Essentially, a cluster works with a Head Node handling all communication outside of the cluster including security and then a number of Compute Nodes forming the rest of the cluster doing all of the hard work. There are useful guides to this information starting here on Technet as well as other guides to user operations and so on. There are plenty of ways to instrument the cluster too.
I think that, most interestingly for me, there is a great opportunity to integrate this HPC stuff into existing infrastructure because the administration is familiar for the IT admins, and the exposed APIs mean that the use of the resources can be exposed usefully to development teams. Because you can use other MPI stacks than MSMPI, then existing parallel programs should move neatly onto CCS too.
There are a couple of places to take a look at if you're interested in exploring CCS. There's a GotDotNet Codegallery here which has information on interacting with the job scheduler and some examples of doing that programmatically.
There is a also a new community site at http://windowshpc.net which will undoubtedly grow over time and is worth a bookmark.
My next question on this... will HPC make Civ IV run any faster..?
