MSMPI is Microsoft’s implementation of MPI (Message passing interface) library. I’ve explained it a few times before. MSMPI Explained, MSMPI SDK patch.
Many people have asked me for examples that they can understand and relate to how MPI is used on Unix. Here’s the information you’ve been looking for!
Compilers supported: MingW’s GCC, microsoft CL, PGI, Intel…. but not GCC under cygwin or SUA.
Caution: For 32 bit, Microsoft MPI library uses __STDCALL, which means one must use MPIAPI macro in function declaration. Here’s an example:
PetscMPIInt PETSC_DLLEXPORT MPIAPI Petsc_DelTag
(MPI_Comm comm,PetscMPIInt keyval,void* attr_val,void* extra_state);
You really just need to make sure mpi.h can be found in the include path, and msmpi.lib can be found in the Library path. Here are a few examples:
Visual Studio:
Include dir in “Additional Include directories” in your project property.
If you are building a 32 bit application, make sure i386 path is used.
64 bit library path.
Finally, make sure msmpi.lib is part of the linker input.
MINGW native windows GCC compiler:
g++ -O3 -g -c Main.cpp -I/c/Program\ Files/Microsoft\ HPC\ Pack\ 2008\ SDK/Include/
g++ -O3 -g Main.o /c/Program\ Files/Microsoft\ HPC\ Pack\ 2008\ SDK/Lib/i386/msmpi.lib -o mympi.exe
Intel compiler under cygwin Unix environment:
Using include path example:
-I/cygdrive/c/Users/wenmingy/petsc-intel/petsc-2.3.3-p15 -I/cygdrive/c/Users/wenmingy/petsc-intel/petsc-2.3.3-p15/bmake/win32_intel -I/cygdrive/c/Users/wenmingy/petsc-intel/petsc-2.3.3-p15/include -I /cygdrive/c/Program\ Files/Microsoft\ HPC\ Pack\ 2008\ SDK/Include
Library path example:
-L/cygdrive/c/Users/wenmingy/petsc-intel/petsc-2.3.3-p15/lib/win32_intel -L/cygdrive/c/Users/wenmingy/petsc-intel/petsc-2.3.3-p15/lib/win32_intel -lpetscts -lpetscsnes -lpetscksp -lpetscdm -lpetscmat -lpetscvec -lpetsc /cygdrive/c/Program\ Files/Microsoft\ HPC\ Pack\ 2008\ SDK/Lib/amd64/msmpi.lib -L/cygdrive/c/Users/wenmingy/petsc-intel/petsc-2.3.3-p15/externalpackages/f2cblaslapack/win32_intel -L/cygdrive/c/Users/wenmingy/petsc-intel/petsc-2.3.3-p15/externalpackages/f2cblaslapack/win32_intel -lf2clapack -L/cygdrive/c/Users/wenmingy/petsc-intel/petsc-2.3.3-p15/externalpackages/f2cblaslapack/win32_intel -L/cygdrive/c/Users/wenmingy/petsc-intel/petsc-2.3.3-p15/externalpackages/f2cblaslapack/win32_intel -lf2cblas Gdi32.lib User32.lib Advapi32.lib Kernel32.lib Ws2_32.lib
resourcekit.windowshpc.net
Finally version 2 is updated and online.
Changes include:
New Parallel computing content section in developers.
Updated many older documents, presentations.
New hpc webcasts from partners.
New marketing white papers.
We will be distributing limited quantities at techED. If you would like to get one of these, please find us at the Windows HPC booth.
The 10th LCI (Linux Cluster Institute) international Conference on High Performance Clustered Computing was hosted at National Center for Atmospheric Research (NCAR) in Boulder, Colorado on 3/9 - 3/12. This year's focus was on large scale system design that address balance between inter-and intra-node parallelism. LCI is a small conference that had roughly 150 attendees. Microsoft was invited not only as a vendor sponsor but also to give a tutorial on Windows HPC Server 2008.
Here's a list of abstracts, technical papers, and program schedule. Microsoft is relatively new to the field of HPC. We were there to listen to HPC users, educators, and developers, and update them about what we are doing to make their lives easier with latest tools, and best practices from the software industry.
Developers are starting to see more user demand for their HPC applications on the Windows HPC Server platform. Like any software projects, due to budget and time constraint, the platform that the developer is not developing on may not get a build. After discussing with several HPC developers at the conference, I have a few ideas as a result, to share with everyone in terms of sustainable cross platform development specific to many HPC apps.
First, there is no silver bullet in getting your HPC code running on the Windows platform, but it is generally not difficult, since most HPC apps are console applications. There are also numerous Unix environments on the Windows platform including SUA, Cygwin, MingW/MSYS. These environments allow you to build your applications without changing your build system too much. The free Windows HPC SDK, enables you to build your HPC application against our MSMPI library, which is industry standard.
Some of the more complex platform issues include dealing with Unix System calls, Pthreads, on Windows. Developers don't necessarily know how to code to specific OS for some of these calls. There are articles, guides and books on them. Of course, for the hyper-impatient developers, I could always offer the advice of hiring a professional Windows programmer to resolve these issues.
As a software engineering practices, it is always a great idea to develop your Linux HPC software with portability in mind. In many cases, by going to a new platform, your code is exercised more, thus revealing some of the bugs not discovered using a single platform. One thing that is a big plus on the Windows platform is that Microsoft has invested billons in Visual studio to help developer write faster, better code. Today's complex software projects demand an IDE for productivity, why not take advantage of the best tool in the industry.
During the discussions with developers, we have concluded that the best way to keep a quality build of your HPC project on Windows is to have at least one of project developer using Visual studio to write the code, that way there is no more last minute scrambling to get a Windows binary built. If the project can spare more cycles, then an automated regression test that checks out and builds the project in a regular basis would also make the build on Windows platform better. This, however may require expertise in maintaining the original Unix build system with the windows platform in mind. The PETSC project does a great job of making things transparent; their python build system runs great under cygwin, thus no additional automated build system is needed on Windows.
References and tools on the Windows platform:
HPC Resource kit. HPC white papers, presentations, tutorials.
Sys Internal Tools. List of tools Utilities.
Windows SDK with free command line compilers, libraries.
When it comes to HPC development, linear algebra libraries are the essential building blocks. You often hear about BLAS/LAPACK, so what exactly are they and how do they relate to the top 500 run benchmark called LINPACK?
BLAS stands for basic linear algebra subprograms, it contains routines that does basic linear algebra functions like vectors and matrix multiplications. It has existed for nearly 30 years. LAPACK, you guessed it, is an abbreviation for linear algebra package. It is a larger set that include BLAS. Heavily utilized in high performance computing, BLAS/LAPACK have been implemented/optimized, re-implemented many times over for many platforms. LINPACK, though the official benchmark for HPC, relies on a specific subroutine DGEMM in BLAS to solve dense matrix in order to obtain measurement of FLOPS (floating point operations per second). LINKPACK, however, has little practical use, as it has been replaced by LAPACK. There are many commercial and non-commercial versions of BLAS/LAPACK. I’ll mention a few that works on windows.
refblas: The official reference implementation from netlib. C and Fortran implantations.
Intel MKL: The Intel Math Kernel Library.
ACML: The AMD Core Math Library.
ATLAS: Automatically Tuned Linear Algebra Software. (uses cygwin as build env)
CUDA SDK: The NVIDIA CUDA SDK includes BLAS functionality for writing C programs that runs on GeForce 8 Series graphics cards.
Goto BLAS: Kazushige
Goto's implementation of BLAS.
uBLAS: A generic C++ template class library providing BLAS functionality. Part of the Boost library. Note that, unlike other implementations, uBLAS focuses on correctness of the algorithms using advanced C++ features, rather than high performance.
GSL: The GNU Scientific Library Contains a multi-platform implementation in C. windows.
BLAS/LAPACk is the very basics of HPC numerical libraries. Many solver libraries such as PETSC and TRILINOS build on top of it. Here are three links back to my previous blogs that explain, Linear, non-linear, and Time-stepping problems.
Math under the hood (linear solvers)
5 min Intro to nonlinear solvers
Explicit and Implicit (time stepping, or PDE solvers)
Overview
This update fixes an issue where the HPC Pack 2008 SDK was attempting to use a function that was not available on olders versions of Windows, such as 32-bit Windows XP. If you encounter the error "The procedure entry point GetProcessIdOfThread could not be located in the dynamic link library KERNEL32.dll" when attempting to run an application that you've built using the HPC Pack 2008 SDK then you should apply this fix and recompile your application.
http://www.microsoft.com/downloads/details.aspx?FamilyID=6415c650-78a3-4b6a-8264-b855173b14d8&DisplayLang=en
To start the download, click the Download button and then:
1) Click Save to copy the download to your computer. 2) Run the download on the machine that you build your applications using the HPC Pack 2008 SDK. 3) Recompile your application with the updated SDK.
High Performance Computing (HPC) Clusters: Reduce Complexity with IBM and Microsoft
Please join us for an informative and exciting webcast with IBM and Microsoft. No matter where you are located in the world, IBM has an HPC cluster solution that is easy to deploy. Learn how IBM can help you reduce the risk and manage growth more easily with the pre-tested, easy-to-deply, easy-to-manage IBM Cluster 1350 solution, and how when combined with Windows HPC Server 2008 you can leverage your current Windows server expertise to accelerate your time to insight on computational analysis. Whether you are a financial analyst or engineer our Windows based HPC cluster will meet the demand of your computational needs. Register today:
Windows HPC Server 2008 environment with System Center Operations Manager 2007: A Dynamic Integration For HPC Fans: Discover how to integrate your Windows HPC Server 2008 environment with System Center Ops Mgr 2007 (SC0M 2007), to leverage a new way of dynamic management and reporting. You'll learn what can be done by using the regular Compute Cluster Server Management Pack to monitor a cluster and how to leverage the Windows 2003 Management Pack and Job Scheduler to understand why a job failed. Learn how to build custom reports with SCOM 2007 and Windows HPC Server 2008 and see how the integration works. Finally, get a sneak peek into what will be done in the Windows HPC Server 2008 Management Pack. Join us today for this informative and highly productive webcast!
Windows HPC Server 2008 - New and Advanced Features
Look-- no slides just kick the tires! Want to find out what's new in Windows HPC Server 2008 and what exactly is "High Productivity Computing"? Join us for an exciting look into Windows HPC Server 2008. We'll take you on a tour of this new version of Windows Compute Cluster Server, highlighting the major advancements in deployment, management, interoperability and scheduling. We promise no slides and tons of live demonstrations plus a lively discussion on how Windows HPC Server 20008 will increase your cluster administrators' productivity.
Scaling up Analysis using Windows HPC Server 2008 presented by Ansys
Join Microsoft and Ansys for an informative HPC webcast. Moving from desktop computing to a cluster can expand the scope of simulation, allowing more simulation studies and reducing the turnaround time for larger models. This presentation will review the performance of simulation software from ANSYS on Windows HPC Server 2008 and highlight customer experiences with cluster deployment based on Windows.
Leveraging Computational Features of Windows HPC Server 2008 using the IMSL® Numerical Libraries with Visual Numerics & Microsoft
Windows HPC Server 2008 is a key component in supporting the development of high performance computational applications within a rich development environment. New capabilities support the development of high performance computing (HPC) applications in a variety of languages such as C/C++, Fortran, and .NET for both multi-threaded and multi-node computing. This presentation will focus on the computation algorithm support for developing HPC applications using the IMSL Numerical Libraries with Windows HPC Server 2008. An example for leveraging a multi-tiered computing environment will be included.
It's the last day of 2008, and I am uploading some of the web casts that we have done for HPC in the last year.
All available at: http://channel9.msdn.com/shows/The+HPC+Show/
(may take a day to fully show up, the the links below should work)
HPC SERVER 2008 - 2008 PARTNER AND MS WEBCASTS:
Bringing speed and visibility to software builds using Windows high-performance clustering with Electric Cloud and Microsoft
Improving the speed of the software build/test/deploy cycle is one of the most effective ways of increasing developer productivity. In order to break the build bottleneck, all limitations to performance must be overcome: dependencies, accuracy, and infrastructure speed. ElectricAccelerator solves the problem of hidden and implicit dependencies, guaranteeing accuracy while enabling parallel execution, and Microsoft Windows HPC Server 2008 provides the most effective cluster infrastructure to support the ElectricAccelerator technology.
Dramatic Acceleration of Excel-based Trading Simulations with Platform and Microsoft
Tired of waiting hours for simulation results? In this solution overview, Platform will review the challenges of running Excel models in a distributed environment, and describe the benefits and details of how to deploy models without wholesale code/macro changes. Learn how to enhance application performance and empower quants and developers to deploy distributed applications quickly without modification, using Platform Symphony's Excel Connector with Windows HPC Server 2008 and Microsoft® Office 2007 Excel. You'll discover how the world's top financial firms can realize new revenue opportunities ahead of the competition by leveraging this joint solution.
High Performance Computing for the Masses with Intel
High Performance Computing has its roots in non-commercial, academic, national lab and governmental environments where very large, compute-intensive workloads and applications are executed. According to recent data from IDC HPC, economically-attractive clusters have become the common denominator across most HPC deployments. In parallel, the continuing maturation of HPC ISV commercial business software, the refinement of operating system & cluster software, and the consistent availability of more computing power at lower cost have opened up the inherent performance advantages of HPC clusters to a much wider community of users. Please join Intel for an overview of the current HPC marketplace and the potential alternatives of utilizing High Performance Computing clusters for both increased productivity and competitive advantage.
HPC in Life Insurance Modeling - Leveraging Milliman's MG-ALFA on the Cray CX1 Supercomputer with Windows HPC Server 2008 to Meet Emerging Insurance Requirements and Deliver Advanced Analytics
MG-ALFA from Milliman is available on the new Cray CX1 deskside supercomputer and is fully integrated with the Windows HPC Server 2008. Join us for this informative webcast and learn about the new Cray CX1 system, get the latest information on the recent developments in version 6.7 of MG-ALFA and explain the benefits of MG-ALFA on the latest version of Windows HPC technology, Windows HPC server 2008. Topics covered will include: Introduction to the recently-introduced Cray CX1 supercomputer; Performance of MG-ALFA on Windows HPCS, Scaling performance for very large system counts running MG-ALFA on Windows HPCS, Technical requirements for the implementation of MG-ALFA on Windows HPCS, Enhancements to Windows HPCS over the prior version of this software, Windows Compute Cluster Server. The Webinar is of interest to anyone evaluating an implementation of the Principles based approach to Capital.
Windows HPC Server Development, the MPI Application Model
Windows HPC Server 2008 introduces innovative features for traditional High Performance Computing solutions. Significant improvements have been implemented across the entire HPC stack. Learn about the new NetworkDirect RDMA technology and how it enables very fast, high-bandwidth, distributed shared memory computing. MS MPI is now integrated with Event Tracing for Windows easing the task of profiling, analysis, and visualization of HPC solution performance. A new Job Scheduler architecture delivers impressive performance while also enabling intelligent scheduling policies and resource management. Traditional HPC solution developers will appreciate the elegance of Microsoft's latest platform technologies while those new to the parallel computing problem domain will want to consider Windows HPC based solutions
Improving Electronic System Design Productivity using Synopsys System Studio and Saber on Windows HPC Server 2008
Synopsys System Studio and Saber products are now available on Windows HPC Server 2008. Join us for this informative webcast and get the latest information on how System Studio further improves performance and signal processing design productivity by taking full advantage of Windows HPC Server 2008 as a stable, extensible environment on high-performance CPUs. Learn how Saber helps automotive and aerospace supply chains meet stringent reliability and safety requirements for mechatronic systems in harsh environments. System Studio is Synopsys' model-based algorithm design and analysis offering providing a unique dataflow simulation engine with the highest performance for exploring, verifying and optimizing digital signal processing algorithms. Saber is Synopsys' technology-leading mechatronic design and analysis software, advancing Robust Design and Design for Six Sigma (DFSS) methodologies into today's automotive, aerospace, and commercial design.
The Development of a Next Generation Networking Interface - Network Direct
Mellanox and Microsoft bring you an informative and informational webcast on Network Direct. The ability of scientists to access the right data and apply it the right way is critical for many scientific disciplines. "The life sciences with genomic and proteonomic data, it's an explosion of information that people around the world should be able to benefit from the results that other people are gathering together," Bill Gates, Supercomputing 2005. Three years after, Microsoft has introduced a new networking interface called Network Direct, which enables high performance applications not only to be able to execute faster and handle large information databases, but also to provide efficient parallelism with effective remote data management. In the future, Network Direct might be the common interface not only for MPI based application but also for sockets and file systems. The session will provide an overview of Network Direct development and its potential benefits, a discussion of the role of Windows for MPI applications, and industry case studies.
HPC in Oil and Gas: Reservoir Simulation using Roxar's Tempest on Windows Compute Cluster Server
Join us for an exciting webcast specific for the Oil and Gas Industry! Predicting the performance of an oil reservoir requires a large amount of computing muscle. This is now available at a reasonable cost using cluster computers with typically 8 - 16 processors. Roxar's Tempest simulator can take full advantage of these clusters using Windows Compute Cluster Server to simplify job submission and administration. This webcast will show you how. Presented by Robert Frost, Tempest Development Manager and Martin Parker Systems Integration Manager, Roxar Corporation.
Taking the Guesswork out of High-performance Computing for Financial Services
Learn how insurers can use Towers Perrin's MoSesT V6.2 software with Microsoft Windows Compute Cluster Server 2003 to run sophisticated Financial models in less time. Changes in the financial services industry are requiring firms to manage risks and capital better. To meet this challenge, insurers need robust financial modeling software integrated with a cost-effective high-performance computing platform, which can extend existing IT investments. Find out how MoSes and WCCS can help save time and increase productivity.
Windows HPC Server Development, the WCF Application Model
Windows HPC Server 2008 introduces a new interactive programming model for High Performance Computing solutions. Learn how to host Windows Communication Foundation (WCF) components within a compute cluster and how to leverage the cluster in various service-oriented client application scenarios. An innovative departure from traditional clustered computing, the HPC+WCF Services model enables ease of solution development, deployment, and management. Early adopters include financial services enterprises with near real-time interactive compute requirements.
I am at Microsoft PDC this week, demonstrating the Microsoft HPCS Server 2008. Please come by the Parallel computing booth and talk to us.
I will be distributing the HPC technical resource kit on a thumb drive at the booth and for everyone that attends the HPC talk on Tuesday right after Lunch.
Yes, it's ready! I have compiled a 70 item kit full of white papers, presentations, videos, and tutorial samples. Took me nearly 3 months to collect, summarize, filter and distill all the materials into THE most comprehensive windows HPC resource kit that you'll find anywhere!
I have thumb drives available upon request. You can also access the materials online at:
http://resourcekit.windowshpc.net/Default.html
I am taking requests for thumb drives, and feedbacks are welcome.
Many people are still wondering exactly what Windows HPC Server 2008 is. On the launch day of the Windows HPC Server 2008. I promise this is the only blog you'll need to read if you are new to the product.
In one line:
MSMPI + Scheduler + WCF broker + Windows productivity story = HPC++ (Windows HPCS 2008)
In a few short paragraphs:
High performance computing goes back to the CRAY days. Mostly used for R&D, to solve large [time (cpu) /space(memory)] complexity problems such as simulating a physical system. These problems include nano-simulation, medical, surface flow, computational fluid dynamics, fusion, mechanical systems, and more. Some of the well known commercial applications include ANSYS’ FLUENT, Schlumberger’s ECLIPSE, Simulia’s ABAQUS. All of these tools use MPI or message passing library, it’s an industrial standard that’s available on Unix, Linux, Cray, and now Windows. MS MPI stack is based on Argonne National Lab’s Mpich2 implementation, with the additional benefits of Network Direct, or fast direct hardware access bypassing kernel stack. MPI requires fast hardware interconnect such as infini-band network cards to achieve 20GB/s in bandwidth, and 2 microsecond in latency. By implementing network direct, we can now compete head to head with Linux/UNIX products. Our product team did a top 500 run lately, and achieved #23 on the world’s top 500 fastest computer list.
Super computers running MPI based programs are often batch oriented, you and 100s of other users submit a job based on resources you need such as disk/memory, number of nodes, and even software licenses. Windows HPC Server 2008's job scheduler and other IT PRO features efficiently scale to thousands of processing cores and provides a comprehensive set of deployment, administration, and monitoring tools that are easy to deploy, manage, and integrate with your existing infrastructure.
V2 or HPCS 2008 has one additional feature called WCF broker. It is essentially a proxy system for a farm of on-demand web services. This programming model is useful for short jobs that are embarrassingly parallel. The nice thing about web service is that things are loaded in memory, so the roundtrip of start to finish of a calculation is fast. WCF model does the growth/shrink management on the number of instances of web services you run on your cluster. Some examples including farming out mortgage asset calculations on your spreadsheet to the web services farm.
My blogs ends with the note that CRAY just released its CX1 mini supercomputer with Windows HPC Server 2008. With the advanced programming productivity suite, no computational scientist should be left behind.
PETSc, stands for portable, extensible toolkit for scientific computation, it is a suite of data structures and routines for the scalable (parallel) solution of scientific applications modeled by partial differential equations. It employs the MPI standard for all message-passing communication. There are hundreds of applications that uses petsc in the area of nano-simulation, medical, fusion, Geo sciences, surface flow, CFD, optimization, and much more.
As one of the most important scientific libraries that run on top of MPI, I decided to try it out for our windows HPC cluster. It turned out that it was quite easy to get running using visual c++ 2008, Microsoft windows MPI stack, and cygwin as a build environment.
I took some simple notes to help anyone that needs to get it running on windows:
1. Install the following: petsc, windows HPCS 2008 SDK, Microsoft Visual C++ 2008, and Cygwin with python.
2. run visual studio command prompt for 32 bit build, and visual studio command prompt win64 for 64bit builds...
This ensures that you have CL (the ms c++ compiler in your path). Then, run c:\cygwin\cygwin.bat to get into the cygwin environment.
3. unpack petsc: gzip -cd petsc*.tar.gz |tar xf -
4. cd into the petsc directory, then Configure petsc, but first run: export PETSC_DIR=`pwd` # THIS SETS UP bash environment for you.
5. Run the command below (one line), this is a 32 bit build. You will have to change the path if you are building 64bit to x64 instead of i386.
config/configure.py --with-cc="win32fe cl" --with-fc=0 --download-c-blas-lapack=1
--with-mpi-include="/cygdrive/c/Program Files/Microsoft HPC Pack 2008 SDK/Include"
--with-mpi-lib="/cygdrive/c/Program Files/Microsoft HPC Pack 2008 SDK/Lib/i386/msmpi.lib"
--useThreads=0 --with-shared=0
6.(only 32bit version).
edit python/BuildSystem/config/packages/MPI.py and comment out the line
> > self.functions = ['MPI_Init', 'MPI_Comm_create'
You'll also need to patch up the code, because 32 bit MSMPI stack uses STDCALL convention, thus it needs a little patch from petsc folks. to add MSMPI. 64 bit always uses fastcall convention, thus this step is not needed.
The patch file basically puts a MPIAPI macro in front of all their calls. This patch is either available upon request or by emailing petsc support and refer to [PETSC #17869].
7. Once this is all configured, type make all.
This will take a few hours...... it's a BIG library to compile.
8. Let's build example 23....in dir: ~/petsc-2.3.3-p13/src/ksp/ksp/examples/tutorials
type make ex23
$ make ex23
output:
/cygdrive/c/Users/wenmingy/petsc-2.3.3-p13/bin/win32fe/win32fe cl -o ex23.o -c -
wd4996 -MT -Z7 -I/cygdrive/c/Users/wenmingy/petsc-2.3.3-p13/src/dm/mesh/sieve -I
/cygdrive/c/Users/wenmingy/petsc-2.3.3-p13 -I/cygdrive/c/Users/wenmingy/petsc-2.
3.3-p13/bmake/cygwin-c-debug -I/cygdrive/c/Users/wenmingy/petsc-2.3.3-p13/includ
e -I/cygdrive/c/Program Files/Microsoft HPC Pack 2008 SDK/Include -D__SDIR__="sr
c/ksp/ksp/examples/tutorials/" ex23.c
Petsc shows you all the link options...
9. Now, let's run it:
$ mpiexec -n 4 ex23 // running 4 processors locally
KSP Object:
type: gmres
GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
GMRES: happy breakdown tolerance 1e-030
maximum iterations=10000, initial guess is zero
tolerances: relative=1e-007, absolute=1e-050, divergence=10000
left preconditioning
PC Object:
type: jacobi
linear system matrix = precond matrix:
Matrix Object:
type=mpiaij, rows=100, cols=100
total: nonzeros=298, allocated nonzeros=700
not using I-node (on process 0) routines
Norm of error 0.000101234, Iterations 502
10. That's it, you just solved it. and you can submit this job to the compute cluster job scheduler, but I am on my laptop, that would be another blog.
As you can see, with some minor tweak, PETSc plays nicely with MSMPI stack.
I would also like to thank Serguei O. for his help!
One of the most frequent questions I get when I talk to customers is, what is my best option for porting my Linux HPC code to windows... As a cross platform developer for the last few years, I had to make sure codes worked on mac/various linux/windows/unix platforms, and it can be a major headache at times. Fortunately, HPC codes are usually not that difficult to port, as they are often console only (has no gui), and they often use MPI, which is an industrial standard.
Porting from Unix to Windows often requires a little bit of research first. There are many options depending on the situation. I would like to introduce several options and state my personal experiences with these options.
1. Cygwin:
Cygwin is a Linux-like environment for Windows. It consists of two parts:
A DLL (cygwin1.dll) which acts as a Linux API emulation layer providing substantial Linux API functionality. A collection of tools which provide Linux look and feel. Pros: Easy port, you can build your Linux apps using the configure script and get it to run with almost no changes to the code.
Cons: Requires the cygwin RUN TIME, and the licensing terms are very unforgiving (GPL). Plus it is only 32 bit! Remember, you are NOT producing win32 code here, everything must run through the cygwin.dll layer.
I personally would only use it as a Unix environment, but not distribute binaries with Cygwin due to the above restrictions.
2. MingW and MSYS:
Mingw and MSYS: A Minimal system providing a POSIX compatible Bourne shell environment, with a small collection of UNIX command line tools.
Pros: POSIX calls, produces win32 code, LGPL licensing, outstanding shell + terminal window.
Cons: 64 bit is in alpha stage, porting usually means more work, since a lot of the linux apis are not posix( it really is minimal).
I originally ported a lot of code using. The nice thing is that you can use mingw's gcc to link in an entire application into one windows static binary exe for distribution. I am not the only one that highly recommend this solution for building windows applications, the folks at trolltech also made it the default compiler environment for their open source edition of QT 4.x. QT is a cross platform GUI system, that has both open source edition, and a supported commercial version. My poor man's QT recommendation is Wx Windows, which will work for a lot of people. The Python Scientific Package I mentioned a couple of months ago uses WxWindows for their TVTK GUI system. I also liked their terminal window, it was a replacement for windows cmd window until powershell came along.
3. Native Compilers
Intel, Microsoft, PGI, and other commercial companies sell commercial c/c++, fortran compilers on the windows platform. One of the most popular solution is the Intel compiler suite, since it contains both fortran and c++ compilers. Almost all of these compilers integrate directly into Microsoft visual studio IDE, which makes life so much easier for developers.
Pros: Commercial support, IDE integration, better debugging, performance analysis tools, 64 bit support!!
Cons: Requires porting to the win32 API for one, have to establish additional build systems with the IDE based solutions, and those ifdefs for a new compiler!
The good news is that porting HPC apps usually require less effort, as they are usually commandline apps. Having a different compiler to exercise the source code often exposes flaws you normally don't see using only GCC. Plus that it is essential to have an IDE to deal with the ever increasing code base of modern scientific code. It is becoming a trend that a lot of developers prefer Visual studio for productivity.
Conclusion:
Choosing how to port your application depends on your specific needs. I have presented 3 different ways to port your HPC application, cygwin, mingw, and a native build environment. I highly recommend the native port approach, as it saves time and effort in the longer run. I will discuss common porting issues in the next few blogs.
Once again, thanks for reading my blog, and I would appreciate any feed backs or questions you may have.
I uploaded two more videos on Technet/Edge. They are worth watching if you are interesting in an overview on the Windows HPC Admin Console. The powerful integrated solution enables system administrators to do their common tasks all under one GUI.
Monitoring and managing a large scale cluster often requires advanced tooling. System Administrators demand tools that help them to manage heterogeneous compute nodes, check cluster status at a glance, identify deviance, correlate node and job information, track changes, and the ability to integrate with existing IT infrastructure. Windows HPC Server 2008 admin console addresses all of the above problems with an integrated solution.
The admin Console includes the following five main areas, charts and reports, configuration, node management, job management and diagnostics. In addition, the console has a “pivoting” feature that allows the system administrator to navigate to different views by keeping the same context. Our Program manager Rae Wang, will go through each of the five areas with demonstrations and simple scenarios in the first video, Monitoring and Management.
For large computing clusters, diagnostics is where system administrators spend a lot of their time. Common tasks include:
- Validate cluster post deployment or configuration change.
- Troubleshoot failures.
- Measure performance degradation over time.
Windows HPC Server 2008 has 16 built-in diagnostics to help Sysadmins do diagnostics with ease. These diagnostic tests can be classified into the following categories, infrastructure, configuration report, and performance. Infrastructure tests include scheduler, system services, connectivity, and Service Oriented Architecture or the WCF broker model. While configuration report has application, network, software updates and system service tests available. Finally, we have two MPIPingPong tests that measure the cluster performance in terms of latency and bandwidth.
The diagnostic tests are flexible and easy to run, and the results are filterable and searchable. System administrators can utilize the test results to further diagnose using built-in tools like clusrun, remote desktop, and node template features.
The Diagnostics Video.
HPC Basic Profile Web Service or the HPCBP for short, is a web service, built using the Windows Communication Foundation (WCF) that provides access to some of HPCS 2008’s core job submission functionality.
Over the last few years, using an open process within the Open Grid Forum (OGF), developers from industry and research from both the open source and commercial software communities, have come to agreement on the web service interface and protocols that can provide greatest interoperability. These set of specifications are encapsulated within the HPC Basic Profile 1.0.
In this 5 parts video, Steven Newhouse, our HPC BP program manager gives a lecture on the details of HPC Basic Profile, the Microsoft SDK, examples of how to submit a job using java client, pre-staging files, and starting a SOA session.
http://channel9.msdn.com/shows/The+HPC+Show/
Here are some great links on grid computing: (thanks, Dave for the additional pointers)
Good source for broad information:
For Science: International Science Grid This Week (online mag)
http://www.isgtw.org/
In General: http://en.wikipedia.org/wiki/Grid_computing
Two major science grid efforts in the US:
The Open Science Grid http://www.opensciencegrid.org
The TeraGrid http://www.teragrid.org/
Other grid related projects:
The major European grid effort related to the LHC high-energy physics
experiment, called LCG, http://lcg.web.cern.ch/LCG/
A European grid project called NorduGrid http://www.nordugrid.org/
The UK e-science program http://en.wikipedia.org/wiki/E-Science
Sun's grid effort: http://www.sun.com/service/sungrid/index.jsp
