I’m James Reinders and I thought we’d talk a little bit about an interface called “message passing interface” which is an API for doing parallelism. It’s usually associated with super computers so as we look at multi-core processors you may or may not be familiar with message passing interface but I think it’s something worth taking a look at and understanding. You may or may not use it but it’s good know that it’s available and what it is.

If you take a look at a multi-core system these days, say it’s a quad-core or a couple of dual-core processors, one thing that you do when you’re programming it is you can assume that every thread of execution, every processor has equal access to memory. So, processor 1 can put something in memory and processor four, or thread four can be looking at the same memory. So, it’s a very easy model – a shared memory model.

In this particular example, we can have a program running on four processors, all reading and writing memory and that’s the form of communication between them. Let’s suppose though that we either build a machine differently or we pretend that it’s different and we limit each processor to only working with data that it exclusively controls and that the only communication between the processors is via message passing. In general, a processor can communicate with any other processor, or any combination. Processor two can pass a message to processor four and so forth.

So, you can have a thread running on each one of these. It’s like having people in rooms next to each other than can’t really communicate like they would if they were in one room with each other sitting across the table talking but instead they have to write down what they want to communicate to each other and slip it back and forth under the doors to each other. It limits the amount of communication that’s going on but that can be a big advantage. It forces a discipline of decomposing a problem so that the problem runs really well on each processor using mostly the local data and occasionally sharing information between them. Programs written using this method generally scale better. It’s really not MPI (message passing interface) that caused them to scale better; it’s the discipline to write a program so it’s more self-contained – processor by processor with local memory utilisation. It actually doesn’t matter if the machine is a shared-memory processor, like a quad-core would be in a modern machine, but just simply using your own portion of memory can increase the performance.

So, that’s a look at MPI. MPI has simple calls such as “MPI_Send” and “MPI_Recv”. There is certainly some initialisation and other calls you can do but you have a call, you pass it, a buffer, a length of a buffer, a few other parameters that get shared back and forth, and a send is executed on one processor, the data is sent to the other that has a corresponding receive to receive the data in. It’s not a very difficult way to program. It requires the discipline to decompose the program and it gets you to a reasonably low level of programming as well, thinking about your parallelism so it’s definitely not for everyone but I think it’s important to understand what message passing is about and understand that it is available and certainly as we see machines get to eight and sixteen cores it’s definitely in the domain of where MPI has been used quite a bit traditionally with super computers; starting at the 10’s of cores and going into the 1000’s of cores so it’s proven itself as a very effective way to program. It’s not necessarily a favoured way to program because it takes a lot of work but it’s a very effective way.

So, an excellent place to learn more information is www.mpi-forum.org. At this place, you can find documentation on MPI, the standards and so forth. Another great place is Wikipedia. Look under message passing interface or search for MPI. You will find a lot of information, sample applications and so forth.