Parallel Computing with MPICH2


MPI is a library of routines that can be used to create parallel programs in C or Fortran77.  MPI is designed to allow users to create programs that can run efficiently on most parallel architectures. MPI can also support distributed program execution on heterogeneous hardware. Such that we can start processes in multiple computer systems to work on the same problem. In this report, we will explain the power of parallel programming using a simple program that utilizes MPI.

About this program 

This program calculates sum and mean of random numbers between 0 and 1. It generates 1000 random numbers in each process and computes the sum and the mean at each process. Then it integrates the sums and means across the processes and calculates the overall sum and overall mean. This result is output to the user.  The parallel program executes on several different machines. Here we have used several virtual machines.


Software Used 

 MPICH2

 MPICH is a freely available, portable implementation of MPI, a standard for message-passing for distributed-memory applications used in parallel computing.

NFS(Network File System)

 NFS is short for Network File System. We used it to create a shared directory to share file easily across computers.

OpenSSH Server 

 We used OpenSSH to set up authentication among computers without having to enter passwords.

NANO

 Simple command line text editor for Linux.

Oracle VM Virtual Box

Ubuntu Linux 13.04


Program in Detail


This program generates 1000 random numbers between 0-1 in each process and calculates both sum and mean separately for each process and finally reduce the result using MPI_SUM function and calculate overall sum and overall mean.

Firstly, we need to import or link necessary header files to run this parallel program written in C.  So at the top we have included stdio.h (to handle basic standard input and output), mpi.h (reference functions in the MPI), math.h (to handle mathematical functions) and stdlib.h (to utilize standard libraries in c).  

 MPI_Init(&argc, &argv)

This function setup the basic MPI environment. Even though we don’t use the command line parameters we can pass as arguments to this function.

 MPI_Comm_size(MPI_COMM_WORLD,&nProcesses)

This function determines the size of the communicator. Here we have the default communicator MPI_COMM_WORLD. Number of processes will be stored in the variable nProcesses.

 MPI_Comm_rank(MPI_COMM_WORLD, &id)

MPI_Comm_rank function Determines the ranks of the calling process in the communicator. The rank then will be stored in the variable id.

MPI_Reduce(&sum,&overall_sum,1,MPI_DOUBLE,MPI_SUM,0,MPI_COMM_WORLD)

This function requires several parameters. The first argument &sum represents an array of elements of given defined data type which is MPI_DOUBLE. In overall_sum variable stores the final result of this function. We need to pass the reducing function. Here we used MPI_SUM which nothing but getting the sum of all the elements.

  MPI_Reduce(&mean,&overall_mean,1,MPI_DOUBLE,MPI_SUM,0,MPI_COMM_WORLD)

This is exactly the same function above described. Instead of Sum this function is used to accumulate mean values.

The calculation part is quite simple. Here each process loop over 1000 times and during each iteration it generates a random number and adds it to the sum. Finally once the loop is finished it computes the mean and store in variable mean.


 Screen Shots 

This screen depicts the master PC (on top) and slave PC (bottom). We have setup MPI, NFS shared folder and the openSSH among these machines.
Master shows the directory listing under the shared folder /mirror. As we can see we have compiled the random_sum.c using MPICC compiler and generate the output file called random_sum.

On the Slave, we have run htop for network analysis.  Slave can access /mirror folder which is our shared folder. Any file added by Master will be available to Slave. This eases the process of copying and pasting each file across the machines.

Figure 02



Now we run the command to execute the program across the machines.

mpiexec –f hosts –n 5 ./rand_sum

hosts is a file that contains all the IP addresses of slaves and master machines. –n is number of processes. Here we defined 5 processes.
Note that before executing this command, slave doesn’t have any processes running with the name “random_sum”.

 
Figure 03
                                                                             
This screen shot was taken while the program execution. The result shown in the console is a trial run. Ignore it for now. 
Now while it is executing you can see the program is running on the Slave machine. Note the Command column. It has random_sum. This shows the parallel processing. The program was executed in the master. But it is running in parallel in the slave.

Figure 04

Following screen shot displays the final result. 

 Generated 50000 random numbers 
Total sum: 24997.119693 and Mean: 0.499942
Ready


Figure 05
So it is clear that the program ran on multiple machines in parallel. 

 Once each machine calculates sum and mean individually they have been accumulated by MPI_Reduce function and display the overall_sum and overall_mean.

 

Comments

  1. AnonymousJuly 12, 2019 at 6:30 AM

    This comment has been removed by the author.

    ReplyDelete
  2. AnonymousJuly 17, 2019 at 6:21 AM

    This comment has been removed by the author.

    ReplyDelete

Post a Comment

Popular posts from this blog

Dependency Injection in ASP.NET MVC 5 with StructureMap

Dependency Injection(DI) is a design pattern that let you write loosely coupled testable code. Typically your classes depend on other classes. Take a look at the following example. The above example shows the " HomeController " of an asp.net mvc application. Within the index method, you can see that it requires to create an instance of " Student " class to get the full name of a student. It is very clear that the Index method is tightly coupled with the Student class.  This is a disadvantage when it comes to testing HomeController . Suppose that you have other methods, with similar types of class dependencies. So, each time when you test the code, you have to pass an actual instance Student class. This is tedious. You don't have the luxury of passing a fake student class to easily test your controllers.  So how do we decouple the Student class from HomeController? This is where DI comes in handy. You can inject dependencies in several ways. Today i will sho
Read more

Real-time web with Node.js & Socket.io

Image
If you are a .net developer, you may be familiar with ASP.net SignalR for real-time web development. It's one of the coolest libraries for ASP.net that allows bi-directional communication between a server and a client. For more details refer  http://www.asp.net/signalr .  This blog post is an intro to use Node.js and Socket.io to achieve real-time web communication with pure JavaScript using a simple chat program example. Socket.io is a library that makes our lives easy to work with web sockets. Of course, in the Javascript world, there are hundred other libraries available to implement web sockets,  examples .  But i like the simplicity of Socket.io.  Step 01 : Setting up Node.js Depending on the operating system you are using, download Node.js from the official site  here . Installation process of both Windows and OSX is pretty much the same. Double click the setup file and follow the straightforward instruction. For Linux, either you can do
Read more

First Look at AngularJS 2.0

Image
AngularJS 2.0 developer preview has been released! This blog post is to experience the new features in the framework with a simple angular2 application. Heads Up : AngularJS 2.0 is written in TypeScript. TypeScript is a super-set of ES6 which is a super-set of ES5 which is currently the browsers fully supported version of JavaScript.  So when writing angular2 components, we can use choose among TypeScript, ES6, ES5 or Dart if you will. For this blog post I use TypeScript. Please refer  5 Min Quick Start Guide  to setup angular2 development environment. Once you have setup the environment and followed the guide, you will be able to see a similar result as follows. Now let's add few more features to this. We will create a shopping list where a user can view the list, add a new item or remove a item from the list. So to begin, let's create the listItems array. Change the app.ts file as follows. Let's create the list.html file which is the view for this c
Read more