The essence of doing distributed memory parallel processing involves sending and receiving data from one processing node to another. The core vehicles for data transfer are MPI_SEND and MPI_RECV. There are some particulars of which the user must be aware. The correct MPI_Data_Type, MPI_Comm and itag variables must be set. The programs in the manual use numerous data types: integer*2, integer*4, double precision, double complex and byte. For each one of these, there is a corresponding MPI data type.
Assume processor P sends a 2-D slice of array pix(ny, ny, k) of data type INTEGER*2 to a 2-D array on processor P + 1 named Y (nx, ny). The number of elements count is simply nx x ny.
But since they are being sent \byte-wise", one is really sending 2 x nx x ny elements. Therefore, the necessary pair of matching SEND and RECV calls are:
This MPI_SEND and MPI_RECV calls are technically called a blocking send and a blocking receive, respectively. In other words, the processor waits (or blocks) for the message to complete before it continues with the executable.
The timing of sending and receiving can be crucial at times. One processor may already know that it has to be sending data, but the receiving node may not be ready. In such cases, it is possible to turn a data receive into a two-stage process with calls to MPI_IRECV and MPI_WAIT. MPI_IRECV performs a non-blocking receive using the handle irequest. The MPI WAIT keeps the MPI_IRECV from completing until it gets irequest and a legitimate status. This method is used in the subroutines t2b, b2t and their variants. Of course there is a matching MPI_SEND call posted by another processor, but it is not shown.
There are also cases in which one processor may finish its series of calculations before other processors do. Therefore a mechanism for pausing is built into MPI as MPI_BARRIER. No processor will pass this statement until all are ready to pass.
There have been several integer variables used in this section that have never been explicitly defined in the code examples. It is good programming practice to define them at the beginning of the code, so they are listed here for completeness.
