There are multiple instances in the code of getting a sum of all the elements in one of the large principal arrays. Since the arrays per processor are only slices, summing up their elements would result in nprocs sums; the sum of these sums is the global sum of interest. After the operation, the result is sent to all processors. Fortunately, there is a call in MPI which will perform global mathematical procedures and return the result back to root (myrank = 0). To do this, MPI_Data_Type and which operation to perform must be defined.
Here is an example of using FORTRAN90 and MPI_SUM to perform the global sum of array A. Each processor has a portion of A, so the partial sum on a processor is made by using the FORTRAN90 operation, SUM on array A. Then the operation MPI_SUM performs a global summation of a single (count = 1) real variable dX from each node and sums it into variable X and all nodes have this information.