Advice for implementing PMPI into simple MPI program?

[Samg381]

Hello,

I am using this simple MPI C program to calculate Pi.

I would like to collect some information regarding the performance of each thread along with it's ID and elapsed time using PMPI.

I am a novice with MPI, and am unfamiliar with what this PMPI implementation might look like. Ideally this would take place all within the same file. Any guidance would be greatly appreciated.

[hzhou]

Try this (I modified the examples/cpi.c from mpich release):

/*
 * Copyright (C) by Argonne National Laboratory
 *     See COPYRIGHT in top-level directory
 */

#include "mpi.h"
#include <stdio.h>
#include <math.h>

double f(double);

double f(double a)
{
    return (4.0 / (1.0 + a * a));
}

double total_time = 0.0;

int main(int argc, char *argv[])
{
    int n, myid, numprocs, i;
    double PI25DT = 3.141592653589793238462643;
    double mypi, pi, h, sum, x;
    double startwtime = 0.0, endwtime;
    int namelen;
    char processor_name[MPI_MAX_PROCESSOR_NAME];

    MPI_Init(&argc, &argv);
    MPI_Comm_size(MPI_COMM_WORLD, &numprocs);
    MPI_Comm_rank(MPI_COMM_WORLD, &myid);
    MPI_Get_processor_name(processor_name, &namelen);

    fprintf(stdout, "Process %d of %d is on %s\n", myid, numprocs, processor_name);
    fflush(stdout);

    n = 10000;  /* default # of rectangles */
    if (myid == 0)
        startwtime = MPI_Wtime();

    MPI_Bcast(&n, 1, MPI_INT, 0, MPI_COMM_WORLD);

    h = 1.0 / (double) n;
    sum = 0.0;
    /* A slightly better approach starts from large i and works back */
    for (i = myid + 1; i <= n; i += numprocs) {
        x = h * ((double) i - 0.5);
        sum += f(x);
    }
    mypi = h * sum;

    MPI_Reduce(&mypi, &pi, 1, MPI_DOUBLE, MPI_SUM, 0, MPI_COMM_WORLD);

    if (myid == 0) {
        endwtime = MPI_Wtime();
        printf("pi is approximately %.16f, Error is %.16f\n", pi, fabs(pi - PI25DT));
        printf("wall clock time = %f\n", endwtime - startwtime);
        fflush(stdout);
    }

    MPI_Finalize();
    printf("rank %d, total reduce time %.8lf sec\n", myid, total_time);
    return 0;
}

int MPI_Reduce(const void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, MPI_Op op, int root, MPI_Comm comm)
{
    double time_start = MPI_Wtime();
    int rc = PMPI_Reduce(sendbuf, recvbuf, count, datatype, op, root, comm);
    total_time += MPI_Wtime() - time_start;
    return rc;
}

You can compile and run with

$ mpicc -o cpi examples/cpi.c
$ mpirun -n 4 ./cpi

[Samg381]

Thank you very much. I was able to compile and run the program, and am seeing the helpful console output. I also see that PMPI was used on the final reduction operation, which makes sense.

Now for my neophyte question... how do I interface with the data gathered by PMPI, or are we already technically doing it here? My end goal here is to be able to instrument an application without having to touch the source code, save for maybe replacing MPI calls for PMPI calls. I see that we're calling the MPI_Reduce function which internally calls PMPI_Reduce, but what is this doing instrumentation wise?

Again I can't thank you enough for the support.

[hzhou]

PMPI_ functions is indentical to MPI_ functions. In a sense, PMPI_ functions are the real MPI functions, and MPI_ functions are just link or aliases to PMPI_ functions. When you define your own MPI_ functions, e.g. MPI_Reduce, it runs your code, and the logging or profiling is entirely your code. See the MPI_Wtime calls in the example.

how do I interface with the data gathered by PMPI, or are we already technically doing it here?

PMPI functions does not generate any "data" other than the MPI operations. The (profiling) data is generated by MPI_Reduce as defined in the example. There are profiling libraries such as MPE that implements MPI wrappers with common profiling code. If you link with one of these profiling libraries, then refer to respective manual for its "data" interface. If you only need profiling a few specific MPI functions. you will find coding your own profiling code, as shown in the example, quite easy.