MPI Quiz

A fun scenario was proposed in the MPI Forum today.  What do you think this code will do?

MPI_Comm comm, save;
MPI_Request req;
MPI_Init(NULL, NULL);
MPI_Comm_dup(MPI_COMM_WORLD, &comm);
MPI_Comm_rank(comm, &rank);
save = comm;
MPI_Isend(smsg, 4194304, MPI_CHAR, rank, 123, comm, &req);
MPI_Comm_free(&comm);
MPI_Recv(rmsg, 4194304, MPI_CHAR, rank, 123, save, MPI_STATUS_IGNORE);

This code is obviously an error, because you’re using comm after you free it.

…but will it work?

The shocking thing is: depending on your MPI implementation, the above code may not crash (!).

The key here is that MPI allows you to free handles while they are still in use.  Since we started a non-blocking send before calling MPI_COMM_FREE, the communicator still exists in the background — even if comm has been set to MPI_COMM_NULL at the end of MPI_COMM_FREE.

But since we saved a copy of the handle that points to the underlying communicator object in “save”, we use it again in the matching receive. Since the communicator has not yet been destroyed, it’s still there… and can be used (!).

But NOT REALLY.

Remember: this is erroneous, according to the MPI specification.

It’s a quirk of the implementation that keeps the communicator around and doesn’t mark it as “freed”, and therefore disallow new operations on it (note: both Open MPI 1.7.x and MPICH 3.0.4 do not fail this program).

But it’s still fun, nonetheless.  🙂

Share