chmod should be a read-modify-write of global meta data

[adammoody]

The chmod wrapper updates the global file meta to record new permissions of the file. In case some other process has changed the global meta data, this probably should be a read-modify-write of the global meta data. The current process may have stale info, e.g.

// if the file has "rw-" bits set
if (rank == 0) {
  MPI_Barrier(COMM_WORLD)
  chmod(path, "-r")
} else if (rank == 1) {
  chmod(path, "+x")
  MPI_Barrier(COMM_WORLD)
}
// here it should now be "-wx"
// rank 1 adds "x", then rank 0 drops "r"

Without the read-modify-write, we'd get "-w-" because rank 0 wouldn't see the change by rank 1.