#725 enable test with mpi oversubscribe

[williamfgc]

OMPI_MCA_rmaps_base_oversubscribe=yes must be either passed to cmake or be already an
env variable when running cmake and ctest

[chuckatkins]

This is specific to openmpi. You'd have no reason to set that env var for another implementation, i.e. mpich, IntelMPI, sgi mpt, mvapich, cray mpich, etc.

[williamfgc]

@chuckatkins no openmpi specific env variable is set. I am only looking for it if the user sets it explicitly as an environment variable. Please take another look. Also, this is only done if ncores < 4. HeatTransfer tests are hardcoded to work with 4 nprocs only. The previous master would fail as-is without the proper check. Other implementations would just run with 2 nprocs as that's what the system has. See #725

[chuckatkins]

The entire logic block here can be reworked to be much simpler I think. The use case is to oversubscribe when < 4 processors are detected and the OpenMPI env var explicitly says it's okay, right? FindMPI already sets MPIEXEC_MAX_NUMPROCS to the detected number of physical cores. Given that, the whole logic block can be reduced to:

if(ADIOS2_HAVE_MPI)
  if(MPIEXEC_MAX_NUMPROCS LESS 4 AND "$ENV{OMPI_MCA_rmaps_base_oversubscribe}")
    message(STATUS "OpenMPI oversubscribe detected: raising MPIEXEC_MAC_NUMPROCS to 4 for testing")
    set(MPIEXEC_MAX_NUM_PROCS 4 CACHE STRING "" FORCE)
  endif()
  ...
endif()

I think that's right. Would that work?

[williamfgc]

@chuckatkins cmake had issues with MPIEXEC_MAX_NUMPROCS https://gitlab.kitware.com/cmake/cmake/merge_requests/1405 which were reproduced on my Mac using different cmake versions. That's why I have to use something more backwards compatible like cmake_host_system_information

[chuckatkins]

We include a private copy of the upstream FindMPI module that should already be patched for this:

ADIOS2/cmake/upstream/FindMPI.cmake

Lines 1291 to 1292 in cefab22

	cmake_host_system_information(RESULT _MPIEXEC_NUMPROCS QUERY NUMBER_OF_PHYSICAL_CORES)
	set(MPIEXEC_MAX_NUMPROCS "${_MPIEXEC_NUMPROCS}" CACHE STRING "Maximum number of processors available to run MPI applications.")

so the cmake_host_system_information workaround in the main CMakeLists.txt shouldn't be necessary anymore. Does it still not work correctly for you?

[williamfgc]

@chuckatkins yes, that works using cmake 3.12 on my Mac (we were forced to upgrade).