Need to be able to specify mapping of GPU to local process

[maddyscientist]

With the advent of NVLink, getting the correct node topology is now more complicated than just ensuring correct NUMA placement of processes for their respective GPUs. For dense GPU systems with multiple NVLinks we need a method to easily map the local process id to GPU device ordinal. Failing to do so will result in sub-optimal use of peer-to-peer bandwidth and message being unnecessarily routed through CPU memory.

[mathiaswagner]

NVML's setCPUAffinity does not work? Or is the mapping needed on the mpirun level ?

[maddyscientist]

Closing this issue, as this required functionality is trivially obtained from the CUDA_VISIBLE_DEVICES environment variable (the order that devices are listed in this variable corresponds to the order that the CUDA runtime sees the device).