remove RDC flags when using CMake language CUDA #5564
Conversation
When compiling with CUDA as a CMake language, the RDC aspect is handled by CMake's CUDA_SEPARABLE_COMPILATION property on targets and the associated CMAKE_CUDA_SEPARABLE_COMPILATION variable that sets the default value for that property. The arguments set by this part of Kokkos' CMake actually conflict with the above.
|
Should |
Yes, that makes sense! Can you help me identify where in the Kokkos CMake files that should be set? I can add it to this PR then if you'd like. |
|
Yeah agreed, we should set that option then, and/or warn if there is a mismatch of that with the Kokkos setting. |
|
Is the place I already modified a good place to set the value? it happens before all Kokkos targets get created right? I'll do that unless someone tells me otherwise |
the Kokkos_ENABLE_CUDA_RELOCATABLE_DEVICE_CODE and CMAKE_CUDA_SEPARABLE_COMPILATION options both control whether CUDA RDC is on when we are building using CUDA as a CMake language. This code makes sure that if either of those options is on, the other one is also on. This way, both older build scripts that used the first option and "proper CMake" scripts that use the latter will have the same effect if building Kokkos with CUDA as a CMake language is enabled.
|
Okay, I added some code to ensure that, if we are building using CMake's CUDA language, then setting either of those options on will set both of them on. Let me know if anything else is missing from this. |
cmake/kokkos_enable_options.cmake
Outdated
| IF (CMAKE_CUDA_SEPARABLE_COMPILATION) | ||
| ELSE() | ||
| SET(CMAKE_CUDA_SEPARABLE_COMPILATION ON) | ||
| ENDIF() |
There was a problem hiding this comment.
| IF (CMAKE_CUDA_SEPARABLE_COMPILATION) | |
| ELSE() | |
| SET(CMAKE_CUDA_SEPARABLE_COMPILATION ON) | |
| ENDIF() | |
| IF (NOT CMAKE_CUDA_SEPARABLE_COMPILATION) | |
| SET(CMAKE_CUDA_SEPARABLE_COMPILATION ON) | |
| ENDIF() |
cmake/kokkos_arch.cmake
Outdated
| NVIDIA --relocatable-device-code=true | ||
| NVHPC -gpu=rdc | ||
| ) | ||
| IF (NOT KOKKOS_COMPILE_LANGUAGE STREQUAL CUDA) |
There was a problem hiding this comment.
This does not seem right
Assume RDC is OFF but we are using NVHPC L462 we'd still be adding the -gpu=nordc flag
There was a problem hiding this comment.
I agree. This if condition should guard from setting any flags.
cmake/kokkos_enable_options.cmake
Outdated
| IF (NOT CMAKE_CUDA_SEPARABLE_COMPILATION) | ||
| SET(CMAKE_CUDA_SEPARABLE_COMPILATION ON) | ||
| ENDIF() |
There was a problem hiding this comment.
I think we might want to raise a warning (or display a STATUS message) when setting the CMAKE_CUDA_SEPARABLE_COMPILATION variable
|
Thank you @masterleinad and @dalg24 for your reviews! I believe I've just addressed both of your suggestions, please let me know if other changes are desired. |
* remove RDC flags when using CMake language CUDA When compiling with CUDA as a CMake language, the RDC aspect is handled by CMake's CUDA_SEPARABLE_COMPILATION property on targets and the associated CMAKE_CUDA_SEPARABLE_COMPILATION variable that sets the default value for that property. The arguments set by this part of Kokkos' CMake actually conflict with the above. * ensure both RDC options are consistent the Kokkos_ENABLE_CUDA_RELOCATABLE_DEVICE_CODE and CMAKE_CUDA_SEPARABLE_COMPILATION options both control whether CUDA RDC is on when we are building using CUDA as a CMake language. This code makes sure that if either of those options is on, the other one is also on. This way, both older build scripts that used the first option and "proper CMake" scripts that use the latter will have the same effect if building Kokkos with CUDA as a CMake language is enabled. * try to apply Daniel Arndt's suggestions * don't set any compiler RDC flags when CMake lang see comment kokkos#5564 (comment) * add warning when changing CMake RDC flag see comment kokkos#5564 (comment) * Nitpick prefer STATUS to WARNING in message Co-authored-by: Damien L-G <[email protected]>
* remove RDC flags when using CMake language CUDA When compiling with CUDA as a CMake language, the RDC aspect is handled by CMake's CUDA_SEPARABLE_COMPILATION property on targets and the associated CMAKE_CUDA_SEPARABLE_COMPILATION variable that sets the default value for that property. The arguments set by this part of Kokkos' CMake actually conflict with the above. * ensure both RDC options are consistent the Kokkos_ENABLE_CUDA_RELOCATABLE_DEVICE_CODE and CMAKE_CUDA_SEPARABLE_COMPILATION options both control whether CUDA RDC is on when we are building using CUDA as a CMake language. This code makes sure that if either of those options is on, the other one is also on. This way, both older build scripts that used the first option and "proper CMake" scripts that use the latter will have the same effect if building Kokkos with CUDA as a CMake language is enabled. * try to apply Daniel Arndt's suggestions * don't set any compiler RDC flags when CMake lang see comment kokkos#5564 (comment) * add warning when changing CMake RDC flag see comment kokkos#5564 (comment) * Nitpick prefer STATUS to WARNING in message Co-authored-by: Damien L-G <[email protected]>
When compiling with CUDA as a CMake language,
the RDC aspect is handled by CMake's
CUDA_SEPARABLE_COMPILATION property on targets and the associated CMAKE_CUDA_SEPARABLE_COMPILATION
variable that sets the default value for that
property.
The arguments set by this part of Kokkos' CMake
actually conflict with the above.