Allow enabling NVTX markers by downstream projects after install

[achirkin]

Allow downstream projects enable NVTX option for the raft::raft target, if it hasn't been enabled at the install time.

[achirkin]

This PR should replace rapidsai/cuml#4684 . Tagging @robertmaynard @vinaydes @cjnolet to continue the discussion, please let me know what you think of it.

Note specifically, if NVTX is enabled at the install time, there is no way to turn it off in this proposal.

[vinaydes]

Looks good to me. I'll close my PR if we decide to merge this one.

[teju85]

Just curious, why not just enable nvtx markers all the time?

[cjnolet]

This LGTM but would like @robertmaynard’s thoughts before merging

[achirkin]

Update, as per prior suggestions:

• I enabled NVTX markers by default when building from source;
• Inverted the build flag to allow to disable them;
• Renamed the corresponding cmake option; hence the cuml PR Allow enabling raft NVTX markers when raft is installed cuml#4718 to adjust for the changes

[cjnolet]

@achirkin the PR looks good. I think there's still a compilation error showing up in the CI logs:

04:31:43 /workspace/cpp/include/raft/common/detail/nvtx.hpp(153): error: expression must have a constant value
04:31:43 /workspace/cpp/include/raft/common/detail/nvtx.hpp(153): note: cannot call non-constexpr function "raft::common::nvtx::detail::nvtxDomainCreateA"
04:31:43 /usr/local/cuda/include/nvToolsExt.h(1502): here
04:31:43           detected during:
04:31:43             instantiation of class "raft::common::nvtx::detail::domain_store<Domain, std::enable_if_t<std::is_same<const char *, std::decay<decltype(Domain::name)>::type>::value, Domain>> [with Domain=raft::common::nvtx::domain::app]" 
04:31:43 (166): here
04:31:43             instantiation of "void raft::common::nvtx::detail::push_range_name<Domain>(const char *) [with Domain=raft::common::nvtx::domain::app]" 
04:31:43 (177): here
04:31:43             instantiation of "void raft::common::nvtx::detail::push_range<Domain,Args...>(const char *, Args...) [with Domain=raft::common::nvtx::domain::app, Args=<const char *, const char *>]" 
04:31:43 /workspace/cpp/include/raft/core/nvtx.hpp(98): here
04:31:43             instantiation of "void raft::common::nvtx::push_range(const char *, Args...) [with Domain=raft::common::nvtx::domain::app, Args=<const char *, const char *>]" 
04:31:43 /workspace/cpp/include/raft/core/nvtx.hpp(141): here
04:31:43             instantiation of "raft::common::nvtx::range<Domain>::range(const char *, Args...) [with Domain=raft::common::nvtx::domain::app, Args=<const char *, const char *>]" 
04:31:43 /workspace/cpp/test/distance/distance_base.cuh(427): here
04:31:43             instantiation of "void raft::distance::DistanceTest<distanceType, DataType>::SetUp() [with distanceType=raft::distance::CorrelationExpanded, DataType=float]" 
04:31:43 
04:31:43 1 error detected in the compilation of "/workspace/cpp/test/distance/dist_correlation.cu".

[ajschmidt8]

Approving ops-codeowner file changes

[achirkin]

Apparently, the NVTX implementation was bugged for cuda <= 11.2 because of the usage of nvtxDomainCreateA as an initializer of a static const pointer. Should work properly now!

[achirkin]

rerun tests

[achirkin]

rerun tests

[cjnolet]

rerun tests

[cjnolet]

@gpucibot merge