Tweaks to Trilinos to get working with pre-installed Kokkos and KokkosKernels (with TriBITS updates) (#11545) #11546
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bartlettroscoe
added
TriBITS
|
Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: Trilinos_PR_gcc-8.3.0
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_gcc-8.3.0-serial
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_gcc-8.3.0-debug
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_clang-11.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_cuda-11.4.2-uvm-off
Jenkins Parameters
Using Repos:
Pull Request Author: bartlettroscoe |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: Trilinos_PR_gcc-8.3.0
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_gcc-8.3.0-serial
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_gcc-8.3.0-debug
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_clang-11.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_cuda-11.4.2-uvm-off
Jenkins Parameters
|
…iles (trilinos#11545) Just need to move where tribits_package_decl() is called before Kokkos defines its options and then call tribits_pkg_export_cache_var() inside of kokkos_option(). NOTE: We don't export the variables Kokkos_ENABLE_TESTS or Kokkos_ENABLE_EXAMPLES because those are special varaibles defined by TriBITS where the project-level variable value may be different than the cache variable value (which is on purpose) and also we don't want to export these variables because downstream packages should not need to know this info. ToDo: Kokkos really should differentiate what options values it exports and which it does not to provide a better defined API (and downstream customers don't need to grep the installed Kokkos_config.h file to figure out this info).
When ParMEITS is found by an upstream cmake project (e.g. KokkosKerenls), the value of HAVE_PARMETIS_VERSION_4_0_3l does not get written into the ParMETISConfig.cmake file so Zoltan2 was configuring with an error. But this is a check for Zoltan2, not other Trilinos packages anyway so this check should be in Zoltan2, not in the FindTPLParMETIS.cmake NOTE: If we want to support exporting various variables into the generated <tplName>Config.cmake files for TriBITS TPLs generated using tribits_tpl_find_include_dirs_and_libraries(), then we will need to do some refactoring in TriBITS to make that possible. But that should not be too hard. This was just not needed in this case.
bartlettroscoe
force-pushed
the
11545-pre-installed-kokkos-tribits
branch
from
2d56274 to
fd8b701
Compare
|
Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: Trilinos_PR_gcc-8.3.0
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_gcc-8.3.0-serial
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_gcc-8.3.0-debug
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_clang-11.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_cuda-11.4.2-uvm-off
Jenkins Parameters
Using Repos:
Pull Request Author: bartlettroscoe |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED Pull Request Auto Testing has PASSED (click to expand)Build InformationTest Name: Trilinos_PR_gcc-8.3.0
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_gcc-8.3.0-serial
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_gcc-8.3.0-debug
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_clang-11.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_cuda-11.4.2-uvm-off
Jenkins Parameters
|
|
Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging |
|
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
4 similar comments
|
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
|
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
|
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
|
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
There was a mistake in the defintion of the test TrilinosInstallTests_find_package_Trilinos in that it created a dummy CMake project right from within the package directory: packages/TrilinosInstallTests/ When I changed the name of a base dir locally, the test failed showing the error: CMake Error: The current CMakeCache.txt directory <common-base>/demo-external-kokkos/Build_Trilinos/packages/TrilinosInstallTests/CMakeCache.txt is different than the directory <common-base>/Build_Trilinos/packages/TrilinosInstallTests where CMakeCache.txt was created. This may result in binaries being created in the wrong place. If you are not sure, reedit the CMakeCache.txt Now the dummy project dir is in a new subdir that is deleted and created anew each time the test runs.
bartlettroscoe
added
the
AT: WIP
When Kokkos is supplied as an external package, the modern CMake imported targets from the Kokkos<Subpkg>Config.cmake files also provide the needed flags. Therefore, there should be no special mention of Kokkos in the Trilinos configure logic.
|
I set up a repo to drive the demonstration of this use case using a Trilinos PR GenConfig configuration in: I did the builds locally a few times and did a final run on the CEE machine 'ceerws1113' using (slightly updated instructions so others can duplicate): and that posted to: showing:
Looking at the configure output for KokkosKernels and Trilinos, you can see that they are using already-installed Kokkos. For example, the KokkosKernels configure shows: and we see something similar for the configure of the rest of Trilinos using pre-installed Kokkos and KokkosKernls: NOTE: Above it shows duplicate printing of lines like: For a large dependency graph, that will become a lot of redundant output, so that is something that I will need to fix in TriBITS. Update 2/17/2023 I updated the load-env scripts to allow people to use other locations for Trilinos (the abs path was accidentally hard-coded into the script |
…rilinos#11545) With the TriBITS modernization refactoring (TriBITSPub/TriBITS#299) and the generalizated handling of intenral and external packages (TriBITSPub/TriBITS#63), we need packages like Kokkos to set critical compiler options as target properties so that they will be exported in the generated IMPORTED targets of the Kokkos<Subpkg>Targets.cmake file. This is needed, for example, to pass some critical compiler flags from the pre-installed Kokkos to downstream CMake configures of KokkosKernels and the rest of Trilinos (see trilinos#11545).
bartlettroscoe
added
AT: RETEST
|
After the new commit, I will run this again to make sure all of the PR builds are working. I will post the results from the CUDA build case the required |
|
Status Flag 'Pull Request AutoTester' - User Requested Retest - Label AT: RETEST will be reset after testing. |
|
Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: Trilinos_PR_gcc-8.3.0
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_gcc-8.3.0-serial
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_gcc-8.3.0-debug
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_clang-11.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_cuda-11.4.2-uvm-off
Jenkins Parameters
Using Repos:
Pull Request Author: bartlettroscoe |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: Trilinos_PR_gcc-8.3.0
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_gcc-8.3.0-serial
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_gcc-8.3.0-debug
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_clang-11.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_cuda-11.4.2-uvm-off
Jenkins Parameters
|
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: Trilinos_PR_gcc-8.3.0
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_gcc-8.3.0-serial
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_gcc-8.3.0-debug
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_clang-11.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_cuda-11.4.2-uvm-off
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_intel-2021.3
Jenkins Parameters
|
trilinos-autotester
removed
the
AT: RETEST
|
FYI: The last PR iteration failed due to two test timeouts shown here:
showing:
Now we have to run all of these builds from scratch again. Fingers crossed :-) |
bartlettroscoe
added
AT: RETEST
|
Status Flag 'Pull Request AutoTester' - User Requested Retest - Label AT: RETEST will be reset after testing. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: Trilinos_PR_gcc-8.3.0
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_gcc-8.3.0-serial
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_gcc-8.3.0-debug
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_clang-11.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_cuda-11.4.2-uvm-off
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_intel-2021.3
Jenkins Parameters
Using Repos:
Pull Request Author: bartlettroscoe |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED Pull Request Auto Testing has PASSED (click to expand)Build InformationTest Name: Trilinos_PR_gcc-8.3.0
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_gcc-8.3.0-serial
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_gcc-8.3.0-debug
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_clang-11.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_cuda-11.4.2-uvm-off
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_intel-2021.3
Jenkins Parameters
|
trilinos-autotester
removed
the
AT: RETEST
|
Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging |
|
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
|
CC: @rppawlo, @csiefer2, @crtrott, @ibaned, @sebrowne, @jwillenbring @trilinos/framework, @trilinos/zoltan2, @trilinos/kokkos, teams, I need at someone to review and approve these changes and get this PR to merge before moving forward with: NOTE: As listed in the tasks above, I will pull off the Kokkos changes and apply them to a Kokkos branch and post a PR to the Kokkos GitHub repo. But that can be done after this PR merges, technically. NOTE: The TriBITS changes have already been reviewed in PR TriBITSPub/TriBITS#560 and merged to Trilinos 'develop' in PR #11746. Please jsut review the non-TriBITS changes. (I am not sure why GitHub is even still showing those TriBITS commits because those commits have already been merged to 'develop'.) |
|
Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging |
|
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
crtrott
approved these changes
Mar 31, 2023
|
Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ crtrott ]! |
|
Status Flag 'Pull Request AutoTester' - Pull Request will be Automerged |
|
Merge on Pull Request# 11546: IS A SUCCESS - Pull Request successfully merged |
trilinos-autotester
removed
the
AT: AUTOMERGE
This was referenced Apr 5, 2023
|
FYI: The Kokkos changes applied to the Kokkos GitHub repo are in the PRs: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR contains changes to TriBITS and Trilinos for:
This brings in the TriBITS changes as of the merge of the TriBITS PR:
and therefore, also includes the TriBITS-only PR:
FYI: The matching PR without the TriBITS changes is:
and that one may be more attractive to review file-by-file. (But this PR #11546 can be reviewed commit by commit just fine).
Testing
See comments below.
Tasks