Pin cudf dependencies during build #1901
Conversation
* Use rapids-cmake to pin cudf dependencies to known working SHA1's * Use git download rapids-cmake So that we can get SHA Use same format of CopyRight Use cudf/rapids_config.cmake which is renamed from fetch_rapids.cmake Update versions to latest Signed-off-by: Gary Shen <[email protected]> * Commit cudf to the latest code before mvn verify Signed-off-by: Gary Shen <[email protected]> * Remove the first commit of cudf since the second one can commit it at once Signed-off-by: Gary Shen <[email protected]> --------- Signed-off-by: Gary Shen <[email protected]> Co-authored-by: Robert Maynard <[email protected]>
Signed-off-by: Jason Lowe <[email protected]>
|
build |
|
Verified RAPIDS Accelerator zstd tests pass with this build. |
No. There's two modes to the build. The default is to use the pinned versions, but the submodule sync build, which automatically runs every few hours, will build in a mode that updates the pinned versions file to whatever versions it finds when it moves to the updated cudf version. See https://github.com/NVIDIA/spark-rapids-jni/pull/1901/files#diff-a4b52f1881e34609ce5cd5e0722da2f2f708d1eef03334723a5628082fe94d31 |
|
I just realized I should update this PR to refresh the pin versions, since it's been almost a week since those were updated in this PR. Should have that updated shortly. |
Signed-off-by: Jason Lowe <[email protected]>
|
build |
|
@jlowe This is awesome! I'm really glad this approach worked out. |
|
Thank @bdice, but all the credit goes to @robertmaynard for the idea and implementation. |
Closes #1666. This restores the changes from #1877 and updates the rapids-cmake version to avoid the issue reported in NVIDIA/spark-rapids#10627.