Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reusing shared libs in gpu ci builds #487

Merged
merged 44 commits into from
Feb 7, 2022
Merged

Reusing shared libs in gpu ci builds #487

Show file tree
Hide file tree
Changes from 39 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
b50344f
Adding cpu checks for ci build
cjnolet Feb 2, 2022
e7a1e98
Updating years
cjnolet Feb 2, 2022
6e0628e
Updates
cjnolet Feb 2, 2022
2c59f69
Fixing libraft_headers
cjnolet Feb 2, 2022
c526379
Fixing per review feedback
cjnolet Feb 2, 2022
39f67d8
Merge branch 'branch-22.04' into imp-2204-ci_cpu_build
cjnolet Feb 2, 2022
d940fac
Add fatbin
cjnolet Feb 2, 2022
3cbf3b1
fixing ci conda recipe paths
cjnolet Feb 3, 2022
133ea65
Fixing typo in conda yaml files
cjnolet Feb 3, 2022
3ea2f05
Reordering conda build calls by dependencie
cjnolet Feb 3, 2022
451f8a8
Installing headers
cjnolet Feb 3, 2022
da2ec4e
Trying mamba
cjnolet Feb 3, 2022
5556e09
trying mambabuild again
cjnolet Feb 3, 2022
390c171
Updates
cjnolet Feb 3, 2022
2ad9f41
nprc
cjnolet Feb 3, 2022
bf1eb26
remove -x
cjnolet Feb 3, 2022
6e7a4d0
verbose
cjnolet Feb 3, 2022
49f44ab
Moving verbose flag
cjnolet Feb 3, 2022
386ca8d
printing verbosityi
cjnolet Feb 3, 2022
a9e9c5a
using cmake message log level
cjnolet Feb 3, 2022
acdf7aa
Trying verbose again
cjnolet Feb 3, 2022
dcc3338
adding cuda-python to build
cjnolet Feb 3, 2022
936aeca
Only installing when not compiling any libs
cjnolet Feb 3, 2022
2a2fe1b
Adding rmm to pyraft conda dependencies
cjnolet Feb 3, 2022
d87e349
Removing the wrong logging
cjnolet Feb 3, 2022
c946926
Breaking out jensenshannon specializations
cjnolet Feb 3, 2022
6ab5d4e
Fixing style
cjnolet Feb 3, 2022
375024c
No clean in conda recipes
cjnolet Feb 4, 2022
f294c8f
Adding dsome logging
cjnolet Feb 4, 2022
8a7368e
Updating logging
cjnolet Feb 4, 2022
147ddf4
Always installing cmake. Separating out nn from dist compile libs
cjnolet Feb 4, 2022
199eb5a
fatbin
cjnolet Feb 4, 2022
499708d
Breaking out separate instantiations for distances to speed up compile
cjnolet Feb 4, 2022
3e301b8
Breakig apart more distance source files
cjnolet Feb 4, 2022
773b985
Making sure files are in correct directoris for s3 upload
cjnolet Feb 4, 2022
fce4880
Adding missing src files
cjnolet Feb 4, 2022
23ea28c
Reusing the shared libs in downstream gpu tests
cjnolet Feb 4, 2022
4db545e
Updates
cjnolet Feb 4, 2022
e8df5dc
Merge branch 'branch-22.04' into build-2204-reuse_shared_libs_in_gpu_…
cjnolet Feb 4, 2022
f434053
Removing additional libraft in gpu built d
cjnolet Feb 4, 2022
cfd54b1
Small updates to the conda package upload script
cjnolet Feb 5, 2022
7d6cb0f
small improvements
cjnolet Feb 7, 2022
2c14990
Merge remote-tracking branch 'rapidsai/branch-22.04' into build-2204-…
cjnolet Feb 7, 2022
ff41cc0
Merge branch 'build-2204-upload_script_updates' into build-2204-reuse…
cjnolet Feb 7, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ VERBOSE_FLAG=""
BUILD_ALL_GPU_ARCH=0
BUILD_TESTS=YES
BUILD_STATIC_FAISS=OFF
COMPILE_LIBRARIES=${BUILD_TESTS}
COMPILE_LIBRARIES=OFF
COMPILE_NN_LIBRARY=OFF
COMPILE_DIST_LIBRARY=OFF
ENABLE_NN_DEPENDENCIES=${BUILD_TESTS}
Expand Down
2 changes: 1 addition & 1 deletion ci/cpu/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ if [[ -e /etc/os-release ]] && (grep -qi centos /etc/os-release); then
else
export PATH=/opt/conda/bin:/usr/local/cuda/bin:$PATH
fi
export PARALLEL_LEVEL=${PARALLEL_LEVEL:-4}
export PARALLEL_LEVEL=${PARALLEL_LEVEL:-8}

# Set home to the job's workspace
export HOME=$WORKSPACE
Expand Down
5 changes: 3 additions & 2 deletions ci/gpu/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,10 +94,11 @@ export LD_LIBRARY_PATH_CACHED=$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH

gpuci_logger "Build C++ and Python targets"
# These should link against the existing shared libs
if hasArg --skip-tests; then
"$WORKSPACE/build.sh" libraft pyraft libraft -v --compile-libs --nogtest
"$WORKSPACE/build.sh" libraft pyraft libraft -v --nogtest
else
"$WORKSPACE/build.sh" libraft pyraft libraft -v --compile-libs
"$WORKSPACE/build.sh" libraft pyraft libraft -v
fi

gpuci_logger "sccache stats"
Expand Down