-
Notifications
You must be signed in to change notification settings - Fork 915
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add cuDF devcontainers #14015
Add cuDF devcontainers #14015
Conversation
/ok to test |
/ok to test |
Removes the unified and isolated devcontainers in favor of just the single one. The unified devcontainers are [still available](https://github.com/rapidsai/devcontainers/blob/d970ac17a894f5d1047fcfd17cffa6d14fe66ec8/.devcontainer/cuda12.0-conda/devcontainer.json#L23-L31) in the [`rapidsai/devcontainers`](https://github.com/rapidsai/devcontainers) repository. Adds an optional job to the `pr.yaml` to [build the cuGraph libs in each devcontainer](https://github.com/trxcllnt/cuspatial/blob/fea/devcontainers/.github/workflows/pr.yaml#L106-L111), so the build caches are populated for devs by CI. Other PRs: * rapidsai/rmm#1328 * rapidsai/kvikio#273 * rapidsai/cudf#14015 * rapidsai/raft#1791 * rapidsai/cumlprims_mg#149 * rapidsai/cuml#5568 * rapidsai/cugraph-ops#538 * rapidsai/cugraph#3838 Authors: - Paul Taylor (https://github.com/trxcllnt) - H. Thomson Comer (https://github.com/thomcom) Approvers: - Mark Harris (https://github.com/harrism) - Jake Awe (https://github.com/AyodeAwe) URL: #1263
/ok to test |
/ok to test |
/ok to test |
We'll have to wait a bit for some of cudf's dependencies to have nightly builds. It's a bit late for this to target 23.12 but we'll be able to finalize this early in the 24.02 development cycle. In the meantime, I'm testing out the unified devcontainers with version 23.12 of everything. |
/ok to test |
/ok to test |
/ok to test |
/ok to test |
/merge |
Blocked by #14452. I requested an admin-merge in the meantime. |
Description
This PR adds some devcontainers to help simplify building the cuDF C++ and Python libraries.
It also adds an optional job to the
pr.yaml
to build the cuDF libs in each devcontainer, so the build caches are populated for devs by CI.A devcontainer can be launched by clicking the "Reopen in Container" button that VSCode shows when opening the repo (or by using the "Rebuild and Reopen in Container" command from the command palette):
Clicking this button will cause VSCode to prompt the user to select one of these devcontainer variants:
On startup, the devcontainer creates or updates the conda/pip environment using
cudf/dependencies.yaml
. The envs/package caches are cached on the host via volume mounts, which are described in more detail in.devcontainer/README.md
.The container includes convenience functions to clean, configure, and build the various cuDF components:
cmake -S ~/cudf/cpp -B ~/cudf/cpp/build
andcmake --build ~/cudf/cpp/build
pip install --editable ~/cudf/cpp
Unlike
build.sh
, these convenience scripts don't install the libraries after building them. Instead, they automatically inject the correct arguments to build the C++ libraries from source and use their build dirs as package roots:Checklist