-
Notifications
You must be signed in to change notification settings - Fork 83
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Full CI support for public builds + switch to use cibuildwheel (#267)
* switch to cibuildwheel + some cleanups * try setting up Python manually * comment out un-needed code + propagate python-version * fix: need to check out first * only build natively; add -v; ensure targeting manylinux * single quotes * restore env setup to get artifact dir * fix artifact dir * fix artifact name * restore & fix artifact name again * build on all platforms! * fix CIBW_BUILD for windows * fix typo * no quotes for wildcard matching * move CIBW_BUILD logic to script * fix win runner name * try to find where pwsh is * try to escape * continue hunting.. * try to overwrite shell * try to install ps * be explicit about shell (why?) * only build for win 64 bits * try to install msvc * install msvc ourselves * fix typo * skip custom cl ver check * install to standard location * try to locate Python include path * switch to public windows runner for now * windows image does not have sudo * pwd on Windows Bash does not use Windows path format * cover all Python versions! * add quotes * align the python version installed via GHA vs used at build time * fix constraint syntax * check if setup-python is causing interference * fix typo * apply a WAR on Linux * fix unbound var * detect Python path after it's installed (by CIBW) * try CIBW_BEFORE_ALL_LINUX * try to restore the pre-py-span setup... * reduce build matrix to experiment with cuda.bindings builds * fix parenthesis * use abs path * defer CUDA_PATH setting * use CIBW_ENVIRONMENT to pass env var * fetch cuda-profiler-api headers * only rely on redist * allow wheel repair to fix the triplet tags * restore full build matrix! * fix wget on Windows; pass PARALLEL_LEVEL to CIBW * switch from wget to curl * windows needs unzip not tar * mv -> rsync * git bash has no wget or rsync... * ensure win-style path on win * convert abs path * debug * another level down * check if it is a race condition on win * clean up unused (but still relevant) code * clean up unused (but still relevant) code - cont'd * consolidate with PYTHON_VERSION_FORMATTED
- Loading branch information
Showing
5 changed files
with
178 additions
and
89 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -13,73 +13,79 @@ inputs: | |
| host-platform: | ||
| required: true | ||
| type: string | ||
| use-container: | ||
| required: true | ||
| type: boolean | ||
| docker-image: | ||
| type: string | ||
| required: true | ||
| upload-enabled: | ||
| required: true | ||
| type: boolean | ||
| python-version: | ||
| required: true | ||
| type: string | ||
|
|
||
| runs: | ||
| using: composite | ||
| steps: | ||
| - name: Build cuda.core wheel | ||
| uses: pypa/[email protected] | ||
| env: | ||
| CIBW_BUILD: ${{ env.CIBW_BUILD }} | ||
| CIBW_ARCHS_LINUX: "native" | ||
| CIBW_BUILD_VERBOSITY: 1 | ||
| # # ensure Python.h & co can be found | ||
| # CIBW_BEFORE_BUILD_WINDOWS: > | ||
| # python -c "import sysconfig; print(sysconfig.get_path('include'))" >> $env:INCLUDE | ||
| with: | ||
| package-dir: ./cuda_core/ | ||
| output-dir: ${{ env.CUDA_CORE_ARTIFACTS_DIR }} | ||
|
|
||
| - if: ${{ inputs.use-container }} | ||
| name: Build (in container) | ||
| shell: bash --noprofile --norc -xeuo pipefail {0} | ||
| run: | | ||
| docker run \ | ||
| -e AWS_REGION \ | ||
| -e AWS_SESSION_TOKEN \ | ||
| -e AWS_ACCESS_KEY_ID \ | ||
| -e AWS_SECRET_ACCESS_KEY \ | ||
| -e GITHUB_TOKEN \ | ||
| -e BINDINGS_ARTIFACTS_DIR="$BINDINGS_ARTIFACTS_DIR" \ | ||
| -e CORE_ARTIFACTS_DIR="$CORE_ARTIFACTS_DIR" \ | ||
| -e UPLOAD_ENABLED="$UPLOAD_ENABLED" \ | ||
| -e USE_CUDA="$USE_CUDA" \ | ||
| -e REPO_DIR="$REPO_DIR" \ | ||
| -e LEGATE_CORE_BUILD_MODE="$LEGATE_CORE_BUILD_MODE" \ | ||
| -e PYTHON_VERSION="$PYTHON_VERSION" \ | ||
| -v "${{ env.REPO_DIR }}:${{ env.REPO_DIR }}" \ | ||
| -v "${{ env.BINDINGS_ARTIFACTS_DIR }}:${{ env.BINDINGS_ARTIFACTS_DIR }}" \ | ||
| -v "${{ env.CORE_ARTIFACTS_DIR }}:${{ env.CORE_ARTIFACTS_DIR }}" \ | ||
| --rm "${{ inputs.docker-image }}" \ | ||
| /bin/bash -c "${{ env.REPO_DIR }}/continuous_integration/scripts/entrypoint ${{ env.REPO_DIR }}/continuous_integration/scripts/build ${{ inputs.build-type}} ${{ inputs.target-device }}" | ||
| - if: ${{ !inputs.use-container }} | ||
| name: Build (without container) | ||
| shell: bash --noprofile --norc -xeuo pipefail {0} | ||
| run: | | ||
| "${{ env.REPO_DIR }}/continuous_integration/scripts/entrypoint" "${{ env.REPO_DIR }}/continuous_integration/scripts/build" "${{ inputs.build-type}}" "${{ inputs.target-device }}" | ||
| - name: Display structure of the bindings artifacts folder (post build) | ||
| - name: List the cuda.core artifacts directory | ||
| shell: bash --noprofile --norc -xeuo pipefail {0} | ||
| run: | | ||
| sudo chown -R $(whoami) ${{ env.BINDINGS_ARTIFACTS_DIR }} | ||
| ls -lahR ${{ env.BINDINGS_ARTIFACTS_DIR }} | ||
| if [[ "${{ inputs.host-platform }}" == win* ]]; then | ||
| export CHOWN=chown | ||
| else | ||
| export CHOWN="sudo chown" | ||
| fi | ||
| $CHOWN -R $(whoami) ${{ env.CUDA_CORE_ARTIFACTS_DIR }} | ||
| ls -lahR ${{ env.CUDA_CORE_ARTIFACTS_DIR }} | ||
| - name: Upload bindings build artifacts | ||
| - name: Upload cuda.core build artifacts | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: ${{ env.BINDINGS_ARTIFACT_NAME }} | ||
| path: ${{ env.BINDINGS_ARTIFACTS_DIR }} | ||
| name: ${{ env.CUDA_CORE_ARTIFACT_NAME }} | ||
| path: ${{ env.CUDA_CORE_ARTIFACTS_DIR }}/*.whl | ||
| if-no-files-found: error | ||
| overwrite: 'true' | ||
|
|
||
| - name: Build cuda.bindings wheel | ||
| uses: pypa/[email protected] | ||
| env: | ||
| CIBW_BUILD: ${{ env.CIBW_BUILD }} | ||
| CIBW_ARCHS_LINUX: "native" | ||
| CIBW_BUILD_VERBOSITY: 1 | ||
| CIBW_ENVIRONMENT_LINUX: > | ||
| CUDA_PATH="$(realpath ./cuda_toolkit)" | ||
| PARALLEL_LEVEL=${{ env.PARALLEL_LEVEL }} | ||
| CIBW_ENVIRONMENT_WINDOWS: > | ||
| CUDA_HOME="$(cygpath -w $(realpath ./cuda_toolkit))" | ||
| # PARALLEL_LEVEL=${{ env.PARALLEL_LEVEL }} | ||
| # # ensure Python.h & co can be found | ||
| # CIBW_BEFORE_BUILD_WINDOWS: > | ||
| # python -c "import sysconfig; print(sysconfig.get_path('include'))" >> $env:INCLUDE | ||
| with: | ||
| package-dir: ./cuda_bindings/ | ||
| output-dir: ${{ env.CUDA_BINDINGS_ARTIFACTS_DIR }} | ||
|
|
||
| - name: Display structure of the core artifacts folder (post build) | ||
| - name: List the cuda.bindings artifacts directory | ||
| shell: bash --noprofile --norc -xeuo pipefail {0} | ||
| run: | | ||
| sudo chown -R $(whoami) ${{ env.CORE_ARTIFACTS_DIR }} | ||
| ls -lahR ${{ env.CORE_ARTIFACTS_DIR }} | ||
| if [[ "${{ inputs.host-platform }}" == win* ]]; then | ||
| export CHOWN=chown | ||
| else | ||
| export CHOWN="sudo chown" | ||
| fi | ||
| $CHOWN -R $(whoami) ${{ env.CUDA_BINDINGS_ARTIFACTS_DIR }} | ||
| ls -lahR ${{ env.CUDA_BINDINGS_ARTIFACTS_DIR }} | ||
| - name: Upload core build artifacts | ||
| - name: Upload cuda.bindings build artifacts | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: ${{ env.CORE_ARTIFACT_NAME }} | ||
| path: ${{ env.CORE_ARTIFACTS_DIR }} | ||
| name: ${{ env.CUDA_BINDINGS_ARTIFACT_NAME }} | ||
| path: ${{ env.CUDA_BINDINGS_ARTIFACTS_DIR }}/*.whl | ||
| if-no-files-found: error | ||
| overwrite: 'true' | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.