Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cross-compile linux_aarch64 & linux_ppc64le #58

Merged
merged 13 commits into from
Dec 19, 2023
Merged

Cross-compile linux_aarch64 & linux_ppc64le #58

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
809e2e0
dummy commit for rerendering
conda-forge-webservices[bot] Dec 19, 2023
e05bfd1
MNT: Re-rendered with conda-build 3.27.0, conda-smithy 3.30.1, and co…
Dec 19, 2023
2e633e3
Switch to cross-compiling for arch builds
jakirkham Dec 19, 2023
3b43512
MNT: Re-rendered with conda-build 3.27.0, conda-smithy 3.30.1, and co…
Dec 19, 2023
7b459c8
Add `cuda-version` to `build` too
jakirkham Dec 19, 2023
b209cd7
Break up `skip`s
jakirkham Dec 19, 2023
4725883
Use `cuda-python` version to select CUDA version
jakirkham Dec 19, 2023
6698452
Add Python cross-compilation dependencies
jakirkham Dec 19, 2023
593242d
Point `CUDA_HOME` to `PREFIX`
jakirkham Dec 19, 2023
5ffc9e5
Set `CUDA_HOME` to `LIBRARY_PREFIX` on Windows
jakirkham Dec 19, 2023
9226503
Check `BUILD_PREFIX` headers last on Windows
jakirkham Dec 19, 2023
d5bd2bd
Revert "Set `CUDA_HOME` to `LIBRARY_PREFIX` on Windows"
jakirkham Dec 19, 2023
bd30a28
Bump `build/number` to `4`
jakirkham Dec 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions .azure-pipelines/azure-pipelines-linux.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion ...mpilercuda-nvcccuda_compiler_version12.0cxx_compiler_version12python3.10.____cpython.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ cxx_compiler:
cxx_compiler_version:
- '12'
docker_image:
- quay.io/condaforge/linux-anvil-aarch64
- quay.io/condaforge/linux-anvil-cos7-x86_64
pin_run_as_build:
python:
min_pin: x.x
Expand Down
2 changes: 1 addition & 1 deletion ...mpilercuda-nvcccuda_compiler_version12.0cxx_compiler_version12python3.11.____cpython.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ cxx_compiler:
cxx_compiler_version:
- '12'
docker_image:
- quay.io/condaforge/linux-anvil-aarch64
- quay.io/condaforge/linux-anvil-cos7-x86_64
pin_run_as_build:
python:
min_pin: x.x
Expand Down
2 changes: 1 addition & 1 deletion ...ompilercuda-nvcccuda_compiler_version12.0cxx_compiler_version12python3.8.____cpython.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ cxx_compiler:
cxx_compiler_version:
- '12'
docker_image:
- quay.io/condaforge/linux-anvil-aarch64
- quay.io/condaforge/linux-anvil-cos7-x86_64
pin_run_as_build:
python:
min_pin: x.x
Expand Down
2 changes: 1 addition & 1 deletion ...ompilercuda-nvcccuda_compiler_version12.0cxx_compiler_version12python3.9.____cpython.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ cxx_compiler:
cxx_compiler_version:
- '12'
docker_image:
- quay.io/condaforge/linux-anvil-aarch64
- quay.io/condaforge/linux-anvil-cos7-x86_64
pin_run_as_build:
python:
min_pin: x.x
Expand Down
2 changes: 1 addition & 1 deletion ...mpilercuda-nvcccuda_compiler_version12.0cxx_compiler_version12python3.10.____cpython.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ cxx_compiler:
cxx_compiler_version:
- '12'
docker_image:
- quay.io/condaforge/linux-anvil-ppc64le
- quay.io/condaforge/linux-anvil-cos7-x86_64
pin_run_as_build:
python:
min_pin: x.x
Expand Down
2 changes: 1 addition & 1 deletion ...mpilercuda-nvcccuda_compiler_version12.0cxx_compiler_version12python3.11.____cpython.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ cxx_compiler:
cxx_compiler_version:
- '12'
docker_image:
- quay.io/condaforge/linux-anvil-ppc64le
- quay.io/condaforge/linux-anvil-cos7-x86_64
pin_run_as_build:
python:
min_pin: x.x
Expand Down
2 changes: 1 addition & 1 deletion ...ompilercuda-nvcccuda_compiler_version12.0cxx_compiler_version12python3.8.____cpython.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ cxx_compiler:
cxx_compiler_version:
- '12'
docker_image:
- quay.io/condaforge/linux-anvil-ppc64le
- quay.io/condaforge/linux-anvil-cos7-x86_64
pin_run_as_build:
python:
min_pin: x.x
Expand Down
2 changes: 1 addition & 1 deletion ...ompilercuda-nvcccuda_compiler_version12.0cxx_compiler_version12python3.9.____cpython.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ cxx_compiler:
cxx_compiler_version:
- '12'
docker_image:
- quay.io/condaforge/linux-anvil-ppc64le
- quay.io/condaforge/linux-anvil-cos7-x86_64
pin_run_as_build:
python:
min_pin: x.x
Expand Down
6 changes: 3 additions & 3 deletions .scripts/build_steps.sh
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions .scripts/run_win_build.bat
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions conda-forge.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
bot:
abi_migration_branches:
- 11.8
build_platform:
linux_aarch64: linux_64
linux_ppc64le: linux_64
conda_build:
pkg_format: '2'
conda_forge_output_validation: true
Expand Down
24 changes: 13 additions & 11 deletions recipe/meta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,17 +17,21 @@ source:
sha256: 7df0f84c4b6210112353374a7d992bfd5bba9c97358b39ea03b61bd634ce9c8a

build:
number: 3
number: 4
# Cython 0.29.x seems not to be fully compatible with Python 3.12, skipping for now.
# This can be removed in a later CUDA Python that supports Cython 3.x.
# xref: https://github.com/conda-forge/cuda-python-feedstock/issues/57
skip: true # [osx or cuda_compiler_version != "12.0" or py == 312]
skip: true # [py == 312]
{% if not (environ.get("cuda_compiler_version")|string()).startswith(major_version|string()) %}
skip: true
{% endif %}
script:
- set "INCLUDE=%INCLUDE%;%BUILD_PREFIX%\Library\include" # [win64]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Windows, the builds ran into an issue finding the crt headers on CI:

 building 'cuda._lib.ccudart.ccudart' extension
  creating build\temp.win-amd64-cpython-311\Release\cuda\_lib\ccudart
  "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -I.\cuda -ID:\bld\cuda-python_1702961331834\_h_env -ID:\bld\cuda-python_1702961331834\_h_env\Library/include -ID:\bld\cuda-python_1702961331834\_h_env\include -ID:\bld\cuda-python_1702961331834\_h_env\Include "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.29.30133\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.29.30133\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" -ID:\bld\cuda-python_1702961331834\_h_env\Library\include -ID:\bld\cuda-python_1702961331834\_h_env\Library\include -ID:\bld\cuda-python_1702961331834\_h_env\Library\include\targets\x64 /d1trimfile:D:\bld\cuda-python_1702961331834\work /EHsc /Tpcuda/_lib/ccudart\ccudart.cpp /Fobuild\temp.win-amd64-cpython-311\Release\cuda/_lib/ccudart\ccudart.obj
  ccudart.cpp
  D:\bld\cuda-python_1702961331834\_h_env\Library/include\vector_types.h(65): fatal error C1083: Cannot open include file: 'crt/host_defines.h': No such file or directory
  error: command 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX64\\x64\\cl.exe' failed with exit code 2
  error: subprocess-exited-with-error

The crt headers are part of the compiler currently. This will be broken out as their own package in CUDA 12.2 ( conda-forge/cuda-nvcc-impl-feedstock#4 )

For now this adds the NVCC includes to the end of INCLUDE so they are searched as well (after exhausting all other search paths)

This fixes the Windows build issue with crt headers

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracking this in issue: #74

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Attempting to address this with PR: #75

- {{ PYTHON }} -m pip install . --no-deps -vv
script_env:
# Ensure that CUDA includes can be found by the host compiler
- CUDA_HOME="${BUILD_PREFIX}/targets/{{ target_name }}" # [linux]
- CUDA_HOME=%BUILD_PREFIX%\Library # [win64]
- CUDA_HOME="${PREFIX}/targets/{{ target_name }}" # [linux]
- CUDA_HOME=%PREFIX%\Library # [win64]
run_exports:
- {{ pin_subpackage('cuda-python', min_pin='x', max_pin='x') }}
ignore_run_exports_from:
Expand All @@ -38,13 +42,11 @@ requirements:
- {{ compiler('c') }}
- {{ compiler('cxx') }}
- {{ compiler('cuda') }}
# The packages with CUDA headers need to be added to "build" so that
# cuda-python can find and parse the headers. The parsing code in
# cuda-python doesn't support splayed layouts, i.e. packages split between
# "build" and "host", because $CUDA_HOME can only point to build or host.
- cuda-nvrtc-dev
- cuda-profiler-api
- cuda-version {{ major_version }}.{{ minor_version }}
- sysroot_{{ target_platform }} 2.17 # [linux]
- cross-python_{{ target_platform }} # [build_platform != target_platform]
- python # [build_platform != target_platform]
- cython <3 # [build_platform != target_platform]
host:
- cuda-cudart-dev
- cuda-nvrtc-dev
Expand All @@ -54,7 +56,7 @@ requirements:
- pyclibrary
- python
- setuptools
- cuda-version {{ major_version }}
- cuda-version {{ major_version }}.{{ minor_version }}
run:
- python
- {{ pin_compatible('cuda-version', min_pin='x', max_pin='x') }}
Expand Down