Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update for CUDA 12.2.2 #8

Merged
merged 14 commits into from
Jan 5, 2024
Merged

Update for CUDA 12.2.2 #8

Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
517c215
Update for CUDA 12.2.2
adibbley Dec 18, 2023
454ce06
Rename to cuda-crt-dev_{{ target_platform }}
adibbley Dec 19, 2023
22368b8
Apply suggestions from code review
adibbley Dec 19, 2023
586782a
Move bin test
adibbley Dec 19, 2023
dc54e67
Split nvvm outputs
adibbley Dec 19, 2023
94a6ca2
Linux only
adibbley Dec 19, 2023
aa268fb
Update nvvm split for windows
adibbley Dec 19, 2023
faaf654
Update crt summary
adibbley Dec 19, 2023
c2b6229
Update crt-tools description
adibbley Dec 20, 2023
22ecea9
Add cuda_version_next variable
adibbley Dec 20, 2023
b1044b4
Update recipe/meta.yaml
adibbley Dec 20, 2023
047d00d
Rename cuda_version_next_major
adibbley Dec 20, 2023
ab93a60
Use string
adibbley Dec 20, 2023
3f5e527
Use pin_subpackage
adibbley Dec 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
200 changes: 175 additions & 25 deletions recipe/meta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
{% set name = "cuda-nvcc-impl" %}
{% set version = "12.1.105" %}
{% set cuda_version = "12.1" %}
{% set version = "12.2.140" %}
{% set cuda_version = "12.2" %}
adibbley marked this conversation as resolved.
Show resolved Hide resolved
{% set cuda_version_next_major = (cuda_version.split(".")[0]|int + 1)|string + ".0a0" %}
{% set platform = "linux-x86_64" %} # [linux64]
{% set platform = "linux-ppc64le" %} # [ppc64le]
{% set platform = "linux-sbsa" %} # [aarch64]
Expand All @@ -18,10 +19,10 @@ package:

source:
url: https://developer.download.nvidia.com/compute/cuda/redist/cuda_nvcc/{{ platform }}/cuda_nvcc-{{ platform }}-{{ version }}-archive.{{ extension }}
sha256: 0b85f7eee17788abbd170b0b493c74ce2e9fd5a9604461b99c2c378165e1083b # [linux64]
sha256: e6e2c4c80e44409cd9006c6f1fd56d7c48c4a8bcd2d130f9405bcb7af16660a0 # [ppc64le]
sha256: 6e795ec791241e9320ec300657408cbfafbe7e79ceda0da46522cc85ced358f4 # [aarch64]
sha256: 3b452d704c92e8af6d59247ee842b2ca8c8e37dd04c015eea59c74c265672074 # [win]
sha256: 3feaab191e1ae3fded02c02b0f24453069ada96b2e3a750251293b36079dc2f1 # [linux64]
sha256: cc515b95932445624a21bd179256dd7211c02000f543cc68fee6a99d735f137c # [ppc64le]
sha256: 3e9df10648af47cd4b1b985be81da9e4706623b6dd6f640e53f6d8badd84c155 # [aarch64]
sha256: 140ca9c560f2fa4f6f7275d815d1d6fa6bbf08901a9bae48152f6c5e50c0bb1c # [win]
patches:
- nvcc.profile.patch # [linux]
- nvcc.profile.patch.win # [win]
Expand All @@ -33,9 +34,7 @@ build:
outputs:
- name: cuda-nvcc-tools
files: # [linux]
- bin # [linux]
- nvvm/bin # [linux]
- nvvm/libdevice # [linux]
- bin/* # [linux]
jakirkham marked this conversation as resolved.
Show resolved Hide resolved
requirements:
build:
- {{ compiler("c") }}
Expand All @@ -44,17 +43,19 @@ outputs:
host:
- arm-variant * {{ arm_variant_type }} # [aarch64]
- cuda-version {{ cuda_version }}
- {{ pin_subpackage("cuda-crt-tools", exact=True) }}
- {{ pin_subpackage("cuda-nvvm-tools", exact=True) }}
Comment on lines +46 to +47
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need these in host? I would have expected them to only be run requirements, especially since they only contain binaries.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The files constrain above is grabbing bin/*. It may make sense to include these just to avoid repackaging duplicate files that either of these packages would already have

Admittedly bin/* may be too sweeping. Though it does ensure that we have gotten everything else that was in bin that is not included in any other package

run:
- {{ pin_compatible("cuda-version", max_pin="x.x") }}
- {{ pin_subpackage("cuda-crt-tools", exact=True) }}
- {{ pin_subpackage("cuda-nvvm-tools", exact=True) }}
run_constrained:
- gcc_impl_{{ target_platform }} >=6,<13 # [linux]
- arm-variant * {{ arm_variant_type }} # [aarch64]
test:
commands:
- test -f $PREFIX/bin/nvcc # [linux]
- test -f $PREFIX/bin/ptxas # [linux]
- test -d $PREFIX/nvvm # [linux]
- test -f $PREFIX/nvvm/bin/cicc # [linux]
about:
home: https://developer.nvidia.com/cuda-toolkit
license_file: LICENSE
Expand All @@ -72,18 +73,14 @@ outputs:
- "*/api-ms-win-core-winrt-*.dll" # [win]
run_exports:
strong:
- cuda-version >={{ cuda_version }},<{{ cuda_version.split(".")[0]|int + 1 }}
- cuda-version >={{ cuda_version }},<{{ cuda_version_next_major }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe put this whole constraint into a variable? It gets used numerous times.

files:
# targets/{{ target_name }}/bin/nvcc is only a symlink
- targets/{{ target_name }}/bin # [linux]
- targets/{{ target_name }}/nvvm # [linux]
- targets/{{ target_name }}/include # [linux]
- targets/{{ target_name }}/lib # [linux]
- Library\bin\crt # [win]
- targets/{{ target_name }}/bin # [linux]
- targets/{{ target_name }}/include/*.h # [linux]
- targets/{{ target_name }}/lib # [linux]
- Library\bin\*.exe # [win]
- Library\bin\nvcc.profile # [win]
- Library\nvvm # [win]
- Library\include\crt # [win]
- Library\include\fatbinary_section.h # [win]
- Library\include\nvPTXCompiler.h # [win]
- Library\lib\nvptxcompiler_static.lib # [win]
Expand All @@ -94,25 +91,27 @@ outputs:
- arm-variant * {{ arm_variant_type }} # [aarch64]
- cuda-version {{ cuda_version }}
- cuda-nvcc-tools {{ cuda_version }}
- {{ pin_subpackage("cuda-crt-dev_" + target_platform, exact=True) }}
- {{ pin_subpackage("cuda-nvvm-dev_" + target_platform, exact=True) }}
Comment on lines +94 to +95
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question as above re: why this is needed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As these are -dev packages, they would make sense to keep (especially if there is something linking to them)

- libgcc-ng >=6 # [linux]
run:
- {{ pin_compatible("cuda-version", max_pin="x.x") }}
- {{ pin_subpackage("cuda-crt-dev_" + target_platform, exact=True) }}
- {{ pin_subpackage("cuda-nvvm-dev_" + target_platform, exact=True) }}
- libgcc-ng >=6 # [linux]
run_constrained:
- gcc_impl_{{ target_platform }} >=6,<13 # [linux]
- arm-variant * {{ arm_variant_type }} # [aarch64]
test:
commands:
- test -L $PREFIX/targets/{{ target_name }}/bin/nvcc # [linux]
- test -L $PREFIX/targets/{{ target_name }}/nvvm # [linux]
- test -f $PREFIX/targets/{{ target_name }}/lib/libnvptxcompiler_static.a # [linux]
- test -f $PREFIX/targets/{{ target_name }}/include/fatbinary_section.h # [linux]
adibbley marked this conversation as resolved.
Show resolved Hide resolved
- test -f $PREFIX/targets/{{ target_name }}/include/nvPTXCompiler.h # [linux]
- if not exist %LIBRARY_BIN%\crt\link.stub exit 1 # [win]
- if not exist %LIBRARY_BIN%\nvcc.exe exit 1 # [win]
- if not exist %LIBRARY_BIN%\nvcc.profile exit 1 # [win]
- if not exist %LIBRARY_LIB%\nvptxcompiler_static.lib exit 1 # [win]
- if not exist %LIBRARY_INC%\nvPTXCompiler.h exit 1 # [win]
- if not exist %LIBRARY_INC%\crt\common_functions.h exit 1 # [win]
about:
home: https://developer.nvidia.com/cuda-toolkit
license_file: LICENSE
Expand All @@ -129,21 +128,21 @@ outputs:
binary_relocation: False
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably ok to keep this. Though quick question, do we have any libraries still here to corrupt? Only see the static library below, which I think patchelf shouldn't touch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed this isn't doing anything now. Left it in since the relocation only seems to break things for us.

run_exports:
strong:
- cuda-version >={{ cuda_version }},<{{ cuda_version.split(".")[0]|int + 1 }}
- cuda-version >={{ cuda_version }},<{{ cuda_version_next_major }}
files: # [linux]
- lib/libnvptxcompiler_static.a # [linux]
- nvvm/include # [linux]
- nvvm/lib64 # [linux]
requirements:
build:
- sysroot_{{ target_platform }} 2.17 # [linux]
host:
- arm-variant * {{ arm_variant_type }} # [aarch64]
- cuda-version {{ cuda_version }}
- cuda-cudart-dev
- {{ pin_subpackage("cuda-nvvm-impl", exact=True) }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more possibly unnecessary host (just marking to make them easier to find in case they can be removed).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we are only packaging one static library here that shouldn't be packaged anywhere else, this is probably safe to drop.

run:
- {{ pin_subpackage("cuda-nvcc-tools", exact=True) }}
- {{ pin_subpackage("cuda-nvcc-dev_" + target_platform, exact=True) }}
- {{ pin_subpackage("cuda-nvvm-impl", exact=True) }}
- {{ pin_compatible("cuda-version", max_pin="x.x") }}
- cuda-cudart-dev
run_constrained:
Expand Down Expand Up @@ -181,6 +180,157 @@ outputs:
Compiler for CUDA applications.
doc_url: https://docs.nvidia.com/cuda/index.html

- name: cuda-crt-tools
files:
- bin/crt # [linux]
- Library\bin\crt # [win]
requirements:
host:
- arm-variant * {{ arm_variant_type }} # [aarch64]
- cuda-version {{ cuda_version }}
run:
robertmaynard marked this conversation as resolved.
Show resolved Hide resolved
- {{ pin_compatible("cuda-version", max_pin="x.x") }}
run_constrained:
- arm-variant * {{ arm_variant_type }} # [aarch64]
test:
commands:
- test -d $PREFIX/bin/crt # [linux]
- test -f $PREFIX/bin/crt/link.stub # [linux]
- if not exist %LIBRARY_BIN%\crt\link.stub exit 1 # [win]
about:
Copy link
Contributor

@vyasr vyasr Dec 20, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not at all required for this PR, but does conda support YAML anchors? If so, we should use them for the about rather than duplicating it once per output in these recipes. We could use an anchor override for the description/summary, which are the only fields that change. If we were to apply it, would recommend doing so to any of our packages with multiple outputs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's an example: https://www.linode.com/docs/guides/yaml-anchors-aliases-overrides-extensions/

home: https://developer.nvidia.com/cuda-toolkit
license_file: LICENSE
license: LicenseRef-NVIDIA-End-User-License-Agreement
license_url: https://docs.nvidia.com/cuda/eula/index.html
summary: CUDA internal tools.
description: |
CUDA internal tools.
doc_url: https://docs.nvidia.com/cuda/index.html

- name: cuda-crt-dev_{{ target_platform }}
build:
noarch: generic
run_exports:
strong:
- cuda-version >={{ cuda_version }},<{{ cuda_version_next_major }}
files:
- targets/{{ target_name }}/include/crt # [linux]
- Library\include\crt # [win]
requirements:
host:
- arm-variant * {{ arm_variant_type }} # [aarch64]
- cuda-version {{ cuda_version }}
run:
- {{ pin_compatible("cuda-version", max_pin="x.x") }}
run_constrained:
- arm-variant * {{ arm_variant_type }} # [aarch64]
test:
commands:
- test -f $PREFIX/targets/{{ target_name }}/include/crt/common_functions.h # [linux]
- if not exist %LIBRARY_INC%\crt\common_functions.h exit 1 # [win]
about:
home: https://developer.nvidia.com/cuda-toolkit
license_file: LICENSE
license: LicenseRef-NVIDIA-End-User-License-Agreement
license_url: https://docs.nvidia.com/cuda/eula/index.html
summary: CUDA internal headers.
description: |
CUDA internal headers.
doc_url: https://docs.nvidia.com/cuda/index.html

- name: cuda-nvvm-tools
files:
- nvvm/bin # [linux]
- nvvm/libdevice # [linux]
- Library\nvvm\bin # [win]
- Library\nvvm\libdevice # [win]
requirements:
build:
- {{ compiler("c") }}
- {{ compiler("cxx") }}
host:
- arm-variant * {{ arm_variant_type }} # [aarch64]
- cuda-version {{ cuda_version }}
run:
- {{ pin_compatible("cuda-version", max_pin="x.x") }}
run_constrained:
- arm-variant * {{ arm_variant_type }} # [aarch64]
test:
commands:
- test -d $PREFIX/nvvm # [linux]
- test -f $PREFIX/nvvm/bin/cicc # [linux]
- if not exist %LIBRARY_PREFIX%\nvvm\bin\cicc.exe exit 1 # [win]
- if not exist %LIBRARY_PREFIX%\nvvm\libdevice exit 1 # [win]
about:
home: https://developer.nvidia.com/cuda-toolkit
license_file: LICENSE
license: LicenseRef-NVIDIA-End-User-License-Agreement
license_url: https://docs.nvidia.com/cuda/eula/index.html
summary: Compiler for CUDA applications.
description: |
Compiler for CUDA applications.
doc_url: https://docs.nvidia.com/cuda/index.html

- name: cuda-nvvm-dev_{{ target_platform }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be noarch: generic just like cuda-crt-dev_{{ target_platform }} is

It is needed because cuda-nvcc-dev_{{ target_platform }} is noarch: generic and depends on both of these

Without noarch: generic the compiler becomes uninstallable in cross-compilation workloads

Suggested change
- name: cuda-nvvm-dev_{{ target_platform }}
- name: cuda-nvvm-dev_{{ target_platform }}
build:
noarch: generic

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixing in PR ( #12 ), which has more context

files: # [linux]
- targets/{{ target_name }}/nvvm # [linux]
requirements:
host:
- arm-variant * {{ arm_variant_type }} # [aarch64]
- cuda-version {{ cuda_version }}
run:
- {{ pin_compatible("cuda-version", max_pin="x.x") }}
run_constrained:
- arm-variant * {{ arm_variant_type }} # [aarch64]
test:
commands:
- test -L $PREFIX/targets/{{ target_name }}/nvvm # [linux]
about:
home: https://developer.nvidia.com/cuda-toolkit
license_file: LICENSE
license: LicenseRef-NVIDIA-End-User-License-Agreement
license_url: https://docs.nvidia.com/cuda/eula/index.html
summary: Compiler for CUDA applications.
description: |
Compiler for CUDA applications.
doc_url: https://docs.nvidia.com/cuda/index.html

- name: cuda-nvvm-impl
build:
# libnvvm.so gets corrupted by patchelf. No need to relocate as it is already relocatable.
binary_relocation: False
files:
- nvvm/include # [linux]
- nvvm/lib64 # [linux]
- Library\nvvm\include # [win]
- Library\nvvm\lib # [win]
requirements:
build:
- {{ compiler("c") }}
- {{ compiler("cxx") }}
host:
- arm-variant * {{ arm_variant_type }} # [aarch64]
- cuda-version {{ cuda_version }}
run:
- {{ pin_compatible("cuda-version", max_pin="x.x") }}
run_constrained:
- arm-variant * {{ arm_variant_type }} # [aarch64]
test:
commands:
- test -d $PREFIX/nvvm/include # [linux]
- test -d $PREFIX/nvvm/lib64 # [linux]
- if not exist %LIBRARY_PREFIX%\nvvm\include exit 1 # [win]
- if not exist %LIBRARY_PREFIX%\nvvm\lib exit 1 # [win]
about:
home: https://developer.nvidia.com/cuda-toolkit
license_file: LICENSE
license: LicenseRef-NVIDIA-End-User-License-Agreement
license_url: https://docs.nvidia.com/cuda/eula/index.html
summary: Compiler for CUDA applications.
description: |
Compiler for CUDA applications.
doc_url: https://docs.nvidia.com/cuda/index.html

about:
home: https://developer.nvidia.com/cuda-toolkit
license_file: LICENSE
Expand Down