Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shim package to interact with an **existing** NVCC install #8229

Merged
merged 20 commits into from
Aug 16, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions recipes/nvcc/LICENSE.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:

* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of NVIDIA CORPORATION nor the names of its
contributors may be used to endorse or promote products derived
from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
62 changes: 62 additions & 0 deletions recipes/nvcc/install_nvcc.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
#!/bin/bash

set -xeuo pipefail

# Default to using `nvcc` to specify `CUDA_HOME`.
if [ -z ${CUDA_HOME+x} ]
then
CUDA_HOME="$(dirname $(dirname $(which nvcc)))"
fi

# Set `CUDA_HOME` in an activation script.
mkdir -p "${PREFIX}/etc/conda/activate.d"
cat > "${PREFIX}/etc/conda/activate.d/${PKG_NAME}_activate.sh" <<EOF
#!/bin/bash

if [[ ! -d "${CUDA_HOME}" ]]
then
echo "Cannot find: ${CUDA_HOME}"
exit 1
fi
if [[ ! -f "${CUDA_HOME}/lib64/stubs/libcuda.so" ]]
then
echo "Cannot find: ${CUDA_HOME}/lib64/stubs/libcuda.so"
exit 1
fi
grep "CUDA Version ${PKG_VERSION}" ${CUDA_HOME}/version.txt &>/dev/null
if [[ $? -ne 0 ]]
then
echo "Version of installed CUDA didn't match package"
exit 1
fi

export CUDA_HOME="${CUDA_HOME}"
export CFLAGS="\${CFLAGS} -I\${CUDA_HOME}/include"
export CPPFLAGS="\${CPPFLAGS} -I\${CUDA_HOME}/include"
export CXXFLAGS="\${CXXFLAGS} -I\${CUDA_HOME}/include"
EOF

# Unset `CUDA_HOME` in a deactivation script.
mkdir -p "${PREFIX}/etc/conda/deactivate.d"
cat > "${PREFIX}/etc/conda/deactivate.d/${PKG_NAME}_deactivate.sh" <<EOF
#!/bin/bash
unset CUDA_HOME
unset CFLAGS
unset CPPFLAGS
unset CXXFLAGS
EOF

# Create `nvcc` script in `bin` so it can be easily run.
mkdir -p "${PREFIX}/bin"
cat > "${PREFIX}/bin/nvcc" <<'EOF'
#!/bin/bash
"${CUDA_HOME}/bin/nvcc" -ccbin "${CXX}" $@
EOF
chmod +x "${PREFIX}/bin/nvcc"

# Add `libcuda.so` shared object stub to the compiler sysroot.
# Needed for things that want to link to `libcuda.so`.
# Stub is used to avoid getting driver code linked into binaries.
CONDA_BUILD_SYSROOT="$(${CC} --print-sysroot)"
mkdir -p "${CONDA_BUILD_SYSROOT}/lib"
ln -s "${CUDA_HOME}/lib64/stubs/libcuda.so" "${CONDA_BUILD_SYSROOT}/lib/libcuda.so"
65 changes: 65 additions & 0 deletions recipes/nvcc/meta.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
{% set name = "nvcc" %}
{% set version = environ.get("CUDA_VER", "9.2") %}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a recipe/conda_build_config.yaml with,

CUDA_VER:
  - 9.2
  - 10.0

and use the value like set version = CUDA_VER

Copy link
Member Author

@jakirkham jakirkham Aug 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 9.2 is really only here to make the linter happy. Personally I'd rather not have it here at all if it could be avoided.

Basically what this is doing is picking up CUDA_VER from the docker image.


package:
name: "{{ name }}"

build:
number: 0
skip: true # [not linux]

outputs:
- name: "{{ name }}_{{ target_platform }}"
version: "{{ version }}"
script: install_nvcc.sh
build:
script_env:
- CUDA_HOME
ignore_run_exports:
- libgcc-ng
run_exports:
strong:
- cudatoolkit {{ version }}|{{ version }}.*
Copy link
Member

@isuruf isuruf Aug 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need the run_exports of the C++ compiler runtime here as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? If the user is using the C++ compiler, that will be added by this anyways.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But the runtimes like libgcc-ng and libstdcxx-ng will not be added. run_exports is not transitive.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think users should be relying on nvcc to add these things. If they are using a C or C++ compiler, they should list those requirements explicitly in their recipe.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then the C and C++ compiler should not be in run requirements. The way the compiler package is set up is that if the user tries to use a C++ compiler in the docker image, it will not be found if it is not an explicit requirement. Having the explicit requirement means that the run_exports for libgcc-ng is added.
But now, if the user adds nvcc_linux-64, they get the C++ compiler, but no run_exports for libgcc-ng.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense. Have dropped the C and C++ compiler from run. They won't be installed now. The test has been updated to list C and C++ as requirements of the test.

requirements:
host:
# Needed to symlink libcuda into sysroot libs.
- {{ compiler("c") }}
run:
test:
requires:
- {{ compiler("c") }}
# Host code is forwarded to a C++ compiler
- {{ compiler("cxx") }}
files:
- test.cu
commands:
# Verify the symlink to the libcuda stub library exists.
- test -f "$(${CC} --print-sysroot)/lib/libcuda.so"
# Verify the activation scripts are in-place.
{% for state in ["activate", "deactivate"] %}
- test -f "${PREFIX}/etc/conda/{{ state }}.d/{{ PKG_NAME }}_{{ state }}.sh"
{% endfor %}
# Try using the activation scripts.
- if [ -z ${CUDA_HOME+x} ]; then echo "CUDA_HOME is unset after activation" && exit 1; else echo "CUDA_HOME is set to '$CUDA_HOME'"; fi
- source ${PREFIX}/etc/conda/deactivate.d/{{ PKG_NAME }}_deactivate.sh
- if [ -z ${CUDA_HOME+x} ]; then echo "CUDA_HOME is unset after deactivation "; else echo "CUDA_HOME is set to '$CUDA_HOME' after deactivation" && exit 1; fi
- source ${PREFIX}/etc/conda/activate.d/{{ PKG_NAME }}_activate.sh
- if [ -z ${CUDA_HOME+x} ]; then echo "CUDA_HOME is unset after activation" && exit 1; else echo "CUDA_HOME is set to '$CUDA_HOME'"; fi
# Try building something.
- nvcc test.cu
about:
home: https://github.com/conda-forge/nvcc-feedstock
license: BSD 3-Clause
license_file: LICENSE.txt
summary: A meta-package to enable the right nvcc.

jakirkham marked this conversation as resolved.
Show resolved Hide resolved
about:
home: https://github.com/conda-forge/nvcc-feedstock
license: BSD 3-Clause
license_file: LICENSE.txt
summary: A meta-package to enable the right nvcc.

extra:
recipe-maintainers:
- isuruf
- jakirkham
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add me as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure would be happy to have your help here. 🙂 Added you in the last commit.

3 changes: 3 additions & 0 deletions recipes/nvcc/test.cu
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
int main() {
return 0;
}