Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cross compile cuda support #209

Closed
wants to merge 3 commits into from
Closed

cross compile cuda support #209

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8929cd9
cross compile cuda support
isuruf Oct 31, 2022
0d2cb01
check that we are in cos8
isuruf Oct 31, 2022
06f32ab
check that we are in cos7/8
isuruf Oct 31, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 52 additions & 3 deletions recipe/cross_compile_support.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@ BUILD_PLATFORM=$(conda info --json | jq -r .platform)

if [ -f ${CI_SUPPORT}/${CONFIG}.yaml ]; then
HOST_PLATFORM=$(cat ${CI_SUPPORT}/${CONFIG}.yaml | shyaml get-value target_platform.0 ${BUILD_PLATFORM})
CUDA_COMPILER_VERSION=$(cat ${CI_SUPPORT}/${CONFIG}.yaml | shyaml get-value cuda_compiler_version.0 None)
fi

HOST_PLATFORM=${HOST_PLATFORM:-${BUILD_PLATFORM}}
CUDA_COMPILER_VERSION=${CUDA_COMPILER_VERSION:-None}

if [[ "${HOST_PLATFORM}" != "${BUILD_PLATFORM}" ]]; then
echo "export CONDA_BUILD_CROSS_COMPILATION=1" >> "${CONDA_PREFIX}/etc/conda/activate.d/conda-forge-ci-setup-activate.sh"
Expand All @@ -15,19 +17,66 @@ if [[ "${HOST_PLATFORM}" != "${BUILD_PLATFORM}" ]]; then
fi
if [[ "${BUILD_PLATFORM}" == "linux-64" && "${HOST_PLATFORM}" == linux-* ]]; then
mamba create -n sysroot_${HOST_PLATFORM} --yes --quiet sysroot_${HOST_PLATFORM}
HOST_PLATFORM_ARCH=${HOST_PLATFORM:6}
if [[ -f ${CI_SUPPORT}/../recipe/yum_requirements.txt ]]; then
for pkg in $(cat ${CI_SUPPORT}/../recipe/yum_requirements.txt); do
if [[ "${pkg}" != "#"* && "${pkg}" != "" ]]; then
mamba install "${pkg}-cos7-${HOST_PLATFORM:6}" -n sysroot_${HOST_PLATFORM} --yes --quiet || true
mamba install "${pkg}-cos7-${HOST_PLATFORM_ARCH}" -n sysroot_${HOST_PLATFORM} --yes --quiet || true
fi
done
fi
export QEMU_LD_PREFIX=$(find ${CONDA_PREFIX}/envs/sysroot_${HOST_PLATFORM} -name sysroot | head -1)
if [ -f ${CI_SUPPORT}/${CONFIG}.yaml ]; then
echo "CMAKE_CROSSCOMPILING_EMULATOR: " >> ${CI_SUPPORT}/${CONFIG}.yaml
echo "- /usr/bin/qemu-$(echo $HOST_PLATFORM | cut -b 7-)-static" >> ${CI_SUPPORT}/${CONFIG}.yaml
echo "- /usr/bin/qemu-${HOST_PLATFORM_ARCH}-static" >> ${CI_SUPPORT}/${CONFIG}.yaml
echo "CROSSCOMPILING_EMULATOR: " >> ${CI_SUPPORT}/${CONFIG}.yaml
echo "- /usr/bin/qemu-$(echo $HOST_PLATFORM | cut -b 7-)-static" >> ${CI_SUPPORT}/${CONFIG}.yaml
echo "- /usr/bin/qemu-${HOST_PLATFORM_ARCH}-static" >> ${CI_SUPPORT}/${CONFIG}.yaml
fi


if [[ "${CUDA_COMPILER_VERSION}" == "11.2" ]]; then
EXTRACT_DIR=$(mktemp -d)
pushd ${EXTRACT_DIR}
if [[ "${HOST_PLATFORM_ARCH}" == "aarch64" ]]; then
CUDA_HOST_PLATFORM_ARCH=sbsa
else
CUDA_HOST_PLATFORM_ARCH=${HOST_PLATFORM_ARCH}
isuruf marked this conversation as resolved.
Show resolved Hide resolved
fi
for f in cuda-cudart-11-2-11.2.72-1 \
cuda-cudart-devel-11-2-11.2.72-1\
cuda-cupti-11-2-11.2.152-1 \
cuda-driver-devel-11-2-11.2.152-1 \
cuda-nvcc-11-2-11.2.152-1 \
cuda-nvml-devel-11-2-11.2.67-1 \
cuda-nvprof-11-2-11.2.67-1 \
cuda-nvrtc-11-2-11.2.152-1 \
cuda-nvrtc-devel-11-2-11.2.152-1 \
cuda-nvtx-11-2-11.2.67-1 \
libcublas-11-2-11.3.1.68-1 \
libcublas-devel-11-2-11.3.1.68-1 \
libcufft-11-2-10.4.1.152-1 \
libcufft-devel-11-2-10.4.1.152-1 \
libcurand-11-2-10.2.3.152-1 \
libcurand-devel-11-2-10.2.3.152-1 \
libcusolver-11-2-11.1.0.152-1 \
libcusolver-devel-11-2-11.1.0.152-1 \
libcusparse-11-2-11.4.1.1152-1 \
libcusparse-devel-11-2-11.4.1.1152-1 \
libnpp-11-2-11.2.1.68-1 \
libnpp-devel-11-2-11.2.1.68-1 \
libnvjpeg-11-2-11.4.0.152-1 \
libnvjpeg-devel-11-2-11.4.0.152-1 ; do
Comment on lines +49 to +72
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of hardcoding all of these, there's a json file in the repo that contains all of these that we could parse, for example here: https://developer.download.nvidia.com/compute/cuda/repos/rhel8/sbsa/version_11.2.2.json

It looks like we're getting the CUDA 11.2.0 versions of these packages as opposed to the 11.2.2 versions as well.

curl -L -O https://developer.download.nvidia.com/compute/cuda/repos/rhel8/${CUDA_HOST_PLATFORM_ARCH}/${f}.${HOST_PLATFORM_ARCH}.rpm
kkraus14 marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we don't need the cross-linux packages and are okay just grabbing the native packages?

bsdtar -xvf ${f}.${HOST_PLATFORM_ARCH}.rpm
done
mv ./usr/local/cuda-${CUDA_COMPILER_VERSION}/targets/${CUDA_HOST_PLATFORM_ARCH}-linux ${CUDA_HOME}/targets/${CUDA_HOST_PLATFORM_ARCH}-linux
popd
rm -rf ${EXTRACT_DIR}
elif [[ "${CUDA_COMPILER_VERSION}" != "None" ]]; then
# FIXME: can use anaconda.org/nvidia packages to get the includes and libs
# for cuda >=11.3.
echo "cross compiling with cuda != 11.2 not supported yet"
exit 1
fi
fi
fi
2 changes: 1 addition & 1 deletion recipe/meta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{% set name = "conda-forge-ci-setup" %}
{% set version = "3.21.0" %}
{% set version = "3.22.0" %}
{% set build = 0 %}
{% set cuda_compiler_version = cuda_compiler_version or "None" %}
{% if cuda_compiler_version == "None" %}
Expand Down