Re-enable builds on OSX64 when it no longer segfaults on import deepspeed
#45
Labels
Comments
weiji14
added a commit
to regro-cf-autotick-bot/deepspeed-feedstock
that referenced
this issue
Dec 16, 2023
To prevent segfault. See conda-forge#45
weiji14
added a commit
that referenced
this issue
Dec 16, 2023
* Rebuild for CUDA 12 w/arch + Windows support The transition to CUDA 12 SDK includes new packages for all CUDA libraries and build tools. Notably, the cudatoolkit package no longer exists, and packages should depend directly on the specific CUDA libraries (libcublas, libcusolver, etc) as needed. For an in-depth overview of the changes and to report problems [see this issue]( conda-forge/conda-forge.github.io#1963 ). Please feel free to raise any issues encountered there. Thank you! 🙏 * MNT: Re-rendered with conda-build 3.27.0, conda-smithy 3.30.1, and conda-forge-pinning 2023.11.29.08.31.37 * Add TORCH_CUDA_ARCH_LIST for CUDA 12.0 Xref conda-forge/pytorch-cpu-feedstock@c8a0f99. Also remove if-then statements for CUDA 10.x, 11.0 and 11.1 * Bump build number to 2 * Add libcublas-dev, libcusolver-dev, libcusparse-dev to build/host deps Needed to compile deepspeed on CUDA 12.0 in conda-forge. * Add cuda-profiler-api, libcurand-dev to build/host deps * Disable builds on OSX64 To prevent segfault. See #45 * MNT: Re-rendered with conda-build 3.28.1, conda-smithy 3.30.1, and conda-forge-pinning 2023.12.16.08.16.38 * Use or instead of comma * MNT: Re-rendered with conda-build 3.28.1, conda-smithy 3.30.1, and conda-forge-pinning 2023.12.16.08.16.38 * Specify as osx and x86_64 * MNT: Re-rendered with conda-build 3.27.0, conda-smithy 3.30.1, and conda-forge-pinning 2023.12.16.08.16.38 --------- Co-authored-by: Wei Ji <[email protected]> Co-authored-by: conda-forge-webservices[bot] <91080706+conda-forge-webservices[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Solution to issue cannot be found in the documentation.
Issue
The segfault was first seen in #38, and described in detail at #40 (comment). Opening this issue as a reminder to re-enable the OSX64 builds at some point.
Note: OSX-ARM64 builds (i.e. the Apple Silicon builds) is still ok, at least as mentioned in #42 (comment). Support for these osx-arm64 chips was added in deepspeedai/DeepSpeed#3907.
Traceback error from https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=845204&view=logs&j=4b3757f9-99c5-5891-6c0c-93d7030cdf7d&t=f047df50-a233-5ae9-bf8f-c7dd8e5684ad&l=4170:
Installed packages
annotated-types: 0.6.0-pyhd8ed1ab_0 conda-forge bzip2: 1.0.8-h10d778d_5 conda-forge ca-certificates: 2023.11.17-h8857fd0_0 conda-forge colorama: 0.4.6-pyhd8ed1ab_0 conda-forge deepspeed: 0.12.2-cpu_py311h0112a6b_2 local filelock: 3.13.1-pyhd8ed1ab_0 conda-forge fsspec: 2023.12.2-pyhca7485f_0 conda-forge gmp: 6.3.0-h93d8f39_0 conda-forge gmpy2: 2.1.2-py311hc5b4402_1 conda-forge hjson-py: 3.1.0-pyhd8ed1ab_0 conda-forge icu: 73.2-hf5e326d_0 conda-forge jinja2: 3.1.2-pyhd8ed1ab_1 conda-forge libabseil: 20230802.1-cxx17_h048a20a_0 conda-forge libblas: 3.9.0-20_osx64_openblas conda-forge libcblas: 3.9.0-20_osx64_openblas conda-forge libcxx: 16.0.6-hd57cbcb_0 conda-forge libexpat: 2.5.0-hf0c8a7f_1 conda-forge libffi: 3.4.2-h0d85af4_5 conda-forge libgfortran: 5.0.0-13_2_0_h97931a8_1 conda-forge libgfortran5: 13.2.0-h2873a65_1 conda-forge libhwloc: 2.9.3-default_h24e0189_1009 conda-forge libiconv: 1.17-hd75f5a5_2 conda-forge liblapack: 3.9.0-20_osx64_openblas conda-forge libopenblas: 0.3.25-openmp_hfef2a42_0 conda-forge libprotobuf: 4.24.4-h0ee05dc_0 conda-forge libsqlite: 3.44.2-h92b6c6a_0 conda-forge libuv: 1.46.0-h0c2f820_0 conda-forge libxml2: 2.11.6-hc0ae0f7_0 conda-forge libzlib: 1.2.13-h8a1eda9_5 conda-forge llvm-openmp: 17.0.6-hb6ac08f_0 conda-forge markupsafe: 2.1.3-py311h2725bcf_1 conda-forge mkl: 2022.2.1-h44ed08c_16952 conda-forge mpc: 1.3.1-h81bd1dd_0 conda-forge mpfr: 4.2.1-h0c69b56_0 conda-forge mpmath: 1.3.0-pyhd8ed1ab_0 conda-forge ncurses: 6.4-h93d8f39_2 conda-forge networkx: 3.2.1-pyhd8ed1ab_0 conda-forge numpy: 1.26.2-py311h93c810c_0 conda-forge openssl: 3.2.0-hd75f5a5_1 conda-forge packaging: 23.2-pyhd8ed1ab_0 conda-forge pip: 23.3.1-pyhd8ed1ab_0 conda-forge psutil: 5.9.5-py311h2725bcf_1 conda-forge py-cpuinfo: 9.0.0-pyhd8ed1ab_0 conda-forge pydantic: 2.5.2-pyhd8ed1ab_0 conda-forge pydantic-core: 2.14.5-py311h5e0f0e4_0 conda-forge pynvml: 11.5.0-pyhd8ed1ab_0 conda-forge python: 3.11.7-h9f0c242_0_cpython conda-forge python_abi: 3.11-4_cp311 conda-forge pytorch: 2.1.0-cpu_mkl_py311h5cbed28_100 conda-forge readline: 8.2-h9e318b2_1 conda-forge setuptools: 68.2.2-pyhd8ed1ab_0 conda-forge sleef: 3.5.1-h6db0672_2 conda-forge sympy: 1.12-pypyh9d50eac_103 conda-forge tbb: 2021.11.0-he51d815_0 conda-forge tk: 8.6.13-h1abcd95_1 conda-forge tqdm: 4.66.1-pyhd8ed1ab_0 conda-forge typing-extensions: 4.9.0-hd8ed1ab_0 conda-forge typing_extensions: 4.9.0-pyha770c72_0 conda-forge tzdata: 2023c-h71feb2d_0 conda-forge wheel: 0.42.0-pyhd8ed1ab_0 conda-forge xz: 5.2.6-h775f41a_0 conda-forgeEnvironment info
The text was updated successfully, but these errors were encountered: