Skip to content

Commit

Permalink
Pin python-libarchive-c. Pin conda and conda-build for VS2019 conda-b…
Browse files Browse the repository at this point in the history
…uild step (#5646)

Fixes: pytorch/vision#8651

Trying to resolve this error during vision and audio windows conda
builds:
https://github.com/pytorch/audio/actions/runs/10829977489/job/30048718670

```
# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "C:\Jenkins\Miniconda3\lib\site-packages\conda\exception_handler.py", line 18, in __call__
        return func(*args, **kwargs)
      File "C:\Jenkins\Miniconda3\lib\site-packages\conda\cli\main.py", line 84, in main_subshell
        exit_code = do_call(args, parser)
      File "C:\Jenkins\Miniconda3\lib\site-packages\conda\cli\conda_argparse.py", line 176, in do_call
        result = plugin_subcommand.action(getattr(args, "_args", args))
      File "C:\Jenkins\Miniconda3\lib\site-packages\conda_build\plugin.py", line 15, in build
If submitted, this report will be used by core maintainers to improve
future releases of conda.
        from .cli.main_build import execute
      File "C:\Jenkins\Miniconda3\lib\site-packages\conda_build\cli\main_build.py", line 19, in <module>
        from .. import api, build, source, utils
      File "C:\Jenkins\Miniconda3\lib\site-packages\conda_build\api.py", line 23, in <module>
        from .config import DEFAULT_PREFIX_LENGTH as _prefix_length
      File "C:\Jenkins\Miniconda3\lib\site-packages\conda_build\config.py", line 23, in <module>
        from .utils import (
      File "C:\Jenkins\Miniconda3\lib\site-packages\conda_build\utils.py", line 49, in <module>
        import libarchive
      File "C:\Jenkins\Miniconda3\lib\site-packages\libarchive\__init__.py", line 1, in <module>
        from .entry import ArchiveEntry
      File "C:\Jenkins\Miniconda3\lib\site-packages\libarchive\entry.py", line 6, in <module>
        from . import ffi
      File "C:\Jenkins\Miniconda3\lib\site-packages\libarchive\ffi.py", line 26, in <module>
        libarchive = ctypes.cdll.LoadLibrary(libarchive_path)
      File "C:\Jenkins\Miniconda3\lib\ctypes\__init__.py", line 452, in LoadLibrary
        return self._dlltype(name)
      File "C:\Jenkins\Miniconda3\lib\ctypes\__init__.py", line 364, in __init__
        if '/' in name or '\\' in name:
    TypeError: argument of type 'NoneType' is not iterable

`$ C:\Jenkins\Miniconda3\Scripts\conda-script.py build -c defaults -c nvidia -c pytorch-nightly --no-anaconda-upload --no-test --python 3.12 --output-folder distr/ packaging/torchaudio`

  environment variables:
                 CIO_TEST=<not set>
    CONDA_ALLOW_SOFTLINKS=false
      CONDA_BUILD_VARIANT=cuda
CONDA_CUDATOOLKIT_CONSTRAINT=- pytorch-cuda=12.1 # [not osx]
                CONDA_ENV=C:\actions-runner\_work\_temp/conda_environment_10829977489
  CONDA_PACKAGE_DIRECTORY=packaging/torchaudio
CONDA_PYTORCH_BUILD_CONSTRAINT=- pytorch==2.5.0.dev20240912
 CONDA_PYTORCH_CONSTRAINT=- pytorch==2.5.0.dev20240912
               CONDA_ROOT=C:\Jenkins\Miniconda3
                CONDA_RUN=conda run -p C:\actions-
                          runner\_work\_temp/conda_environment_10829977489
                CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1
           CURL_CA_BUNDLE=<not set>
        GITHUB_EVENT_PATH=C:\actions-runner\_work\_temp\_github_workflow\event.json
              GITHUB_PATH=C:\actions-runner\_work\_temp\_runner_file_commands\add_path_eb2fa2af-
                          6e4a-4eab-96eb-e09885dfb14d
                 INFOPATH=C:\Program Files\Git\usr\local\info;C:\Program
                          Files\Git\usr\share\info;C:\Program Files\Git\usr\info;C:\Program
                          Files\Git\share\info
               LD_PRELOAD=<not set>
                  MANPATH=C:\Program Files\Git\usr\local\man;C:\Program
                          Files\Git\usr\share\man;C:\Program Files\Git\usr\man;C:\Program
                          Files\Git\share\man
            ORIGINAL_PATH=C:\Jenkins\Miniconda3\Scripts;C:\Windows\system32;C:\Windows;C:\Window
                          s\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\
                          System32\OpenSSH;C:\Program Files\Amazon\cfn-
                          bootstrap;C:\ProgramData\chocolatey\bin;C:\Program
                          Files\Amazon\AWSCLIV2;C:\Program Files\Git\cmd;C:\Program
                          Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\Program Files
                          (x86)\Windows Kits\10\Windows Performance
                          Toolkit;C:\Users\runneruser\AppData\Local\Microsoft\WindowsApps
                     PATH=C;C:\Program Files\NVIDIA GPU Computing
                          Toolkit\CUDA\v12.1\bin;C:\Users\runneruser\bin;C:\Program
                          Files\Git\usr\local\bin;C:\Program Files\Git\usr\bin;C:\Program
                          Files\Git\usr\bin;C:\Program Files\Git\opt\bin;C:\Jenkins\Miniconda3\S
                          cripts;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Wind
                          ows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Pro
                          gram Files\Amazon\cfn-
                          bootstrap;C:\ProgramData\chocolatey\bin;C:\Program
                          Files\Amazon\AWSCLIV2;C:\Program Files\Git\cmd;C:\Program
                          Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\Program Files
                          (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Users\runneruser\
                          AppData\Local\Microsoft\WindowsApps;C:\Program
                          Files\Git\usr\bin\vendor_perl;C:\Program Files\Git\usr\bin\core_perl
          PKG_CONFIG_PATH=C:\Program Files\Git\usr\lib\pkgconfig;C:\Program
                          Files\Git\usr\share\pkgconfig;C:\Program Files\Git\lib\pkgconfig
             PSMODULEPATH=C:\Windows\system32\WindowsPowerShell\v1.0\Modules;C:\Program
                          Files\WindowsPowerShell\Modules
           PYTHON_VERSION=3.12
```

Cause of the failure. New python-libarchive-c release:
https://anaconda.org/anaconda/python-libarchive-c/files?version=5.1

During setup binary builds we install following conda and conda-build:
```
    - conda-build=24.3.0
    - conda=24.4.0
```

However later we install these ones in different conda env:
```
    conda-24.7.1               |   py39haa95532_0         963 KB
    conda-build-24.7.1         |   py39haa95532_0         587 KB
```

This PR makes sure workflow is using same conda and conda-build
  • Loading branch information
atalman authored Sep 12, 2024
1 parent 056b000 commit 32033b0
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion .github/actions/setup-binary-builds/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ runs:
if [[ "${PACKAGE_TYPE:-}" == "conda" ]]; then
# For conda package host python version is irrelevant
export PYTHON_VERSION=3.9
export CONDA_BUILD_EXTRA="conda=24.4.0 conda-build=24.3.0"
export CONDA_BUILD_EXTRA="conda=24.4.0 conda-build=24.3.0 python-libarchive-c=2.9"
else
# For wheel builds we don't need neither conda nor conda-build
export CONDA_BUILD_EXTRA=""
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/build_conda_windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ jobs:
export VSDEVCMD_ARGS=''
source "${BUILD_ENV_FILE}"
source /c/Jenkins/Miniconda3/etc/profile.d/conda.sh
conda install -yq conda-build "conda-package-handling!=1.5.0"
conda install -yq conda=24.4.0 conda-build=24.3.0 "conda-package-handling!=1.5.0" python-libarchive-c=2.9
conda build \
-c defaults \
Expand Down

0 comments on commit 32033b0

Please sign in to comment.