Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ensure correct version is reported by pip tools for 0.19.1 and older #105

Closed
1 task done
bschindler opened this issue Dec 17, 2024 · 5 comments
Closed
1 task done

Ensure correct version is reported by pip tools for 0.19.1 and older #105

bschindler opened this issue Dec 17, 2024 · 5 comments
Labels
bug help wanted

Comments

@bschindler
Copy link

Solution to issue cannot be found in the documentation.

  • I checked the documentation.

Issue

I found this merge request which describes exactly the issue I am facing: #104

I am facing exactly this issue, but am unable to move to pytorch 2.5 at this time (which is required for 0.20.1). We are currently at 2.3, and looking into 2.4. Would it be possible to apply this fix to 0.18.1 and 0.19.1 such that these can be depended upon by pip packages?

Thank you

Installed packages

# packages in environment at /opt/miniconda/envs/torchvision:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                  2_kmp_llvm    conda-forge
bzip2                     1.0.8                h4bc722e_7    conda-forge
ca-certificates           2024.12.14           hbcca054_0    conda-forge
cpython                   3.10.16         py310hd8ed1ab_1    conda-forge
cuda-cudart               12.6.77              h5888daf_0    conda-forge
cuda-cudart_linux-64      12.6.77              h3f2d84a_0    conda-forge
cuda-nvrtc                12.6.85              hbd13f7d_0    conda-forge
cuda-nvtx                 12.6.77              hbd13f7d_0    conda-forge
cuda-version              12.6                 h7480c83_3    conda-forge
cudnn                     9.3.0.75             h3e9b439_1    conda-forge
filelock                  3.16.1             pyhd8ed1ab_1    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
fsspec                    2024.10.0          pyhd8ed1ab_1    conda-forge
giflib                    5.2.2                hd590300_0    conda-forge
gmp                       6.3.0                hac33072_2    conda-forge
gmpy2                     2.1.5           py310he8512ff_3    conda-forge
jinja2                    3.1.4              pyhd8ed1ab_1    conda-forge
lcms2                     2.16                 hb7c19ff_0    conda-forge
ld_impl_linux-64          2.43                 h712a8e2_2    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libabseil                 20240722.0      cxx17_h5888daf_1    conda-forge
libblas                   3.9.0           26_linux64_openblas    conda-forge
libcblas                  3.9.0           26_linux64_openblas    conda-forge
libcublas                 12.6.4.1             hbd13f7d_0    conda-forge
libcufft                  11.3.0.4             hbd13f7d_0    conda-forge
libcurand                 10.3.7.77            hbd13f7d_0    conda-forge
libcusolver               11.7.1.2             hbd13f7d_0    conda-forge
libcusparse               12.5.4.2             hbd13f7d_0    conda-forge
libdeflate                1.23                 h4ddbbb0_0    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc                    14.2.0               h77fa898_1    conda-forge
libgcc-ng                 14.2.0               h69a702a_1    conda-forge
libgfortran               14.2.0               h69a702a_1    conda-forge
libgfortran5              14.2.0               hd5240d6_1    conda-forge
libgomp                   14.2.0               h77fa898_1    conda-forge
libhwloc                  2.11.2          default_h0d58e46_1001    conda-forge
libiconv                  1.17                 hd590300_2    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
liblapack                 3.9.0           26_linux64_openblas    conda-forge
liblzma                   5.6.3                hb9d3cd8_1    conda-forge
libmagma                  2.8.0                h7847c38_1    conda-forge
libmagma_sparse           2.8.0                h0af6554_0    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libnvjitlink              12.6.85              hbd13f7d_0    conda-forge
libnvjpeg                 12.3.3.54            h5888daf_0    conda-forge
libopenblas               0.3.28          pthreads_h94d23a6_1    conda-forge
libpng                    1.6.44               hadc24fc_0    conda-forge
libprotobuf               5.28.2               h5b01275_0    conda-forge
libsqlite                 3.47.2               hee588c1_0    conda-forge
libstdcxx                 14.2.0               hc0a3c3a_1    conda-forge
libstdcxx-ng              14.2.0               h4852527_1    conda-forge
libtiff                   4.7.0                hd9ff511_3    conda-forge
libtorch                  2.4.1           cuda120_hcf1373b_303    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libuv                     1.49.2               hb9d3cd8_0    conda-forge
libwebp-base              1.4.0                hd590300_0    conda-forge
libxcb                    1.17.0               h8a09558_0    conda-forge
libxcrypt                 4.4.36               hd590300_1    conda-forge
libxml2                   2.13.5               h0d44e9d_1    conda-forge
libzlib                   1.3.1                hb9d3cd8_2    conda-forge
llvm-openmp               19.1.5               h024ca30_0    conda-forge
markupsafe                3.0.2           py310h89163eb_1    conda-forge
mkl                       2023.2.0         h84fe81f_50496    conda-forge
mpc                       1.3.1                h24ddda3_1    conda-forge
mpfr                      4.2.1                h90cbb55_3    conda-forge
mpmath                    1.3.0              pyhd8ed1ab_1    conda-forge
nccl                      2.23.4.1             h2b5d15b_3    conda-forge
ncurses                   6.5                  he02047a_1    conda-forge
networkx                  3.4.2              pyh267e887_2    conda-forge
numpy                     2.2.0           py310h5851e9f_0    conda-forge
openjpeg                  2.5.3                h5fbd93e_0    conda-forge
openssl                   3.4.0                hb9d3cd8_0    conda-forge
pillow                    11.0.0          py310hfeaa1f3_0    conda-forge
pip                       24.3.1             pyh145f28c_1    conda-forge
pthread-stubs             0.4               hb9d3cd8_1002    conda-forge
python                    3.10.16         he725a3c_1_cpython    conda-forge
python_abi                3.10                    5_cp310    conda-forge
pytorch                   2.4.1           cuda120_py310hf7eb567_303    conda-forge
readline                  8.2                  h8228510_1    conda-forge
sleef                     3.7                  h1b44611_2    conda-forge
sympy                     1.13.3           pyh2585a3b_104    conda-forge
tbb                       2021.13.0            hceb3a55_1    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
torchvision               0.19.1          cuda120py310h8f8fa82_1    conda-forge
typing_extensions         4.12.2             pyha770c72_1    conda-forge
tzdata                    2024b                hc8b5060_0    conda-forge
xorg-libxau               1.0.12               hb9d3cd8_0    conda-forge
xorg-libxdmcp             1.1.5                hb9d3cd8_0    conda-forge
zstd                      1.5.6                ha6fb4c9_0    conda-forge

Environment info

active environment : debug_env
    active env location : /opt/miniconda/envs/debug_env
            shell level : 3
       user config file : /root/.condarc
 populated config files : /opt/miniconda/.condarc
          conda version : 23.1.0
    conda-build version : not installed
         python version : 3.10.12.final.0
       virtual packages : __archspec=1=x86_64
                          __cuda=12.2=0
                          __glibc=2.35=0
                          __linux=6.1.85=0
                          __unix=0=0
       base environment : /opt/miniconda  (writable)
      conda av data dir : /opt/miniconda/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /opt/miniconda/pkgs
                          /root/.conda/pkgs
       envs directories : /opt/miniconda/envs
                          /root/.conda/envs
               platform : linux-64
             user-agent : conda/23.1.0 requests/2.31.0 CPython/3.10.12 Linux/6.1.85+ ubuntu/22.04.4 glibc/2.35
                UID:GID : 0:0
             netrc file : None
           offline mode : False
@bschindler bschindler added the bug label Dec 17, 2024
@hmaarrfk
Copy link
Contributor

It is really hard to do these things retroactively....

If you want to help:

  1. Fork.
  2. Create a branch with a good name.
  3. Please make a PR to the main branch.
  4. Ensure you build the version you want by closesly following the old recipe
  5. Ensure the build number is "bumped" according to the old version.
  6. Pin to the version of pytorch/libtorch you need manually.
  7. Build what you need locally. using python build-locally.py (this is critical)
  8. Upload it to your channel. (this is critical)
  9. Wait for our review. (this can take time).
  10. If accepted, we will make a branch, and retarget your PR to a new branch prior to merging.

I don't promise we will merge, but once you've uploaded a build to your own branch, you should be mostly unblocked to continue your work.

@h-vetinari h-vetinari mentioned this issue Dec 17, 2024
Merged
@h-vetinari
Copy link
Member

It is really hard to do these things retroactively....

0.19 at least is not that long ago, so it shouldn't be that hard. I gave it a shot in #106

@hmaarrfk
Copy link
Contributor

I wasn’t meaning to be hard. Just realistic in expectations of the steps to take to make it happen!!!

Thanks for jumping in!!!

@h-vetinari
Copy link
Member

This was fixed in #106. If you need 0.18 or pytorch before 2.3, you'll need to do it yourself though. ;-)

If so, you can take the v0.19.x branch and try rebasing it (interactively, deleting irrelevant commits) based on the last commit that still has 0.18 (this is essentially how I created #106; by rebasing main on the last commit for 0.19, plus some fixes that were necessary due to changes in pinnings and ambient libraries).

@bschindler
Copy link
Author

Thank you so much @h-vetinari - I was indeed contemplating whether I just want to add a post-patch step to fix up the situation as I have no experience with creating conda packages at all. This makes my life so much easier, thank you again!

No problems with 0.18, I'll first check whether I can do the 0.19/torch 2.4.1 switch first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug help wanted
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@hmaarrfk @bschindler @h-vetinari