Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to PyMC5 #108

Merged
merged 2 commits into from
Dec 12, 2024
Merged

Update to PyMC5 #108

merged 2 commits into from
Dec 12, 2024

Conversation

rgieseke
Copy link
Collaborator

@rgieseke rgieseke commented Dec 9, 2024

This PR updates Attrici to the latest PyMC 🚀 version and removes the workarounds and outdated dependencies resulting from using PyMC3.

  • removes dependency on Python 3.11
  • adds CI tests for Python 3.12
  • removes Theano dependency

@matthiasmengel
Copy link
Member

Tried to install without constraining to python=3.11. This fails (mamba picked python3.13.1:

Collecting pytensor<2.18,>=2.17.0 (from pymc->attrici==1.1.1.dev112+g15cbd10)
  Using cached pytensor-2.17.1.tar.gz (3.5 MB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [57 lines of output]
      Collecting setuptools>=48.0.0
        Using cached setuptools-75.6.0-py3-none-any.whl.metadata (6.7 kB)
      Collecting cython
        Downloading Cython-3.0.11-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.2 kB)
      Collecting numpy<1.26,>=1.17.0
        Using cached numpy-1.25.2.tar.gz (10.8 MB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'error'
        error: subprocess-exited-with-error
      
        × Getting requirements to build wheel did not run successfully.
        │ exit code: 1
        ╰─> [32 lines of output]
            Traceback (most recent call last):
              File "/home/mengel/miniforge3/envs/attrici_dev/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
                main()
                ~~~~^^
              File "/home/mengel/miniforge3/envs/attrici_dev/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
                json_out['return_val'] = hook(**hook_input['kwargs'])
                                         ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
              File "/home/mengel/miniforge3/envs/attrici_dev/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 112, in get_requires_for_build_wheel
                backend = _build_backend()
              File "/home/mengel/miniforge3/envs/attrici_dev/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
                obj = import_module(mod_path)
              File "/home/mengel/miniforge3/envs/attrici_dev/lib/python3.13/importlib/__init__.py", line 88, in import_module
                return _bootstrap._gcd_import(name[level:], package, level)
                       ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
              File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
              File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
              File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
              File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
              File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
              File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
              File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
              File "<frozen importlib._bootstrap_external>", line 1026, in exec_module
              File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
              File "/tmp/pip-build-env-w4c2qxjc/overlay/lib/python3.13/site-packages/setuptools/__init__.py", line 16, in <module>
                import setuptools.version
              File "/tmp/pip-build-env-w4c2qxjc/overlay/lib/python3.13/site-packages/setuptools/version.py", line 1, in <module>
                import pkg_resources
              File "/tmp/pip-build-env-w4c2qxjc/overlay/lib/python3.13/site-packages/pkg_resources/__init__.py", line 2172, in <module>
                register_finder(pkgutil.ImpImporter, find_on_path)
                                ^^^^^^^^^^^^^^^^^^^
            AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?
            [end of output]
      
        note: This error originates from a subprocess, and is likely not a problem with pip.
      error: subprocess-exited-with-error
      
      × Getting requirements to build wheel did not run successfully.
      │ exit code: 1
      ╰─> See above for output.
      
      note: This error originates from a subprocess, and is likely not a problem with pip.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

install with python=3.11 works. (Just to report this)

@matthiasmengel
Copy link
Member

matthiasmengel commented Dec 12, 2024

Example from USAGE.md

attrici detrend --gmt-file ./tests/data/20CRv3-ERA5_germany_ssa_gmt.nc --input-file ./tests/data/20CRv3-ERA5_germany_obs.nc --variable tas --stop-date "2023-12-31" --report-variables ds y cfact logp

fails for me with

2024-12-12 12:28:26.937 | INFO     | attrici.detrend:get_task_indices:153 - This is task 0 which will do runs 0 to 3
WARNING (pytensor.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
2024-12-12 12:28:30.310 | INFO     | attrici.detrend:detrend:390 - This is task 0 working on lat,lon 51.25,9.25
Traceback (most recent call last):
  File "/home/mengel/miniforge3/envs/attrici_dev/bin/attrici", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/mengel/projects/attrici/attrici/cli.py", line 46, in main
    args.func(args)
  File "/home/mengel/projects/attrici/attrici/commands/detrend.py", line 21, in run
    detrend(config)
  File "/home/mengel/projects/attrici/attrici/util.py", line 40, in wrapped
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mengel/projects/attrici/attrici/detrend.py", line 394, in detrend
    detrend_cell(
  File "/home/mengel/projects/attrici/attrici/detrend.py", line 205, in detrend_cell
    Path(config.output_dir)
  File "/home/mengel/miniforge3/envs/attrici_dev/lib/python3.11/pathlib.py", line 871, in __new__
    self = cls._from_parts(args)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mengel/miniforge3/envs/attrici_dev/lib/python3.11/pathlib.py", line 509, in _from_parts
    drv, root, parts = self._parse_args(args)
                       ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mengel/miniforge3/envs/attrici_dev/lib/python3.11/pathlib.py", line 493, in _parse_args
    a = os.fspath(a)
        ^^^^^^^^^^^^
TypeError: expected str, bytes or os.PathLike object, not NoneType

@swillner
Copy link
Collaborator

Sorry that USAGE.md is still outdated. Please also provide an --output-dir parameter (e.g. --output-dir .)

@matthiasmengel
Copy link
Member

Sorry that USAGE.md is still outdated. Please also provide an --output-dir parameter (e.g. --output-dir .)

Then, shouldnt I get a different error message like output-dir needs to be set and is missing.

@rgieseke rgieseke changed the base branch from tests-adjustment to develop December 12, 2024 15:32
rgieseke and others added 2 commits December 12, 2024 16:35
- removes dependency on Python 3.11
- adds CI tests for Python 3.12
- removes Theano dependency

Co-authored-by: Sven Willner <[email protected]>
Co-authored-by: Robert Gieseke <[email protected]>
@matthiasmengel matthiasmengel merged commit 192dad3 into develop Dec 12, 2024
2 checks passed
@rgieseke
Copy link
Collaborator Author

Sorry that USAGE.md is still outdated. Please also provide an --output-dir parameter (e.g. --output-dir .)

Then, shouldnt I get a different error message like output-dir needs to be set and is missing.

Good point, I'll create an issue for that!

@rgieseke rgieseke deleted the pymc5-update branch December 12, 2024 16:01
@rgieseke rgieseke mentioned this pull request Dec 12, 2024
rgieseke added a commit that referenced this pull request Dec 13, 2024
PyMC5 currently has issues with 3.13

See #108 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants