Python 3.13 compatibility

[jameswilburlewis]

basemap does not "pip install" cleanly under Python 3.13. Might be related to some of the pinned upper versions of package dependencies? This environment had numpy 1.26.4 installed (so not a numpy 2.0 issue).

(pycharm_py313) jwl@jwl-new-mac pyspedas_py313 % pip install basemap       
Collecting basemap
  Using cached basemap-1.3.9.zip (156 kB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [63 lines of output]
      Ignoring numpy: markers 'python_version == "3.10"' don't match your environment
      Ignoring numpy: markers 'sys_platform == "darwin" and (python_version >= "3.7" and python_version <= "3.9")' don't match your environment
      Ignoring numpy: markers 'sys_platform != "darwin" and (python_version >= "3.7" and python_version <= "3.9")' don't match your environment
      Ignoring numpy: markers 'python_version == "2.7" or (python_version >= "3.4" and python_version <= "3.6")' don't match your environment
      Ignoring numpy: markers 'python_version == "2.6" or (python_version >= "3.2" and python_version <= "3.3")' don't match your environment
      Ignoring cython: markers 'python_version == "3.2"' don't match your environment
      Collecting setuptools
        Using cached setuptools-75.4.0-py3-none-any.whl.metadata (6.8 kB)
      Collecting wheel
        Using cached wheel-0.45.0-py3-none-any.whl.metadata (2.3 kB)
      Collecting numpy==1.23.3
        Using cached numpy-1.23.3.tar.gz (10.7 MB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'error'
        error: subprocess-exited-with-error
      
        × Getting requirements to build wheel did not run successfully.
        │ exit code: 1
        ╰─> [32 lines of output]
            Traceback (most recent call last):
              File "/Users/jwl/anaconda3/envs/pycharm_py313/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
                main()
                ~~~~^^
              File "/Users/jwl/anaconda3/envs/pycharm_py313/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
                json_out['return_val'] = hook(**hook_input['kwargs'])
                                         ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
              File "/Users/jwl/anaconda3/envs/pycharm_py313/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 112, in get_requires_for_build_wheel
                backend = _build_backend()
              File "/Users/jwl/anaconda3/envs/pycharm_py313/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
                obj = import_module(mod_path)
              File "/Users/jwl/anaconda3/envs/pycharm_py313/lib/python3.13/importlib/__init__.py", line 88, in import_module
                return _bootstrap._gcd_import(name[level:], package, level)
                       ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
              File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
              File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
              File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
              File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
              File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
              File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
              File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
              File "<frozen importlib._bootstrap_external>", line 1022, in exec_module
              File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
              File "/private/var/folders/_4/hbrtgs_j1qlgl2f3fwjw2_kc0000gn/T/pip-build-env-83asxxgw/overlay/lib/python3.13/site-packages/setuptools/__init__.py", line 16, in <module>
                import setuptools.version
              File "/private/var/folders/_4/hbrtgs_j1qlgl2f3fwjw2_kc0000gn/T/pip-build-env-83asxxgw/overlay/lib/python3.13/site-packages/setuptools/version.py", line 1, in <module>
                import pkg_resources
              File "/private/var/folders/_4/hbrtgs_j1qlgl2f3fwjw2_kc0000gn/T/pip-build-env-83asxxgw/overlay/lib/python3.13/site-packages/pkg_resources/__init__.py", line 2172, in <module>
                register_finder(pkgutil.ImpImporter, find_on_path)
                                ^^^^^^^^^^^^^^^^^^^
            AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?
            [end of output]
 

[DWesl]

Failing in import setuptools.version is odd; does pip install --no-build-isolation basemap do the same thing?

[molinav]

Currently, Python 3.13 is still not supported by basemap (there is also a pin <3.13 for Python in setup.py). I am aware that Python 3.13 was released last October, and I would like to add the support, but currently I am a bit overloaded at my job.

Hopefully in December I will get some time free (aka Christmas present releases).

[jameswilburlewis]

@molinav Thanks for checking, I hope I've been good enough this year to be deserving of a nice Christmas release!

[cvanelteren]

Can you checkout #611? If there are some issues I can include it in the PR

[wreckdump]

Are we still having issue with python 3.13? I would like to know if I should update the python version or not.

[cvanelteren]

The PR will install it for 3.13, but we are fixing it on another PR in a more profound way. It is still a bit yank right. Depending on your needs it is worth it or not.

[molinav]

@jameswilburlewis Support for Python 3.13 in basemap has been added in #619. However, from your traceback, I may assume that you are trying to install on MacOS, right? So far, the MacOS wheels on PyPI were built manually by myself in a personal Mac mini M1, but I would like to include the build workflows for MacOS before the basemap release 2.0.0, so that these wheels do not have to rely on manual builds.

[jameswilburlewis]

@molinav Yes, that's correct, we would like to support pypi installs on MacOS. Hopefully with binary wheels supporting ARM, on Ventura and later OS versions. We don't have an urgent need for Python 3.13 support yet, so if it takes a little longer to get your build workflows working the way you want, we're happy to wait. I appreciate the effort!

[cvanelteren]

hmm strange it seems to build on latest arm for me https://github.com/cvanelteren/basemap/actions/runs/13516075905/job/37764944503

[cvanelteren]

ah nvm I thought it was built from source; I put a PR up for adding the wheels #620

[molinav]

@cvanelteren basemap should build properly on MacOS x86_64 and arm64, I used to make it manually in my personal Mac mini M1 for previous releases. The thing is to see if this can be integrated into the automated builds, so that I can get rid of this manual step.

[cvanelteren]

@molinav done!

[molinav]

@jameswilburlewis May I ask you to try the wheel below for MacOS ARM64 and Python 3.13? I picked it from one of the last workflows in the develop branch. I had to change the extension from .whl to .zip; otherwise, I could not attach it in my message.

basemap-2.0.0.dev0-cp313-cp313-macosx_11_0_arm64.zip

[jameswilburlewis]

@molinav I think it worked! There was a slight snag installing it because pyproj apparently doesn't have a binary wheel yet for Python 3.13/MacOs Ventura/ARM M2. Rather than try building it from source, I installed pyproj from conda (Generally a terrible idea, but I wanted to cut to the chase and see if pyspedas could successfully use basemap...). The plots look reasonable, and that pyspedas module is passing its unit tests. So it looks good from here!

[molinav]

I was checking in PyPI and indeed it is quite unfortunate:
https://pypi.org/project/pyproj/3.7.1/#files

The pyproj library is targeting to relatively new MacOS wheel tags for pyproj 3.7.1, and it leaves Ventura out for MacOS ARM64:
pyproj-3.7.1-cp313-cp313-macosx_14_0_arm64.whl
pyproj-3.7.1-cp313-cp313-macosx_13_0_x86_64.whl

If one goes to pyproj 3.6.1, when they were supporting only until Python 3.12, the MacOS wheel tags were the typical legacy ones:
pyproj-3.6.1-cp312-cp312-macosx_11_0_arm64.whl
pyproj-3.6.1-cp312-cp312-macosx_10_9_x86_64.whl

I wonder whether this wheel tag upgrade is really needed by pyproj (because of some requirement when building PROJ 9.5.1), or whether it is part of this dropping-support tendency in the Python ecosystem. At least I can say that basemap did its part in the migration to Python 3.13!

[molinav]

The change in pyproj that is moving the MacOS deployment target was introduced in these PRs:
https://github.com/pyproj4/pyproj/pull/1410/files (part of pyproj 3.7.0)
https://github.com/pyproj4/pyproj/pull/1475/files (part of pyproj 3.7.1)

It looks to me that the macos_deployment_target variables are simply updated to match the MacOS version of the host runner. However, I do not see why this is necessary.

[molinav]

With PRs #619 and #620, the main problem indicated here is then solved, so I will be closing the issue.

It may still be worth it to report to the pyproj developers about this thing with the MacOS deployment target, so that your whole environment continues working in MacOS Ventura.

[jameswilburlewis]

@molinav There are a few other packages that are slow to support MacOs Ventura with Python 3.13 wheels. I think this is partly due to Github's choice to only offer runners for Sonoma and maybe Sequoia, so projects that do their builds with Github Actions have to go to extra effort to support Ventura. Thanks very much for your work supporting basemap on this platform!