tests/test_threadpoolctl.py::test_command_line_empty regression with Python 3.12

[mgorny]

When running the test suite against Python 3.12.0b4, I'm getting the following test failure:

_______________________________________________________ test_command_line_empty _______________________________________________________

    def test_command_line_empty():
        output = subprocess.check_output((sys.executable + " -m threadpoolctl").split())
>       assert json.loads(output.decode("utf-8")) == []
E       AssertionError: assert [{'user_api': 'openmp', 'internal_api': 'openmp', 'prefix': 'libgomp', 'filepath': '/usr/lib/gcc/x86_64-pc-linux-gnu/13/libgomp.so.1.0.0', 'version': None, 'num_threads': 12}] == []
E         Left contains one more item: {'filepath': '/usr/lib/gcc/x86_64-pc-linux-gnu/13/libgomp.so.1.0.0', 'internal_api': 'openmp', 'num_threads': 12, 'prefix': 'libgomp', ...}
E         Full diff:
E           [
E         -  ,
E         +  {'filepath': '/usr/lib/gcc/x86_64-pc-linux-gnu/13/libgomp.so.1.0.0',
E         +   'internal_api': 'openmp',
E         +   'num_threads': 12,
E         +   'prefix': 'libgomp',
E         +   'user_api': 'openmp',
E         +   'version': None},
E           ]

tests/test_threadpoolctl.py:535: AssertionError

The test passes with Python 3.11.

[ogrisel]

Thanks @mgorny for the report but I cannot reproduce with Python 3.12.0 (final):

py312 ❯ pytest -v -k test_command_line_empty
===================================================================== test session starts =====================================================================
platform darwin -- Python 3.12.0, pytest-7.4.2, pluggy-1.3.0 -- /Users/ogrisel/mambaforge/envs/py312/bin/python3.12
cachedir: .pytest_cache
rootdir: /Users/ogrisel/code/threadpoolctl
plugins: cov-4.1.0
collected 63 items / 62 deselected / 1 selected                                                                                                               

tests/test_threadpoolctl.py::test_command_line_empty PASSED                                                                                             [100%]

============================================================== 1 passed, 62 deselected in 0.09s ===============================================================

Do you confirm we can close this issue?

[mgorny]

I'm afraid I can still reproduce it with Python 3.12.0 final. Perhaps it is Linux-specific?

I've also tried 85d329a and it happens there still. Clearly Python version is relevant:

$ python3.11 -m threadpoolctl 
[]
$ python3.12 -m threadpoolctl 
[
  {
    "user_api": "openmp",
    "internal_api": "openmp",
    "num_threads": 12,
    "prefix": "libgomp",
    "filepath": "/usr/lib/gcc/x86_64-pc-linux-gnu/13/libgomp.so.1.0.0",
    "version": null
  }
]

[ogrisel]

Ok so it looks like python itself is linked against libgomp by default on Python 3.12? That's unexpected. Let me update our CI in a PR to check.

[mgorny]

Ok so it looks like python itself is linked against libgomp by default on Python 3.12?

I don't think that's the case, I'm afraid. Or at least trivial grep -R libgomp reveals nothing.

[mgorny]

Oh, I see what's going on. Python 3.12 links to libb2.so (BLAKE2 provider), and libb2.so links to libgomp.so.

[ogrisel]

That's very interesting.

[ogrisel]

Oh, I see what's going on. Python 3.12 links to libb2.so (BLAKE2 provider), and libb2.so links to libgomp.so.

Although I cannot reproduce on a conda-forge installation, even on Linux.

How did you install python3.12 on your host?

EDIT: from your github bio, I guess it's the Python package from gentoo. I can try to replicate with podman or docker.

[ogrisel]

I guess we will have to make our tests robust to Python installs that come with libgomp linked in by default.

[mgorny]

Yeah, it's system Python package. FWICS CPython just picks libb2 up if it's installed at build time (with pkg-config module). To reproduce the specific problem, you have to also build libb2 with --enable-openmp, I think.

I guess we will have to make our tests robust to Python installs that come with libgomp linked in by default.

Yeah, I suppose any library in the stack could incidentally bring OpenMP in.

[ogrisel]

@mgorny please let me know if #155 works for you.

[mgorny]

Thanks. I can confirm that it fixes the issue for us on Gentoo — however, as I've noted on the PR, other Linux systems are bound to use different paths than /usr/lib/, so they'll probably eventually be affected as well.