Convert _sysconfigdata to a JSON file

[FFY00]

Feature or enhancement

Proposal:

Currently, as part of the sysconfig internals, we generate a _sysconfigdata Python module containing the variables for sysconfig.get_config_vars(). This data is consists of all Makefile variables, which can be encoded as JSON data.

The use of a module instead of a text file is historic, and adds unnecessary complexity. Converting _sysconfigdata to a JSON file would simplify the sysconfig implementation. Additionally, it also makes it easier for tools to introspect the Python installation, as this would remove the necessity to execute Python code to read _sysconfigdata.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs

• GH-127178: convert _sysconfigdata to a JSON file #127180
• GH-127178: install a _sysconfig_vars_(...).json file in the stdlib directory #127302
• GH-127178: improve compatibility in _sysconfig_vars_(...).json #128558

[FFY00]

Converting the module to a JSON file seems to be causing some issues on special platforms, such as WASM. Let's install a JSON file together with the module as a starting point, which will enable the introspection use-case, and then explore the possibility of replacing the module with it.

[nascheme]

It seems this change causes the JIT build bots to fail:

FAIL: test_sysconfigdata_json (test.test_sysconfig.TestSysConfig.test_sysconfigdata_json)
11766
----------------------------------------------------------------------
11767
Traceback (most recent call last):
11768
  File "/home/runner/work/cpython/cpython/Lib/test/test_sysconfig.py", line 654, in test_sysconfigdata_json
11769
    self.assertEqual(system_config_vars, json_config_vars)
[...]
-  'base': '/usr/local',
+  'base': '/home/runner/work/cpython/build',
   'datarootdir': '/usr/local/share',
-  'exec_prefix': '/usr/local',
+  'exec_prefix': '/home/runner/work/cpython/build',
   'host_exec_prefix': '/usr/local',
   'host_prefix': '/usr/local',
   'implementation': 'Python',
   'implementation_lower': 'python',
-  'installed_base': '/usr/local',
-  'installed_platbase': '/usr/local',
-  'platbase': '/usr/local',
+  'installed_base': '/home/runner/work/cpython/build',
+  'installed_platbase': '/home/runner/work/cpython/build',
+  'platbase': '/home/runner/work/cpython/build',
   'platlibdir': 'lib',
-  'prefix': '/usr/local',
+  'prefix': '/home/runner/work/cpython/build',
   'py_version': '3.14.0a2+',
   'py_version_nodot': '314',
   'py_version_nodot_plat': '',
   'py_version_short': '3.14',
   'userbase': '/home/runner/.local'}

https://github.com/python/cpython/actions/runs/12090193539/job/33716932397?pr=127426

[FFY00]

How does the prefix change from /usr/local (what is configured in the Makefile) to /home/runner/work/cpython/build? Are you setting DESTDIR?

[FFY00]

Hum... no. It is setting the prefix.

cpython/.github/workflows/jit.yml

Line 135 in 38264a0

./configure --prefix="$(pwd)/../build"

Let me do some debugging.

[FFY00]

Ah, no. I think you are getting python from a following build, which does not set the prefix, having it default to /usr/local.

cpython/.github/workflows/jit.yml

Line 144 in 38264a0

./configure --enable-experimental-jit ${{ matrix.debug && '--with-pydebug' || '' }} --build=x86_64-linux-gnu --host="$HOST" --with-build-python=../build/bin/python3 --with-pkg-config=no ac_cv_buggy_getaddrinfo=no ac_cv_file__dev_ptc=no ac_cv_file__dev_ptmx=yes

This seems to me like a bug in the --with-build-python option which keeps the installation directory, but not the prefix.

[FFY00]

Ah, I got it. The sysconfig data is generated by the interpreter passed to --with-build-python, but doesn't match the running interpreter.

[FFY00]

I opened GH-127429.