Skip to content

Commit

Permalink
PEP 674: Address Petr's review
Browse files Browse the repository at this point in the history
  • Loading branch information
vstinner committed Feb 7, 2022
1 parent a46be77 commit e945677
Showing 1 changed file with 49 additions and 35 deletions.
84 changes: 49 additions & 35 deletions pep-0674.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
PEP: 674
Title: Disallow using Py_TYPE() and Py_SIZE() macros as l-values
Title: Disallow using macros as l-values
Author: Victor Stinner <[email protected]>
Status: Draft
Type: Standards Track
Expand All @@ -10,16 +10,8 @@ Python-Version: 3.11
Abstract
========

Incompatible C API change disallowing using macros, especially
``Py_TYPE()`` and ``Py_SIZE()``, as l-values to:

* Allow evolving CPython internals (change the ``PyObject`` structure);
* Ease the C API implementation on other Python implementation;
* Help migrating existing C extensions to the HPy API.

Only 7 out of the top 5000 PyPI projects (0.1%) are affected by this
PEP. An additional 23 projects just have to regenerate their Cython
code.
Disallow using macros as l-values. For example,
``Py_TYPE(obj) = new_type`` now fails with a compiler error.

In practice, the majority of affected projects only have to make two
changes:
Expand Down Expand Up @@ -301,6 +293,9 @@ Backwards Compatibility

The proposed C API changes are backward incompatible on purpose.

In practice, only ``Py_TYPE()`` and ``Py_SIZE()`` macros are used as
l-values.

This change does not follow the :pep:`387` deprecation process. There is
no known way to emit a deprecation warning only when a macro is used as
an l-value, but not when it's used differently (ex: as a r-value).
Expand All @@ -309,46 +304,44 @@ The following 4 macros are left unchanged to reduce the number of
affected projects: ``PyDescr_NAME()``, ``PyDescr_TYPE()``,
``PyList_GET_ITEM()`` and ``PyTuple_GET_ITEM()``.

Statistics
----------

In total (projects on PyPI and not on PyPI), 41 projects are affected by
this PEP:

This comment has been minimized.

Copy link
@encukou

encukou Feb 7, 2022

Member

Huh? This sounds like you were able to review all projects that use Python's C API. I don't think that's possible. How did you choose the projects to review?

This comment has been minimized.

Copy link
@vstinner

vstinner Feb 7, 2022

Author Member

I wasn't sure how to express that. I proposed #2311 : is it better? (please reply on the PR ;-))


* 14 projects (34%) are already fixed
* 27 projects (66%) are not fixed yet
(pending fix or generated Cython code)

On January 27, 2022, the PEP affects 27 projects (0.5%) of the top 5000
PyPI projects:

* 22 projects (0.4%) have to regenerate their Cython code
* 5 projects (0.1%) have a pending fix

Top 5000 PyPI
-------------

On January 27, 2022, a code search on the top 5000 PyPI projects found
that only 7 projects are affected (0.1%):
Projects with a pending fix (5):

* datatable (1.0.0):
`pending PR <https://github.com/h2oai/datatable/pull/3231>`__
* guppy3 (3.1.2): `fixed <https://github.com/zhuyifei1999/guppy3/commit/4cb9fcb5d75327544a6875b6caabfdffb70a7e29>`__
* pickle5 (0.0.12): backport for Python <= 3.7
* pysha3 (1.0.2): backport for Python <= 3.5
* guppy3 (3.1.2):
`fixed <https://github.com/zhuyifei1999/guppy3/commit/4cb9fcb5d75327544a6875b6caabfdffb70a7e29>`__
* python-snappy (0.6.0):
`fixed <https://github.com/andrix/python-snappy/commit/1a539d71d5b1ceaf9a2291f21f686cf53a46d707>`__
* recordclass (0.17.1):
`fixed <https://bitbucket.org/intellimath/recordclass/commits/d20d72fa3cdbdcf96c72941560041460adeecff1>`__
* scipy (1.7.3): fixed (need to update boost)

All of those projects are fixed (but not released), have a pending fix,
or should not be fixed (backports).

An additional 23 projects only have to regenerate their Cython code to
use ``Py_SET_TYPE()`` and ``Py_SET_SIZE()``.

Only 2 macros are used as an l-value: ``Py_TYPE()`` and ``Py_SIZE()``.
Moreover, 22 projects have to regenerate their Cython code.

Projects released with a fix
----------------------------

Projects released with a fix (14):
Projects released with a fix (10):

* bitarray (1.6.2):
`commit <https://github.com/ilanschnell/bitarray/commit/a0cca9f2986ec796df74ca8f42aff56c4c7103ba>`__
* boost (1.78.0): `commit
<https://github.com/boostorg/python/commit/500194edb7833d0627ce7a2595fec49d0aae2484>`__
* breezy (3.2.1): `bug report <https://bugs.launchpad.net/brz/+bug/1904868>`__
* Cython (0.29.20): `commit <https://github.com/cython/cython/commit/d8e93b332fe7d15459433ea74cd29178c03186bd>`__
* duplicity (0.8.18):
`commit <https://git.launchpad.net/duplicity/commit/duplicity/_librsyncmodule.c?id=bbaae91b5ac6ef7e295968e508522884609fbf84>`__
* gobject-introspection (1.70.0):
`MR <https://gitlab.gnome.org/GNOME/gobject-introspection/-/merge_requests/243>`__
* immutables (0.15):
`commit <https://github.com/MagicStack/immutables/commit/45105ecd8b56a4d88dbcb380fcb8ff4b9cc7b19c>`__
* mercurial (5.7):
Expand All @@ -364,9 +357,30 @@ Projects released with a fix (14):
* PyGObject (3.42.0)
* pyside2 (5.15.1):
`bug report <https://bugreports.qt.io/browse/PYSIDE-1436>`__
* python-zstd (1.5.0.3):
* zstd (1.5.0.3):
`commit <https://github.com/sergey-dryabzhinsky/python-zstd/commit/8aa6d7a4b250e1f0a4e27b4107c39dc516c87f96>`__

There are also two backport projects which are affected by this PEP:

* pickle5 (0.0.12): backport for Python <= 3.7
* pysha3 (1.0.2): backport for Python <= 3.5

They must not be used and cannot be used on Python 3.11.

Other affected projects
-----------------------

Other projects released with a fix (4):

* boost (1.78.0):
`commit <https://github.com/boostorg/python/commit/500194edb7833d0627ce7a2595fec49d0aae2484>`__
* breezy (3.2.1):
`bug report <https://bugs.launchpad.net/brz/+bug/1904868>`__
* duplicity (0.8.18):
`commit <https://git.launchpad.net/duplicity/commit/duplicity/_librsyncmodule.c?id=bbaae91b5ac6ef7e295968e508522884609fbf84>`__
* gobject-introspection (1.70.0):
`MR <https://gitlab.gnome.org/GNOME/gobject-introspection/-/merge_requests/243>`__


Relationship with the HPy project
=================================
Expand Down

0 comments on commit e945677

Please sign in to comment.