PEP 674: Leave PyDescr_TYPE() unchanged (#2277)

Add Version History.

pep-0674.rst

@@ -1,5 +1,5 @@
 PEP: 674
-Title: Disallow using macros as l-value
+Title: Disallow using macros as an l-value
 Author: Victor Stinner <[email protected]>
 Status: Draft
 Type: Standards Track
@@ -10,15 +10,15 @@ Python-Version: 3.11
 Abstract
 ========
 
-Incompatible C API change disallowing using macros as l-value to:
+Incompatible C API change disallowing using macros as an l-value to:
 
 * Allow evolving CPython internals;
 * Ease the C API implementation on other Python implementation;
 * Help migrating existing C extensions to the HPy API.
 
-On the PyPI top 5000 projects, only 14 projects (0.3%) are affected by 4
-macro changes. Moreover, 24 projects just have to regenerate their
-Cython code to use ``Py_SET_TYPE()``.
+Only 13 out of the top 5000 PyPI projects (0.3%) are affected by 2 macro
+changes. An additional 22 projects just have to regenerate their Cython
+code.
 
 In practice, the majority of affected projects only have to make two
 changes:
@@ -32,8 +32,8 @@ changes:
 Rationale
 =========
 
-Using a macro as a l-value
---------------------------
+Using a macro as a an l-value
+-----------------------------
 
 In the Python C API, some functions are implemented as macro because
 writing a macro is simpler than writing a regular function. If a macro
@@ -48,7 +48,7 @@ This macro can be used as a **r-value** to **get** an object type::
 
     type = Py_TYPE(object);
 
-It can also be used as **l-value** to **set** an object type::
+It can also be used as an **l-value** to **set** an object type::
 
     Py_TYPE(object) = new_type;
 
@@ -73,7 +73,7 @@ first to fix this C API compatibility issue. It is a concrete example of
 a Python optimization blocked indirectly by the C API.
 
 This issue was already fixed in Python 3.10: the ``Py_REFCNT()`` macro
-has been already modified to disallow using it as a l-value.
+has been already modified to disallow using it as an l-value.
 
 These statements are endorsed by Sam Gross (nogil developer).
 
@@ -119,8 +119,8 @@ These statements are endorsed by Tim Felgentreff (GraalVM Python developer).
 Specification
 =============
 
-Disallow using macros as l-value
---------------------------------
+Disallow using macros as an l-value
+-----------------------------------
 
 PyObject and PyVarObject macros
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -207,12 +207,6 @@ PyDateTime "GET" macros
 * ``PyDateTime_TIME_GET_SECOND()``
 * ``PyDateTime_TIME_GET_TZINFO()``
 
-PyDescr macros
-^^^^^^^^^^^^^^
-
-* ``PyDescr_NAME()``
-* ``PyDescr_TYPE()``
-
 Port C extensions to Python 3.11
 --------------------------------
 
@@ -231,57 +225,71 @@ losing support with older Python versions. The project provides a header
 file which provides ``Py_SET_REFCNT()``, ``Py_SET_TYPE()`` and
 ``Py_SET_SIZE()`` functions to Python 3.8 and older.
 
-PyTuple_GET_ITEM() and PyList_GET_ITEM()
-----------------------------------------
-
-The ``PyTuple_GET_ITEM()`` and ``PyList_GET_ITEM()`` macros are left
-unchanged.
-
-The code pattern ``&PyTuple_GET_ITEM(tuple, 0)`` and
-``&PyList_GET_ITEM(list, 0)`` is still commonly used to get access to
-the inner ``PyObject**`` array.
-
-Changing these macros is out of the scope of this PEP.
 
 
 Backwards Compatibility
 =======================
 
 The proposed C API changes are backward incompatible on purpose.
 
-At December 1, 2021, a code search on the PyPI top 5000 projects (4760
-projects in practice, others don't have a source archive) found that
-`only 14 projects are affected
-<https://bugs.python.org/issue45476#msg407456>`_ (0.3%):
+On January 26, 2022, a code search on the top 5000 PyPI projects (4762
+projects in practice; others don't have a source archive) found that
+only 13 projects are affected (0.3%):
 
+* Cython (0.29.26)
+* PyGObject (3.42.0)
 * datatable (1.0.0)
-* frozendict (2.1.1)
 * guppy3 (3.1.2)
-* M2Crypto (0.38.0)
-* mecab-python3 (1.0.4)
-* mypy (0.910)
-* Naked (0.1.31)
+* numpy (1.22.1)
 * pickle5 (0.0.12)
+* psycopg2 (2.9.3)
+* pycurl (7.44.1)
 * pysha3 (1.0.2)
 * python-snappy (0.6.0)
-* recordclass (0.16.3)
+* recordclass (0.17.1)
 * scipy (1.7.3)
 * zodbpickle (2.2.0)
-* zstd (1.5.0.2)
 
-These 14 projects only use 4 macros as l-value:
+Of these 13 projects, only 2 macros are used as an l-value:
+``Py_TYPE()`` and ``Py_SIZE()``.
 
-* ``PyDescr_NAME()`` and ``PyDescr_TYPE()`` (2 projects)
-* ``Py_SIZE()`` (8 projects)
-* ``Py_TYPE()`` (4 projects)
+An additional 22 projects just have to regenerate their Cython code to
+use ``Py_SET_TYPE()`` and ``Py_SET_SIZE()``.
 
-Moreover, `24 projects just have to regenerate their Cython code
-<https://bugs.python.org/issue45476#msg407416>`_ to use
-``Py_SET_TYPE()``.
+This change does not follow the :pep:`387` deprecation process. There is
+no known way to emit a deprecation warning only when a macro is used as
+an l-value, but not when it's used differently (ex: as a r-value).
 
-This change does not follow the :pep:`387` deprecation process. There is no
-known way to emit a deprecation warning only when a macro is used as a
-l-value, but not when it's used differently (ex: as a r-value).
+PyTuple_GET_ITEM() and PyList_GET_ITEM() are left unchanged
+-----------------------------------------------------------
+
+The ``PyTuple_GET_ITEM()`` and ``PyList_GET_ITEM()`` macros are left
+unchanged.
+
+The code patterns ``&PyTuple_GET_ITEM(tuple, 0)`` and
+``&PyList_GET_ITEM(list, 0)`` are still commonly used to get access to
+the inner ``PyObject**`` array.
+
+Changing these macros is out of the scope of this PEP.
+
+PyDescr_NAME() and PyDescr_TYPE() are left unchanged
+----------------------------------------------------
+
+The ``PyDescr_NAME()`` and ``PyDescr_TYPE()`` macros are left unchanged.
+
+These macros give access to ``PyDescrObject.d_name`` and
+``PyDescrObject.d_type`` members. They can be used as l-values to set
+these members.
+
+The SWIG project uses these macros as l-values to set these members. It
+would be possible to modify SWIG to prevent setting ``PyDescrObject``
+structure members directly, but it is not really worth it since the
+``PyDescrObject`` structure is not performance critical and is unlikely
+to change soon.
+
+See the `bpo-46538 <https://bugs.python.org/issue46538>`_ "[C API] Make
+the PyDescrObject structure opaque: PyDescr_NAME() and PyDescr_TYPE()"
+issue for more details.
 
 
 Relationship with the HPy project
@@ -332,8 +340,9 @@ extensions to HPy is expected to take a few years.
 
 This PEP proposes to make the C API "less bad" by fixing one problem
 which is clearily identified as causing practical issues: macros used as
-l-values.  This PEP only requires to update a minority of C extensions,
-and usually only a few lines need to be changed in impacted extensions.
+l-values.  This PEP only requires updating a minority of C
+extensions, and usually only a few lines need to be changed in impacted
+extensions.
 
 For example, NumPy 1.22 is made of 307,300 lines of C code, and adapting
 NumPy to the this PEP only modified 11 lines (use Py_SET_TYPE and
@@ -358,8 +367,8 @@ example, a ``Py_SET_TYPE()`` function has been added to Python 3.9 and
 the ``Py_TYPE()`` documentation now requires to use the
 ``Py_SET_TYPE()`` function to set an object type.
 
-If developers use macros as l-value, it's their responsibility when
-their code breaks, not the Python responsibility. We are operating under
+If developers use macros as an l-value, it's their responsibility when
+their code breaks, not Python's responsibility. We are operating under
 the consenting adults principle: we expect users of the Python C API to
 use it as documented and expect them to take care of the fallout, if
 things break when they don't.
@@ -370,9 +379,9 @@ API documentation. The majority of developers are only using CPython and
 so are not aware of compatibility issues with other Python
 implementations.
 
-Moreover, continuing to allow using macros as l-value does not help the
-HPy project and leaves the burden of emulating them on GraalVM's Python
-implementation.
+Moreover, continuing to allow using macros as an l-value does not help
+the HPy project, and leaves the burden of emulating them on GraalVM's
+Python implementation.
 
 
 Macros already modified
@@ -423,6 +432,15 @@ References
   <https://bugs.python.org/issue30459>`_ (May 2017)
 
 
+Version History
+===============
+
+* Version 3: No longer change PyDescr_TYPE() and PyDescr_NAME() macros
+* Version 2: Add "Relationship with the HPy project" section, remove
+  the PyPy section
+* Version 1: First public version
+
+
 Copyright
 =========