fix SystemError
raised from PyUnicodeDecodeError_Create
on PyPy 3.10
#3297
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In pydantic/pydantic-core#659 a
SystemError
is reported in the Pydantic test suite for PyPy 3.10:Turns out that this is down to our PyPy implementation of
PyUnicodeDecodeError_Create
. Looks like there's a behavioural change in Python 3.10 to error when callingPyObject_CallFunction
with a#
.There's a
PY_SSIZE_T_CLEAN
macro which must be defined to enable these formats, which has the effect in the C header of redirectingPyObject_CallFunction
to_PyObject_CallFunction_SizeT
.I don't think we can do this automatic redirection, because it's a breaking change that leads to UB as
PyObject_CallFunction
on Python 3.9 and older would accept#
formats but treat the length asc_int
instead ofPy_ssize_t
. So I've opted to just expose_PyObject_CallFunction_SizeT
and change our implementation to use that. I verified by hand this fixes the Pydantic test suite on PyPy 3.10, and continues to work on PyPy 3.9.A further note - it looks like on Python 3.13 the differences between
PyObject_CallFunction
and_PyObject_CallFunction_SizeT
will be removed (and the latter symbol removed from the header, but kept in the ABI). That's a problem for the future though 😄