-
-
Notifications
You must be signed in to change notification settings - Fork 30.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
gh-102433: Use inspect.getattr_static
in typing._ProtocolMeta.__instancecheck__
#103034
gh-102433: Use inspect.getattr_static
in typing._ProtocolMeta.__instancecheck__
#103034
Conversation
inspect.getattr_static
in typing._ProtocolMeta.__instancecheck__
inspect.getattr_static
in typing._ProtocolMeta.__instancecheck__
I'd be very interested to know if any maintainers of libraries that use type annotations at runtime have any concerns about this proposed change. Off the top of my head, I can think of:
|
I’m on vacation, please find another reviewer. |
Thanks for tagging me, I'll discuss with the pydantic team and respond in a few days. |
Thanks so much for the apropos ping, @AlexWaygood. You do the good typing work! Sadly, this...
...is a non-starter. That's measurably unacceptable degradation. The existing Python ≥ 3.11's renewed focus on runtime performance is germane here. Efficiency is no longer orthogonal to our goals in the Python ecosystem. Efficiency is increasingly a goal in and of itself. Do not sacrifice quantifiable speed at the golden altar of well-intended but ultimately futile attempts to force parity between static and runtime type-checkers. Static and runtime type-checkers will inevitably disagree. These are profoundly different understandings of what type-checking even means. Reducing the scope of that disagreement is useful – but preserving (or ideally improving) runtime performance is substantially more useful. Static type-checking serves Python's runtime needs by improving the robustness and reliability of Python at runtime. Static analysis is subordinate to runtime. Runtime always comes first. Not last. We're all here for what Python accomplishes at runtime. To paraphrase President Truman:
|
Should be fine for cattrs, we have separate build and run stages for the de/serializers. |
Thanks for the feedback @leycec! To be clear, I fully agree with your sentiment here, and have no wish to make the CPython runtime subservient to the needs of static typing:
I realise that in my summary of this PR at the top, I maybe should have been clearer about the bug that this PR is trying to fix. Ultimately this PR is first and foremost about fixing unexpected behaviour at runtime. Improving consistency between how static type checkers and the runtime see certain classes is a "nice to have" that comes along with the fix. Here is a demonstration of the user-visible bug that this PR is trying to fix (which has been reported to the CPython repo multiple times, so is clearly surprising behaviour for users). Properties, other descriptors, and REPL session demonstrating the bug as it exists on CPython `main`>>> import time
>>> from typing import runtime_checkable, Protocol
>>> @runtime_checkable
... class HasX(Protocol):
... x: int
...
>>> class A:
... @property
... def x(self):
... raise RuntimeError('no')
...
>>> class B:
... @property
... def x(self):
... time.sleep(3600)
... return 42
...
>>> class C:
... def __getattr__(self, name):
... raise RuntimeError('no')
...
>>> isinstance(A(), HasX)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\alexw\AppData\Local\Programs\Python\Python311\Lib\typing.py", line 1968, in __instancecheck__
if all(hasattr(instance, attr) and
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\alexw\AppData\Local\Programs\Python\Python311\Lib\typing.py", line 1968, in <genexpr>
if all(hasattr(instance, attr) and
^^^^^^^^^^^^^^^^^^^^^^^
File "<stdin>", line 4, in x
RuntimeError: no
>>> isinstance(C(), HasX)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\alexw\AppData\Local\Programs\Python\Python311\Lib\typing.py", line 1968, in __instancecheck__
if all(hasattr(instance, attr) and
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\alexw\AppData\Local\Programs\Python\Python311\Lib\typing.py", line 1968, in <genexpr>
if all(hasattr(instance, attr) and
^^^^^^^^^^^^^^^^^^^^^^^
File "<stdin>", line 3, in __getattr__
RuntimeError: no
>>> isinstance(B(), HasX) # this will take an hour to evaluate I don't like (at all) that this results in a 46% performance degradation for simple
|
There are three options here that I can see:
|
IMHO, correct behavior should always come first, then maybe good performance. As far as |
I'm pretty sure this shouldn't affect pydantic since we tend to avoid I'm therefore neutral on it. FWIW (and it's not worth much since it's probably too late), my personal opinion is that torturing |
FWIW I agree fully with this, and I'm quite glad that protocols are by default not runtime-checkable. But, as you say, it's almost certainly too late to change this now without going through a painful deprecation period that (in my opinion) would cause more trouble than it's worth. |
I agree with you both. The checking of run-time protocols would probably be best left to run-time type checkers, but getting rid of run-time checkable protocols would be a PITA at this point.
|
Note the recent comment on the issue reinforcing that the current behavior of
I agree that these are the options. I think the advice we would have to document in the case of (3) is basically un-followable. If you are doing I agree with @agronholm that we should prioritize correct behavior first, and then seek to optimize it. IMO correct behavior (purely from a runtime perspective, ignoring static typing) is that we should not execute arbitrary descriptor code in an |
I'm going to merge this once the CI passes. We've already merged two optimisation PRs into
There are two more PRs (which are slightly riskier) that have yet to be decided on:
If anybody has any opinions on either of those, feedback is very welcome! Once decisions have been taken on the two open PRs, I'll create another PR adding a separate NEWS entry summarising all of the performance work that's been done (and the internal changes to I'd also like to look into optimising |
Many thanks to everybody who took their time to provide feedback on this PR, I really appreciate it! |
Sorry for my delay replying here. 🙈 Thanks a lot for pinging me. 🙏 As FastAPI and my other tools heavily depend and interact with Pydantic (or copy its ideas), if this doesn't affect Pydantic (as Samuel confirmed above), it doesn't affect FastAPI and friends. 🚀 |
…a.__instancecheck__` (python#103034)
…a.__instancecheck__` (python#103034)
Changelog: ============ - Fix use of @deprecated on classes with __new__ but no __init__. - Fix regression in version 4.6.1 where comparing a generic class against a runtime-checkable protocol using isinstance() would cause AttributeError to be raised if using Python 3.7. - Change deprecated @runtime to formal API @runtime_checkable in the error message. - Fix regression in 4.6.0 where attempting to define a Protocol that was generic over a ParamSpec or a TypeVarTuple would cause TypeError to be raised. - typing_extensions is now documented at https://typing-extensions.readthedocs.io/en/latest/. - Add typing_extensions.Buffer, a marker class for buffer types, as proposed by PEP 688. Equivalent to collections.abc.Buffer in Python 3.12. - Backport two CPython PRs fixing various issues with typing.Literal: python/cpython#23294 and python/cpython#23383. Both CPython PRs were originally, and both were backported to Python >=3.9.1, but no earlier. - A side effect of one of the changes is that equality comparisons of Literal objects will now raise a TypeError if one of the Literal objects being compared has a mutable parameter. (Using mutable parameters with Literal is not supported by PEP 586 or by any major static type checkers.) - Literal is now reimplemented on all Python versions <= 3.10.0. - Backport CPython PR 26067, ensuring that isinstance() calls on protocols raise TypeError when the protocol is not decorated with @runtime_checkable. - Backport several significant performance improvements to runtime-checkable protocols that have been made in Python 3.12 (see python/cpython#74690 for details). - A side effect of one of the performance improvements is that the members of a runtime-checkable protocol are now considered "frozen" at runtime as soon as the class has been created. Monkey-patching attributes onto a runtime-checkable protocol will still work, but will have no impact on isinstance() checks comparing objects to the protocol. See "What's New in Python 3.12" for more details. - isinstance() checks against runtime-checkable protocols now use inspect.getattr_static() rather than hasattr() to lookup whether attributes exist (backporting python/cpython#103034). - Backport the ability to define __init__ methods on Protocol classes, a change made in Python 3.11 (originally implemented in python/cpython#31628 - Speedup isinstance(3, typing_extensions.SupportsIndex) by >10x on Python <3.12. - Add typing_extensions versions of SupportsInt, SupportsFloat, SupportsComplex, SupportsBytes, SupportsAbs and SupportsRound. These have the same semantics as the versions from the typing module, but isinstance() checks against the typing_extensions versions are >10x faster at runtime on Python <3.12. - Add __orig_bases__ to non-generic TypedDicts, call-based TypedDicts, and call-based NamedTuples. - Add typing_extensions.get_original_bases, a backport of types.get_original_bases, introduced in Python 3.12 (CPython PR python/cpython#101827, originally - This function should always produce correct results when called on classes constructed using features from typing_extensions. - Constructing a call-based TypedDict using keyword arguments for the fields now causes a DeprecationWarning to be emitted. This matches the behaviour of typing.TypedDict on 3.11 and 3.12. - Backport the implementation of NewType from 3.10 (where it is implemented as a class rather than a function). This allows user-defined NewTypes to be pickled. - Fix tests and import on Python 3.12, where typing.TypeVar can no longer be subclassed. - Add typing_extensions.TypeAliasType, a backport of typing.TypeAliasType from PEP 695. - Backport changes to the repr of typing.Unpack that were made in order to implement PEP 692 (backport of python/cpython#104048). (From OE-Core rev: 2b1d07c7deb4f0247765bc737fb11a1747143edf) Signed-off-by: Wang Mingyu <[email protected]> Signed-off-by: Alexandre Belloni <[email protected]>
Changelog: ============ - Fix use of @deprecated on classes with __new__ but no __init__. - Fix regression in version 4.6.1 where comparing a generic class against a runtime-checkable protocol using isinstance() would cause AttributeError to be raised if using Python 3.7. - Change deprecated @runtime to formal API @runtime_checkable in the error message. - Fix regression in 4.6.0 where attempting to define a Protocol that was generic over a ParamSpec or a TypeVarTuple would cause TypeError to be raised. - typing_extensions is now documented at https://typing-extensions.readthedocs.io/en/latest/. - Add typing_extensions.Buffer, a marker class for buffer types, as proposed by PEP 688. Equivalent to collections.abc.Buffer in Python 3.12. - Backport two CPython PRs fixing various issues with typing.Literal: python/cpython#23294 and python/cpython#23383. Both CPython PRs were originally, and both were backported to Python >=3.9.1, but no earlier. - A side effect of one of the changes is that equality comparisons of Literal objects will now raise a TypeError if one of the Literal objects being compared has a mutable parameter. (Using mutable parameters with Literal is not supported by PEP 586 or by any major static type checkers.) - Literal is now reimplemented on all Python versions <= 3.10.0. - Backport CPython PR 26067, ensuring that isinstance() calls on protocols raise TypeError when the protocol is not decorated with @runtime_checkable. - Backport several significant performance improvements to runtime-checkable protocols that have been made in Python 3.12 (see python/cpython#74690 for details). - A side effect of one of the performance improvements is that the members of a runtime-checkable protocol are now considered "frozen" at runtime as soon as the class has been created. Monkey-patching attributes onto a runtime-checkable protocol will still work, but will have no impact on isinstance() checks comparing objects to the protocol. See "What's New in Python 3.12" for more details. - isinstance() checks against runtime-checkable protocols now use inspect.getattr_static() rather than hasattr() to lookup whether attributes exist (backporting python/cpython#103034). - Backport the ability to define __init__ methods on Protocol classes, a change made in Python 3.11 (originally implemented in python/cpython#31628 - Speedup isinstance(3, typing_extensions.SupportsIndex) by >10x on Python <3.12. - Add typing_extensions versions of SupportsInt, SupportsFloat, SupportsComplex, SupportsBytes, SupportsAbs and SupportsRound. These have the same semantics as the versions from the typing module, but isinstance() checks against the typing_extensions versions are >10x faster at runtime on Python <3.12. - Add __orig_bases__ to non-generic TypedDicts, call-based TypedDicts, and call-based NamedTuples. - Add typing_extensions.get_original_bases, a backport of types.get_original_bases, introduced in Python 3.12 (CPython PR python/cpython#101827, originally - This function should always produce correct results when called on classes constructed using features from typing_extensions. - Constructing a call-based TypedDict using keyword arguments for the fields now causes a DeprecationWarning to be emitted. This matches the behaviour of typing.TypedDict on 3.11 and 3.12. - Backport the implementation of NewType from 3.10 (where it is implemented as a class rather than a function). This allows user-defined NewTypes to be pickled. - Fix tests and import on Python 3.12, where typing.TypeVar can no longer be subclassed. - Add typing_extensions.TypeAliasType, a backport of typing.TypeAliasType from PEP 695. - Backport changes to the repr of typing.Unpack that were made in order to implement PEP 692 (backport of python/cpython#104048). (From OE-Core rev: a37154b9166323d05cca970ebb37bee0d5250893) Signed-off-by: Wang Mingyu <[email protected]> Signed-off-by: Alexandre Belloni <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Changelog: ============ - Fix use of @deprecated on classes with __new__ but no __init__. - Fix regression in version 4.6.1 where comparing a generic class against a runtime-checkable protocol using isinstance() would cause AttributeError to be raised if using Python 3.7. - Change deprecated @runtime to formal API @runtime_checkable in the error message. - Fix regression in 4.6.0 where attempting to define a Protocol that was generic over a ParamSpec or a TypeVarTuple would cause TypeError to be raised. - typing_extensions is now documented at https://typing-extensions.readthedocs.io/en/latest/. - Add typing_extensions.Buffer, a marker class for buffer types, as proposed by PEP 688. Equivalent to collections.abc.Buffer in Python 3.12. - Backport two CPython PRs fixing various issues with typing.Literal: python/cpython#23294 and python/cpython#23383. Both CPython PRs were originally, and both were backported to Python >=3.9.1, but no earlier. - A side effect of one of the changes is that equality comparisons of Literal objects will now raise a TypeError if one of the Literal objects being compared has a mutable parameter. (Using mutable parameters with Literal is not supported by PEP 586 or by any major static type checkers.) - Literal is now reimplemented on all Python versions <= 3.10.0. - Backport CPython PR 26067, ensuring that isinstance() calls on protocols raise TypeError when the protocol is not decorated with @runtime_checkable. - Backport several significant performance improvements to runtime-checkable protocols that have been made in Python 3.12 (see python/cpython#74690 for details). - A side effect of one of the performance improvements is that the members of a runtime-checkable protocol are now considered "frozen" at runtime as soon as the class has been created. Monkey-patching attributes onto a runtime-checkable protocol will still work, but will have no impact on isinstance() checks comparing objects to the protocol. See "What's New in Python 3.12" for more details. - isinstance() checks against runtime-checkable protocols now use inspect.getattr_static() rather than hasattr() to lookup whether attributes exist (backporting python/cpython#103034). - Backport the ability to define __init__ methods on Protocol classes, a change made in Python 3.11 (originally implemented in python/cpython#31628 - Speedup isinstance(3, typing_extensions.SupportsIndex) by >10x on Python <3.12. - Add typing_extensions versions of SupportsInt, SupportsFloat, SupportsComplex, SupportsBytes, SupportsAbs and SupportsRound. These have the same semantics as the versions from the typing module, but isinstance() checks against the typing_extensions versions are >10x faster at runtime on Python <3.12. - Add __orig_bases__ to non-generic TypedDicts, call-based TypedDicts, and call-based NamedTuples. - Add typing_extensions.get_original_bases, a backport of types.get_original_bases, introduced in Python 3.12 (CPython PR python/cpython#101827, originally - This function should always produce correct results when called on classes constructed using features from typing_extensions. - Constructing a call-based TypedDict using keyword arguments for the fields now causes a DeprecationWarning to be emitted. This matches the behaviour of typing.TypedDict on 3.11 and 3.12. - Backport the implementation of NewType from 3.10 (where it is implemented as a class rather than a function). This allows user-defined NewTypes to be pickled. - Fix tests and import on Python 3.12, where typing.TypeVar can no longer be subclassed. - Add typing_extensions.TypeAliasType, a backport of typing.TypeAliasType from PEP 695. - Backport changes to the repr of typing.Unpack that were made in order to implement PEP 692 (backport of python/cpython#104048). Signed-off-by: Wang Mingyu <[email protected]> Signed-off-by: Alexandre Belloni <[email protected]>
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [typing-extensions](https://togithub.com/python/typing_extensions) ([changelog](https://togithub.com/python/typing_extensions/blob/main/CHANGELOG.md)) | `==4.5.0` -> `==4.6.3` | [![age](https://badges.renovateapi.com/packages/pypi/typing-extensions/4.6.3/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/pypi/typing-extensions/4.6.3/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/pypi/typing-extensions/4.6.3/compatibility-slim/4.5.0)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/pypi/typing-extensions/4.6.3/confidence-slim/4.5.0)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>python/typing_extensions</summary> ### [`v4.6.3`](https://togithub.com/python/typing_extensions/blob/HEAD/CHANGELOG.md#Release-463-June-1-2023) [Compare Source](https://togithub.com/python/typing_extensions/compare/4.6.2...4.6.3) - Fix a regression introduced in v4.6.0 in the implementation of runtime-checkable protocols. The regression meant that doing `class Foo(X, typing_extensions.Protocol)`, where `X` was a class that had `abc.ABCMeta` as its metaclass, would then cause subsequent `isinstance(1, X)` calls to erroneously raise `TypeError`. Patch by Alex Waygood (backporting the CPython[https://github.com/python/cpython/pull/105152](https://togithub.com/python/cpython/pull/105152)l/105152). - Sync the repository's LICENSE file with that of CPython. `typing_extensions` is distributed under the same license as CPython itself. - Skip a problematic test on Python 3.12.0b1. The test fails on 3.12.0b1 due to a bug in CPython, which will be fixed in 3.12.0b2. The `typing_extensions` test suite now passes on 3.12.0b1. ### [`v4.6.2`](https://togithub.com/python/typing_extensions/blob/HEAD/CHANGELOG.md#Release-462-May-25-2023) [Compare Source](https://togithub.com/python/typing_extensions/compare/4.6.1...4.6.2) - Fix use of `@deprecated` on classes with `__new__` but no `__init__`. Patch by Jelle Zijlstra. - Fix regression in version 4.6.1 where comparing a generic class against a runtime-checkable protocol using `isinstance()` would cause `AttributeError` to be raised if using Python 3.7. ### [`v4.6.1`](https://togithub.com/python/typing_extensions/blob/HEAD/CHANGELOG.md#Release-461-May-23-2023) [Compare Source](https://togithub.com/python/typing_extensions/compare/4.6.0...4.6.1) - Change deprecated `@runtime` to formal API `@runtime_checkable` in the error message. Patch by Xuehai Pan. - Fix regression in 4.6.0 where attempting to define a `Protocol` that was generic over a `ParamSpec` or a `TypeVarTuple` would cause `TypeError` to be raised. Patch by Alex Waygood. ### [`v4.6.0`](https://togithub.com/python/typing_extensions/blob/HEAD/CHANGELOG.md#Release-460-May-22-2023) [Compare Source](https://togithub.com/python/typing_extensions/compare/4.5.0...4.6.0) - `typing_extensions` is now documented at https://typing-extensions.readthedocs.io/en/latest/. Patch by Jelle Zijlstra. - Add `typing_extensions.Buffer`, a marker class for buffer types, as proposed by PEP 688. Equivalent to `collections.abc.Buffer` in Python 3.12. Patch by Jelle Zijlstra. - Backport two CPython PRs fixing various issues with `typing.Literal`: [https://github.com/python/cpython/pull/23294](https://togithub.com/python/cpython/pull/23294)3294 [https://github.com/python/cpython/pull/23383](https://togithub.com/python/cpython/pull/23383)ll/23383. Both CPython PRs were originally by Yurii Karabas, and both were backported to Python >=3.9.1, but no earlier. Patch by Alex Waygood. A side effect of one of the changes is that equality comparisons of `Literal` objects will now raise a `TypeError` if one of the `Literal` objects being compared has a mutable parameter. (Using mutable parameters with `Literal` is not supported by PEP 586 or by any major static type checkers.) - `Literal` is now reimplemented on all Python versions <= 3.10.0. The `typing_extensions` version does not suffer from the bug that was fixed in [https://github.com/python/cpython/pull/29334](https://togithub.com/python/cpython/pull/29334)9334. (The CPython bugfix was backported to CPython 3.10.1 and 3.9.8, but no earlier.) - Backport [CPython PR 26067](https://togithub.com/python/cpython/pull/26067) (originally by Yurii Karabas), ensuring that `isinstance()` calls on protocols raise `TypeError` when the protocol is not decorated with `@runtime_checkable`. Patch by Alex Waygood. - Backport several significant performance improvements to runtime-checkable protocols that have been made in Python 3.12 ([https://github.com/python/cpython/issues/74690](https://togithub.com/python/cpython/issues/74690)es/74690 for details). Patch by Alex Waygood. A side effect of one of the performance improvements is that the members of a runtime-checkable protocol are now considered “frozen” at runtime as soon as the class has been created. Monkey-patching attributes onto a runtime-checkable protocol will still work, but will have no impact on `isinstance()` checks comparing objects to the protocol. See ["What's New in Python 3.12"](https://docs.python.org/3.12/whatsnew/3.12.html#typing) for more details. - `isinstance()` checks against runtime-checkable protocols now use `inspect.getattr_static()` rather than `hasattr()` to lookup whether attributes exist (backport[https://github.com/python/cpython/pull/103034](https://togithub.com/python/cpython/pull/103034)3034). This means that descriptors and `__getattr__` methods are no longer unexpectedly evaluated during `isinstance()` checks against runtime-checkable protocols. However, it may also mean that some objects which used to be considered instances of a runtime-checkable protocol on older versions of `typing_extensions` may no longer be considered instances of that protocol using the new release, and vice versa. Most users are unlikely to be affected by this change. Patch by Alex Waygood. - Backport the ability to define `__init__` methods on Protocol classes, a change made in Python 3.11 (originally implemented[https://github.com/python/cpython/pull/31628](https://togithub.com/python/cpython/pull/31628)ll/31628 by Adrian Garcia Badaracco). Patch by Alex Waygood. - Speedup `isinstance(3, typing_extensions.SupportsIndex)` by >10x on Python <3.12. Patch by Alex Waygood. - Add `typing_extensions` versions of `SupportsInt`, `SupportsFloat`, `SupportsComplex`, `SupportsBytes`, `SupportsAbs` and `SupportsRound`. These have the same semantics as the versions from the `typing` module, but `isinstance()` checks against the `typing_extensions` versions are >10x faster at runtime on Python <3.12. Patch by Alex Waygood. - Add `__orig_bases__` to non-generic TypedDicts, call-based TypedDicts, and call-based NamedTuples. Other TypedDicts and NamedTuples already had the attribute. Patch by Adrian Garcia Badaracco. - Add `typing_extensions.get_original_bases`, a backport of [`types.get_original_bases`](https://docs.python.org/3.12/library/types.html#types.get_original_bases), introduced in Python 3.12 (CPython[https://github.com/python/cpython/pull/101827](https://togithub.com/python/cpython/pull/101827)l/101827, originally by James Hilton-Balfe). Patch by Alex Waygood. This function should always produce correct results when called on classes constructed using features from `typing_extensions`. However, it may produce incorrect results when called on some `NamedTuple` or `TypedDict` classes that use `typing.{NamedTuple,TypedDict}` on Python <=3.11. - Constructing a call-based `TypedDict` using keyword arguments for the fields now causes a `DeprecationWarning` to be emitted. This matches the behaviour of `typing.TypedDict` on 3.11 and 3.12. - Backport the implementation of `NewType` from 3.10 (where it is implemented as a class rather than a function). This allows user-defined `NewType`s to be pickled. Patch by Alex Waygood. - Fix tests and import on Python 3.12, where `typing.TypeVar` can no longer be subclassed. Patch by Jelle Zijlstra. - Add `typing_extensions.TypeAliasType`, a backport of `typing.TypeAliasType` from PEP 695. Patch by Jelle Zijlstra. - Backport changes to the repr of `typing.Unpack` that were made in order to implement [PEP 692](https://peps.python.org/pep-0692/) (backport of [https://github.com/python/cpython/pull/104048](https://togithub.com/python/cpython/pull/104048)4048). Patch by Alex Waygood. </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/allenporter/flux-local). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS4xMDUuMiIsInVwZGF0ZWRJblZlciI6IjM1LjEwNS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
It was caused by the new [4.6 version of typing-extensions](https://github.com/python/typing_extensions/blob/main/CHANGELOG.md#release-460-may-22-2023) used by jsonschema-spec to validate protocol. > isinstance() checks against runtime-checkable protocols now use inspect.getattr_static() rather than hasattr() to lookup whether attributes exist (backporting python/cpython#103034). This means that descriptors and getattr methods are no longer unexpectedly evaluated during isinstance() checks against runtime-checkable protocols. However, it may also mean that some objects which used to be considered instances of a runtime-checkable protocol on older versions of typing_extensions may no longer be considered instances of that protocol using the new release, and vice versa. Most users are unlikely to be affected by this change. Patch by Alex Waygood. Temporary workaround will be downgrading typing-extensions to version below 4.6: pip install "typing-extensions<4.6" Bug #50048
Changelog: ============ - Fix use of @deprecated on classes with __new__ but no __init__. - Fix regression in version 4.6.1 where comparing a generic class against a runtime-checkable protocol using isinstance() would cause AttributeError to be raised if using Python 3.7. - Change deprecated @runtime to formal API @runtime_checkable in the error message. - Fix regression in 4.6.0 where attempting to define a Protocol that was generic over a ParamSpec or a TypeVarTuple would cause TypeError to be raised. - typing_extensions is now documented at https://typing-extensions.readthedocs.io/en/latest/. - Add typing_extensions.Buffer, a marker class for buffer types, as proposed by PEP 688. Equivalent to collections.abc.Buffer in Python 3.12. - Backport two CPython PRs fixing various issues with typing.Literal: python/cpython#23294 and python/cpython#23383. Both CPython PRs were originally, and both were backported to Python >=3.9.1, but no earlier. - A side effect of one of the changes is that equality comparisons of Literal objects will now raise a TypeError if one of the Literal objects being compared has a mutable parameter. (Using mutable parameters with Literal is not supported by PEP 586 or by any major static type checkers.) - Literal is now reimplemented on all Python versions <= 3.10.0. - Backport CPython PR 26067, ensuring that isinstance() calls on protocols raise TypeError when the protocol is not decorated with @runtime_checkable. - Backport several significant performance improvements to runtime-checkable protocols that have been made in Python 3.12 (see python/cpython#74690 for details). - A side effect of one of the performance improvements is that the members of a runtime-checkable protocol are now considered "frozen" at runtime as soon as the class has been created. Monkey-patching attributes onto a runtime-checkable protocol will still work, but will have no impact on isinstance() checks comparing objects to the protocol. See "What's New in Python 3.12" for more details. - isinstance() checks against runtime-checkable protocols now use inspect.getattr_static() rather than hasattr() to lookup whether attributes exist (backporting python/cpython#103034). - Backport the ability to define __init__ methods on Protocol classes, a change made in Python 3.11 (originally implemented in python/cpython#31628 - Speedup isinstance(3, typing_extensions.SupportsIndex) by >10x on Python <3.12. - Add typing_extensions versions of SupportsInt, SupportsFloat, SupportsComplex, SupportsBytes, SupportsAbs and SupportsRound. These have the same semantics as the versions from the typing module, but isinstance() checks against the typing_extensions versions are >10x faster at runtime on Python <3.12. - Add __orig_bases__ to non-generic TypedDicts, call-based TypedDicts, and call-based NamedTuples. - Add typing_extensions.get_original_bases, a backport of types.get_original_bases, introduced in Python 3.12 (CPython PR python/cpython#101827, originally - This function should always produce correct results when called on classes constructed using features from typing_extensions. - Constructing a call-based TypedDict using keyword arguments for the fields now causes a DeprecationWarning to be emitted. This matches the behaviour of typing.TypedDict on 3.11 and 3.12. - Backport the implementation of NewType from 3.10 (where it is implemented as a class rather than a function). This allows user-defined NewTypes to be pickled. - Fix tests and import on Python 3.12, where typing.TypeVar can no longer be subclassed. - Add typing_extensions.TypeAliasType, a backport of typing.TypeAliasType from PEP 695. - Backport changes to the repr of typing.Unpack that were made in order to implement PEP 692 (backport of python/cpython#104048). (From OE-Core rev: a37154b9166323d05cca970ebb37bee0d5250893) Signed-off-by: Wang Mingyu <[email protected]> Signed-off-by: Alexandre Belloni <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
…python>=3.12.0` Any objects using `__getattr__`, e.g. `Parameter` break on older versions [see PR](python/cpython#103034)
Is this an intended/known change? Could it be noted in the changelog here as an example of case that will no longer work? |
@GDYendell, I will admit that I did indeed fail to consider how this would change the behaviour of runtime-checkable protocols when used in conjunction with bound instance methods that have extra attributes monkeypatched onto them. Can you possibly open a new issue about this? I haven't got the time to look at it right now, and comment threads on long-merged PRs are unfortunately likely to get forgotten about. |
Use
getattr_static
rather thanhasattr
in_ProtocolMeta.__instancecheck__
, in order to avoid unexpectedly evaluating descriptors and__getattr__
methods in the process of callingisinstance()
against runtime-checkable protocols.This slows down
isinstance()
checks against runtime-checkable protocols by around 46% (on my machine), but the consensus in python/typing#1363 is that the performance penalty is worth it because:isinstance()
check causing an expensive (or otherwise badly behaved) property or__getattr__
method to be evaluated is pretty surprising behaviour.@runtime_checkable
#102936.)In various subtle edge cases, this change will mean that some classes that used to be considered structural "subclasses" of a certain runtime-checkable protocol will no longer be considered subclasses of that protocol (and vice versa). As such, this can't be backported. (Perhaps we should consider documenting the existing behaviour for 3.11 and 3.10.)
isinstance
onruntime_checkable
Protocol
has side-effects for@property
methods #102433Note that most discussion has actually taken place in a
python/typing
issue rather than a CPython issue:isinstance()
work at runtime when comparing classes with properties against runtime-checkable protocols? typing#1363