From f5205c67e2ad239d77fe25514cf423084ec64ac0 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Wed, 2 Aug 2023 14:40:40 -0700 Subject: [PATCH 01/18] Update after Erik's feedback --- pep-0722.rst | 205 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 205 insertions(+) create mode 100644 pep-0722.rst diff --git a/pep-0722.rst b/pep-0722.rst new file mode 100644 index 00000000000..8f1cdb8192e --- /dev/null +++ b/pep-0722.rst @@ -0,0 +1,205 @@ +PEP: 722 +Title: Stricter Type Guards +Author: Rich Chiodo , Eric Traut , Erik De Bonte +Sponsor: +PEP-Delegate: +Discussions-To: https://github.com/python/typing/discussions/1013 +Status: Draft +Type: Standards Track +Topic: Typing +Content-Type: text/x-rst +Created: 28-Jul-2023 +Python-Version: 3.10 +Post-History: +Resolution: + + +Abstract +======== + +:pep:`647` created a special return type annotation ``TypeGuard`` that allowed +type checkers to narrow types. + +This PEP further refines :pep:`647` by allowing type checkers to narrow types +even further when a ``TypeGuard`` function returns false. + +Motivation +========== + +`TypeGuards `__ are used throughout Python libraries to allow a +type checker to narrow what the type of something is when the ``TypeGuard`` +returns true. + +.. code-block:: python + + def is_str(val: str | int) -> TypeGuard[str]: + return isinstance(val, str) + + def func(val: str | int): + if is_str(val): + # Type checkers can assume val is a 'str' in this branch + +However, in the ``else`` clause, :pep:`647` didn't prescribe what the type might +be: + +.. code-block:: python + + def is_str(val: str | int) -> TypeGuard[str]: + return isinstance(val, str) + + def func(val: str | int): + if is_str(val): + # Type checkers can assume val is a 'str' in this branch + else: + # Type here is not narrowed. It is still 'str | int' + + +This PEP proposes that when the type argument of the ``TypeGuard`` is a subtype +of the type of the first input argument, then the false case can be further +narrowed. + +This changes the example above like so: + +.. code-block:: python + + def is_str(val: str | int) -> TypeGuard[str]: + return isinstance(val, str) + + def func(val: str | int): + if is_str(val): + # Type checkers can assume val is a 'str' in this branch + else: + # Type checkers can assume val is an 'int' in this branch + +Since the ``TypeGuard`` type (or output type) is a subtype of the input argument +type, a type checker can determine that the only possible type in the ``else`` +is the other type in the Union. In this example, it is safe to assume that if +``is_str`` returns false, then type of the ``val`` argument is an ``int``. + +Unsafe Narrowing +-------------------- + +There are cases where this further type narrowing is not possible. Here's an +example: + +.. code-block:: python + + def is_str_list(val: list[int | str]) -> TypeGuard[list[str]] + return all(isinstance(x, str) for x in val) + + def func(val: list[int | str]): + if is_str_list(val): + # Type checker assumes list[str] here + else: + # Type checker cannot assume list[int] here + +Since ``list`` is invariant, it doesn't have any subtypes. This means type +checkers cannot narrow the type to ``list[int]`` in the false case. +``list[str]`` is not a subtype of ``list[str | int]``. + +Type checkers should not assume any narrowing in the false case when the +``TypeGuard`` type is not a subtype of the input argument type. + +However, narrowing in the true case is still possible. In the example above, the +type checker can assume the list is a ``list[str]`` if the ``TypeGuard`` +function returns true. + +Specification +============= + +This PEP requires no new changes to the language. It is merely modifying the +definition of ``TypeGuard`` for type checkers. Runtimes should already be +behaving in this way. + +Existing ``TypeGuard`` usage may change though, as described below. + + +Backwards Compatibility +======================= + +For preexisting code this should require no changes, but should simplify this +use case here: + +.. code-block:: python + + class A(): + pass + class B(): + pass + + def is_A(x: A | B) -> TypeGuard[A]: + return is_instance(x, A) + + + def is_B(x: A | B) -> TypeGuard[B]: + return is_instance(x, B) + + + def test(x: A | B): + if is_A(x): + # Do stuff assuming x is an 'A' + return + assert is_B(x) + + # Do stuff assuming x is a 'B' + return + + +This use case becomes this instead: + +.. code-block:: python + + class A(): + pass + class B(): + pass + + def is_A(x: A | B) -> TypeGuard[A]: + return is_instance(x, A) + + + def test(x: A | B): + if is_A(x): + # Do stuff assuming x is an 'A' + return + + # Do stuff assuming x is a 'B' + return + + +How to Teach This +================= + +The belief is that new users will assume this is how ``TypeGuard`` works in the +first place. Meaning this change should make ``TypeGuard`` easier to teach. + + +Reference Implementation +======================== + +A reference `implementation `__ of this idea exists in Pyright. + + +Rejected Ideas +============== + +Originally a new ``StrictTypeGuard`` construct was proposed. A +``StrictTypeGuard`` would be similar to to a ``TypeGuard`` except it would +explicitly state that output type was a subtype of the input type. Type checkers +would validate that the output type was a subtype of the input type. + +See this comment: `StrictTypeGuard proposal `__ + +This was rejected because for most cases it's not necessary. Most people assume +the negative case for ``TypeGuard`` anyway, so why not just change the +specification to match their assumptions? + +Footnotes +========= +.. _typeguards: https://peps.python.org/pep-0647/ + +Copyright +========= + +This document is placed in the public domain or under the CC0-1.0-Universal +license, whichever is more permissive. \ No newline at end of file From 00d524744efa49df31574593d4b686caa721582b Mon Sep 17 00:00:00 2001 From: rchiodo Date: Wed, 2 Aug 2023 14:40:40 -0700 Subject: [PATCH 02/18] Update after Erik's feedback --- pep-0722.rst | 142 ++++++++++++++++++++++++++++----------------------- 1 file changed, 77 insertions(+), 65 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index 98cfbf1d2dd..8f1cdb8192e 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -1,6 +1,6 @@ PEP: 722 Title: Stricter Type Guards -Author: Rich Chiodo , Eric Traut +Author: Rich Chiodo , Eric Traut , Erik De Bonte Sponsor: PEP-Delegate: Discussions-To: https://github.com/python/typing/discussions/1013 @@ -17,87 +17,99 @@ Resolution: Abstract ======== -This PEP further refines `TypeGuards `__ to -indicate when negative type narrowing is deemed safe. - -[I'd suggest mentioning PEP 647 explicitly here rather than having the opaque link. You can link to a PEP in RST using :pep:`647` ] -[I think more context is needed here for readers to understand what "negative" means. Maybe one sentence explaining what typeguards currently do and then another about the negative issue.] +:pep:`647` created a special return type annotation ``TypeGuard`` that allowed +type checkers to narrow types. +This PEP further refines :pep:`647` by allowing type checkers to narrow types +even further when a ``TypeGuard`` function returns false. Motivation ========== -`TypeGuards `__ are used throughout python -libraries but cannot be used to determine the negative case: +`TypeGuards `__ are used throughout Python libraries to allow a +type checker to narrow what the type of something is when the ``TypeGuard`` +returns true. + +.. code-block:: python -[Again, more context is needed for the user to understand what "the negative case" means.] -[Also what does "determine the negative case" mean? Maybe something like "narrow the type in the negative case" would be more clear? Also see the use of that phrase below the code block.] -[python should be capitalized] + def is_str(val: str | int) -> TypeGuard[str]: + return isinstance(val, str) + + def func(val: str | int): + if is_str(val): + # Type checkers can assume val is a 'str' in this branch -:: -[I'm wondering if `::` is equivalent to `.. code-block:: python` -- You may need the latter to get proper colorization. Check after you build your RST to HTML.] +However, in the ``else`` clause, :pep:`647` didn't prescribe what the type might +be: + +.. code-block:: python def is_str(val: str | int) -> TypeGuard[str]: return isinstance(val, str) def func(val: str | int): if is_str(val): - reveal_type(val) # str + # Type checkers can assume val is a 'str' in this branch else: - reveal_type(val) # str | int + # Type here is not narrowed. It is still 'str | int' + -This inability to determine the negative case makes ``TypeGuard`` not as useful as -it could be. +This PEP proposes that when the type argument of the ``TypeGuard`` is a subtype +of the type of the first input argument, then the false case can be further +narrowed. -This PEP proposes that in cases where the output type is a *strict* subtype of -the input type, the negative case can be computed. This changes the example so -that the ``int`` case is possible: -["output type" -- might need to define this term or use something else. I don't see that term used in PEP 647.] -["This changes the example" -- maybe rephrase this to clarify that the code of the example is unchanged, but type checkers can interpret it differently?] -["is possible" seems pretty vague] -[What does strict subtype mean? And why is it italicized?] +This changes the example above like so: -:: +.. code-block:: python def is_str(val: str | int) -> TypeGuard[str]: return isinstance(val, str) def func(val: str | int): if is_str(val): - reveal_type(val) # str + # Type checkers can assume val is a 'str' in this branch else: - reveal_type(val) # int + # Type checkers can assume val is an 'int' in this branch + +Since the ``TypeGuard`` type (or output type) is a subtype of the input argument +type, a type checker can determine that the only possible type in the ``else`` +is the other type in the Union. In this example, it is safe to assume that if +``is_str`` returns false, then type of the ``val`` argument is an ``int``. -Since the output type is a *strict* subtype of the -input, a type checker can determine that the only possible type in the ``else`` is the -other input type(s). -["the other input type(s)" -- There's only one input type. It's a Union. Suggest rephrasing this. I'm not sure if talking about the types using set theory (input -- output) would make this more clear (or more generic) or worse.] +Unsafe Narrowing +-------------------- -If the output type is not a *strict* subtype of the input type, -the negative cannot be assumed to be the intuitive opposite: -["intuitive opposite" -- opposite is the incorrect term here and I think intuition doesn't belong in a PEP :)] +There are cases where this further type narrowing is not possible. Here's an +example: -:: +.. code-block:: python def is_str_list(val: list[int | str]) -> TypeGuard[list[str]] return all(isinstance(x, str) for x in val) def func(val: list[int | str]): if is_str_list(val): - reveal_type(val) # list[str] + # Type checker assumes list[str] here else: - reveal_type(val) # list[str | int] + # Type checker cannot assume list[int] here -Since ``list`` is invariant, it doesn't have any subtypes, so type checkers -can't narrow the type in the negative case. +Since ``list`` is invariant, it doesn't have any subtypes. This means type +checkers cannot narrow the type to ``list[int]`` in the false case. +``list[str]`` is not a subtype of ``list[str | int]``. + +Type checkers should not assume any narrowing in the false case when the +``TypeGuard`` type is not a subtype of the input argument type. + +However, narrowing in the true case is still possible. In the example above, the +type checker can assume the list is a ``list[str]`` if the ``TypeGuard`` +function returns true. Specification ============= This PEP requires no new changes to the language. It is merely modifying the -definition of ``TypeGuard`` for type checkers. The runtime should already be +definition of ``TypeGuard`` for type checkers. Runtimes should already be behaving in this way. -["should" -- "The runtime" sounds singular, so if you mean CPython alone, I'd remove "should". If you mean that all Python runtimes should be behaving this way, I'd clarify that.] Existing ``TypeGuard`` usage may change though, as described below. @@ -108,49 +120,50 @@ Backwards Compatibility For preexisting code this should require no changes, but should simplify this use case here: -:: +.. code-block:: python - A = TypeVar("A") - B = TypeVar("B") + class A(): + pass + class B(): + pass def is_A(x: A | B) -> TypeGuard[A]: - raise NotImplementedError + return is_instance(x, A) - def after_is_A(x: A | B) -> TypeGuard[B]: - return True + def is_B(x: A | B) -> TypeGuard[B]: + return is_instance(x, B) def test(x: A | B): if is_A(x): - reveal_type(x) + # Do stuff assuming x is an 'A' return - assert after_is_A(x) + assert is_B(x) - reveal_type(x) + # Do stuff assuming x is a 'B' return -["after_is_A" is confusing me -- is there a better name? "is_not_A"?] -[Can/should you use PEP 695 syntax for the TypeVars?] -becomes this instead -["becomes this instead" is not a grammatically correct continuation of the sentence before the first code block. Maybe rephrase the sentence to "Preexisting code should require no changes, but code like this...can be simplified to this:"] -[Add comments in these code blocks showing the expected inferred type as you did above? I think then you won't need the reveal_type calls?] +This use case becomes this instead: -:: +.. code-block:: python - A = TypeVar("A") - B = TypeVar("B") + class A(): + pass + class B(): + pass def is_A(x: A | B) -> TypeGuard[A]: - return isinstance(x, A) + return is_instance(x, A) def test(x: A | B): if is_A(x): - reveal_type(x) + # Do stuff assuming x is an 'A' return - reveal_type(x) + + # Do stuff assuming x is a 'B' return @@ -164,8 +177,7 @@ first place. Meaning this change should make ``TypeGuard`` easier to teach. Reference Implementation ======================== -A reference implementation of this idea exists in Pyright. -[Would there be value in pointing the reader to the implementation?] +A reference `implementation `__ of this idea exists in Pyright. Rejected Ideas @@ -179,8 +191,8 @@ would validate that the output type was a subtype of the input type. See this comment: `StrictTypeGuard proposal `__ This was rejected because for most cases it's not necessary. Most people assume -the negative case for ``TypeGuard`` anyway, so why not just change the specification -to match their assumptions? +the negative case for ``TypeGuard`` anyway, so why not just change the +specification to match their assumptions? Footnotes ========= From 4c4a0e916579eb118617c6b58a3923873c137cf7 Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Thu, 3 Aug 2023 12:37:30 -0700 Subject: [PATCH 03/18] Update pep-0722.rst --- pep-0722.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index 8f1cdb8192e..da475f8ae5d 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -108,8 +108,8 @@ Specification ============= This PEP requires no new changes to the language. It is merely modifying the -definition of ``TypeGuard`` for type checkers. Runtimes should already be -behaving in this way. +definition of ``TypeGuard`` for type checkers. Runtimes are already behaving +in this way. Existing ``TypeGuard`` usage may change though, as described below. @@ -202,4 +202,4 @@ Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal -license, whichever is more permissive. \ No newline at end of file +license, whichever is more permissive. From 4f90e9cb3bae1b2c3d14aa5845489a76ab3a1f2e Mon Sep 17 00:00:00 2001 From: rchiodo Date: Thu, 3 Aug 2023 13:17:28 -0700 Subject: [PATCH 04/18] Add the deliberate mistake case --- pep-0722.rst | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/pep-0722.rst b/pep-0722.rst index 8f1cdb8192e..d89dd9b2724 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -104,6 +104,43 @@ However, narrowing in the true case is still possible. In the example above, the type checker can assume the list is a ``list[str]`` if the ``TypeGuard`` function returns true. +Creating invalid narrowing +-------------------------- + +The new ``else`` case for a ``TypeGuard`` can be setup incorrectly. Here's an example: + +.. code-block:: python + + def is_positive_int(val: int | str) -> TypeGuard[int]: + return isinstance(val, int) and val > 0 + + def func(val: int | str): + if is_positive_int(val): + # Type checker assumes int here + else: + # Type checker assumes str here + +A type checker will assume for the else case that the value is ``str``. This +is a change in behavior from :pep:`647` but as that pep stated `here `__ +there are many ways a determined or uninformed developer can subvert type safety. + +A better way to handle this example would be something like so: + +.. code-block:: python + + PosInt = NewType('PosInt', int) + + def is_positive_int(val: PosInt | int | str) -> TypeGuard[PosInt]: + return isinstance(val, int) and val > 0 + + def func(val: int | str): + if is_positive_int(val): + # Type checker assumes PosInt here + else: + # Type checker assumes str | int here + + + Specification ============= From f00e7a203542e9609811d690612163a5da8f0ef6 Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Thu, 3 Aug 2023 13:57:35 -0700 Subject: [PATCH 05/18] Update pep-0722.rst Co-authored-by: Erik De Bonte --- pep-0722.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index f4095ece657..d5482f316e0 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -17,8 +17,7 @@ Resolution: Abstract ======== -:pep:`647` created a special return type annotation ``TypeGuard`` that allowed -type checkers to narrow types. +:pep:`647` introduced the concept of ``TypeGuard`` functions which return true if their input parameter matches their target type. For example, a function that returns ``TypeGuard[str]`` is assumed to return ``true`` if and only if it's input parameter is a ``str``. This allows type checkers to narrow types in this positive case. This PEP further refines :pep:`647` by allowing type checkers to narrow types even further when a ``TypeGuard`` function returns false. From c895a0adb447f56dcb6a65f232caad43a0337b3d Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Thu, 3 Aug 2023 13:57:52 -0700 Subject: [PATCH 06/18] Update pep-0722.rst Co-authored-by: Erik De Bonte --- pep-0722.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0722.rst b/pep-0722.rst index d5482f316e0..e0c64cf6929 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -26,7 +26,7 @@ Motivation ========== `TypeGuards `__ are used throughout Python libraries to allow a -type checker to narrow what the type of something is when the ``TypeGuard`` +type checker to narrow the type of something when the ``TypeGuard`` returns true. .. code-block:: python From 5305af45f8aeadc759ee017a8acff535d680007d Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Thu, 3 Aug 2023 13:58:52 -0700 Subject: [PATCH 07/18] Update pep-0722.rst Co-authored-by: Erik De Bonte --- pep-0722.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index e0c64cf6929..ff83d5690c5 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -92,9 +92,8 @@ example: else: # Type checker cannot assume list[int] here -Since ``list`` is invariant, it doesn't have any subtypes. This means type +Since ``list`` is invariant, it doesn't have any subtypes. ``list[str]`` is not a subtype of ``list[str | int]``. This means type checkers cannot narrow the type to ``list[int]`` in the false case. -``list[str]`` is not a subtype of ``list[str | int]``. Type checkers should not assume any narrowing in the false case when the ``TypeGuard`` type is not a subtype of the input argument type. From b6335f07e73acd362b2f009825f49892f42eabad Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Thu, 3 Aug 2023 13:59:01 -0700 Subject: [PATCH 08/18] Update pep-0722.rst Co-authored-by: Erik De Bonte --- pep-0722.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index ff83d5690c5..5b716a8c2d2 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -152,8 +152,8 @@ Existing ``TypeGuard`` usage may change though, as described below. Backwards Compatibility ======================= -For preexisting code this should require no changes, but should simplify this -use case here: +For preexisting code this should require no changes, but will allow +use cases such as the one below to be simplified: .. code-block:: python From f24ca7cfbb9e8f54a0ea760621227a18db58c55f Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Thu, 3 Aug 2023 14:02:17 -0700 Subject: [PATCH 09/18] Update pep-0722.rst Co-authored-by: Erik De Bonte --- pep-0722.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0722.rst b/pep-0722.rst index 5b716a8c2d2..6024b56c68a 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -96,7 +96,7 @@ Since ``list`` is invariant, it doesn't have any subtypes. ``list[str]`` is not checkers cannot narrow the type to ``list[int]`` in the false case. Type checkers should not assume any narrowing in the false case when the -``TypeGuard`` type is not a subtype of the input argument type. +``TypeGuard`` type argument is not a subtype of the input argument type. However, narrowing in the true case is still possible. In the example above, the type checker can assume the list is a ``list[str]`` if the ``TypeGuard`` From cbf907a560825bae3d7a2973404256d958edaeb9 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Thu, 3 Aug 2023 14:03:14 -0700 Subject: [PATCH 10/18] More review feedback --- pep-0722.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/pep-0722.rst b/pep-0722.rst index 5b716a8c2d2..205d770bca8 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -180,7 +180,8 @@ use cases such as the one below to be simplified: return -This use case becomes this instead: +With this proposed change, the code above continues to work but could be +simplified by removing the assertion that x is of type B in the negative case: .. code-block:: python From e5a71ea171e7a06bbe6067e6311f8889ed97c4b7 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Thu, 3 Aug 2023 14:03:56 -0700 Subject: [PATCH 11/18] More review feedback --- pep-0722.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0722.rst b/pep-0722.rst index 8a607e7a820..254dab19c3e 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -72,7 +72,7 @@ This changes the example above like so: Since the ``TypeGuard`` type (or output type) is a subtype of the input argument type, a type checker can determine that the only possible type in the ``else`` -is the other type in the Union. In this example, it is safe to assume that if +is the other type in the ``Union``. In this example, it is safe to assume that if ``is_str`` returns false, then type of the ``val`` argument is an ``int``. Unsafe Narrowing From a32efa40d3acd3692736d5b0c0e4ad30c88d6bfd Mon Sep 17 00:00:00 2001 From: rchiodo Date: Thu, 3 Aug 2023 14:10:58 -0700 Subject: [PATCH 12/18] Fix 80 column limit --- pep-0722.rst | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index 254dab19c3e..e39a03d3905 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -17,7 +17,11 @@ Resolution: Abstract ======== -:pep:`647` introduced the concept of ``TypeGuard`` functions which return true if their input parameter matches their target type. For example, a function that returns ``TypeGuard[str]`` is assumed to return ``true`` if and only if it's input parameter is a ``str``. This allows type checkers to narrow types in this positive case. +:pep:`647` introduced the concept of ``TypeGuard`` functions which return true +if their input parameter matches their target type. For example, a function that +returns ``TypeGuard[str]`` is assumed to return ``true`` if and only if it's +input parameter is a ``str``. This allows type checkers to narrow types in this +positive case. This PEP further refines :pep:`647` by allowing type checkers to narrow types even further when a ``TypeGuard`` function returns false. @@ -72,8 +76,8 @@ This changes the example above like so: Since the ``TypeGuard`` type (or output type) is a subtype of the input argument type, a type checker can determine that the only possible type in the ``else`` -is the other type in the ``Union``. In this example, it is safe to assume that if -``is_str`` returns false, then type of the ``val`` argument is an ``int``. +is the other type in the ``Union``. In this example, it is safe to assume that +if ``is_str`` returns false, then type of the ``val`` argument is an ``int``. Unsafe Narrowing -------------------- @@ -92,8 +96,9 @@ example: else: # Type checker cannot assume list[int] here -Since ``list`` is invariant, it doesn't have any subtypes. ``list[str]`` is not a subtype of ``list[str | int]``. This means type -checkers cannot narrow the type to ``list[int]`` in the false case. +Since ``list`` is invariant, it doesn't have any subtypes. ``list[str]`` is not +a subtype of ``list[str | int]``. This means type checkers cannot narrow the +type to ``list[int]`` in the false case. Type checkers should not assume any narrowing in the false case when the ``TypeGuard`` type argument is not a subtype of the input argument type. @@ -105,7 +110,8 @@ function returns true. Creating invalid narrowing -------------------------- -The new ``else`` case for a ``TypeGuard`` can be setup incorrectly. Here's an example: +The new ``else`` case for a ``TypeGuard`` can be setup incorrectly. Here's an +example: .. code-block:: python @@ -120,7 +126,8 @@ The new ``else`` case for a ``TypeGuard`` can be setup incorrectly. Here's an ex A type checker will assume for the else case that the value is ``str``. This is a change in behavior from :pep:`647` but as that pep stated `here `__ -there are many ways a determined or uninformed developer can subvert type safety. +there are many ways a determined or uninformed developer can subvert +type safety. A better way to handle this example would be something like so: From 708f60c9cb21bb9a65d2c8887da9574f9c7d1fdc Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Thu, 3 Aug 2023 14:55:48 -0700 Subject: [PATCH 13/18] Update pep-0722.rst Co-authored-by: Erik De Bonte --- pep-0722.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index e39a03d3905..2cf59fae259 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -23,8 +23,8 @@ returns ``TypeGuard[str]`` is assumed to return ``true`` if and only if it's input parameter is a ``str``. This allows type checkers to narrow types in this positive case. -This PEP further refines :pep:`647` by allowing type checkers to narrow types -even further when a ``TypeGuard`` function returns false. +This PEP further refines :pep:`647` by allowing type checkers to also narrow types +when a ``TypeGuard`` function returns false. Motivation ========== From 662341faf516a111c96fc2e6a2a4cc658f914dc5 Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Thu, 3 Aug 2023 14:57:04 -0700 Subject: [PATCH 14/18] Update pep-0722.rst Co-authored-by: Erik De Bonte --- pep-0722.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0722.rst b/pep-0722.rst index 2cf59fae259..242d394924a 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -58,7 +58,7 @@ be: This PEP proposes that when the type argument of the ``TypeGuard`` is a subtype -of the type of the first input argument, then the false case can be further +of the type of the first input parameter, then the false case can be further narrowed. This changes the example above like so: From 24b88bab29fc5e181513d42d56461a858677f28b Mon Sep 17 00:00:00 2001 From: rchiodo Date: Thu, 3 Aug 2023 15:07:36 -0700 Subject: [PATCH 15/18] More feedback --- pep-0722.rst | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index 242d394924a..1baa7302015 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -79,6 +79,18 @@ type, a type checker can determine that the only possible type in the ``else`` is the other type in the ``Union``. In this example, it is safe to assume that if ``is_str`` returns false, then type of the ``val`` argument is an ``int``. + +Specification +============= + +This PEP requires no new changes to the language. It is merely modifying the +definition of ``TypeGuard`` for type checkers. Runtimes are already behaving +in this way. + + +Existing ``TypeGuard`` usage may change though, as described in +`Backwards Compatibility`_. + Unsafe Narrowing -------------------- @@ -145,17 +157,6 @@ A better way to handle this example would be something like so: # Type checker assumes str | int here - -Specification -============= - -This PEP requires no new changes to the language. It is merely modifying the -definition of ``TypeGuard`` for type checkers. Runtimes are already behaving -in this way. - -Existing ``TypeGuard`` usage may change though, as described below. - - Backwards Compatibility ======================= From a6822195fe5ca01f895c05eaa778b14749914121 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Thu, 3 Aug 2023 15:10:58 -0700 Subject: [PATCH 16/18] Review feedback --- pep-0722.rst | 9 --------- 1 file changed, 9 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index 1baa7302015..0ee3c2d74bb 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -33,15 +33,6 @@ Motivation type checker to narrow the type of something when the ``TypeGuard`` returns true. -.. code-block:: python - - def is_str(val: str | int) -> TypeGuard[str]: - return isinstance(val, str) - - def func(val: str | int): - if is_str(val): - # Type checkers can assume val is a 'str' in this branch - However, in the ``else`` clause, :pep:`647` didn't prescribe what the type might be: From 6287be5c7dac1aeb9b7d9c326ed141e29270382b Mon Sep 17 00:00:00 2001 From: rchiodo Date: Thu, 3 Aug 2023 15:36:46 -0700 Subject: [PATCH 17/18] Some more subtle word changes --- pep-0722.rst | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index 0ee3c2d74bb..9a9a90d3176 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -49,8 +49,8 @@ be: This PEP proposes that when the type argument of the ``TypeGuard`` is a subtype -of the type of the first input parameter, then the false case can be further -narrowed. +of the type of the first input parameter, then the false return case can be +further narrowed. This changes the example above like so: @@ -65,10 +65,11 @@ This changes the example above like so: else: # Type checkers can assume val is an 'int' in this branch -Since the ``TypeGuard`` type (or output type) is a subtype of the input argument -type, a type checker can determine that the only possible type in the ``else`` -is the other type in the ``Union``. In this example, it is safe to assume that -if ``is_str`` returns false, then type of the ``val`` argument is an ``int``. +Since the ``TypeGuard`` type (or output type) is a subtype of the first input +parameter type, a type checker can determine that the only possible type in the +``else`` is the other type in the ``Union``. In this example, it is safe to +assume that if ``is_str`` returns false, then type of the ``val`` argument is an +``int``. Specification @@ -104,7 +105,7 @@ a subtype of ``list[str | int]``. This means type checkers cannot narrow the type to ``list[int]`` in the false case. Type checkers should not assume any narrowing in the false case when the -``TypeGuard`` type argument is not a subtype of the input argument type. +``TypeGuard`` type argument is not a subtype of the first input parameter type. However, narrowing in the true case is still possible. In the example above, the type checker can assume the list is a ``list[str]`` if the ``TypeGuard`` From a2e0b22bce007ac1437ef3a1217244269cab4d50 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Thu, 3 Aug 2023 15:48:04 -0700 Subject: [PATCH 18/18] Change verbiage a little more --- pep-0722.rst | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index 9a9a90d3176..1d7502d22a6 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -86,8 +86,8 @@ Existing ``TypeGuard`` usage may change though, as described in Unsafe Narrowing -------------------- -There are cases where this further type narrowing is not possible. Here's an -example: +There are cases where this further type narrowing is not possible though. Here's +an example: .. code-block:: python @@ -111,7 +111,7 @@ However, narrowing in the true case is still possible. In the example above, the type checker can assume the list is a ``list[str]`` if the ``TypeGuard`` function returns true. -Creating invalid narrowing +User error -------------------------- The new ``else`` case for a ``TypeGuard`` can be setup incorrectly. Here's an @@ -152,8 +152,9 @@ A better way to handle this example would be something like so: Backwards Compatibility ======================= -For preexisting code this should require no changes, but will allow -use cases such as the one below to be simplified: +For preexisting code this PEP should require no changes. + +However, some use cases such as the one below can be simplified: .. code-block:: python