From c5b8a433548de559a69c5db591402af39197847f Mon Sep 17 00:00:00 2001 From: Michael Lee Date: Mon, 19 Nov 2018 15:45:01 -0800 Subject: [PATCH 01/18] Add first draft for literal types --- pep-9999.rst | 743 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 743 insertions(+) create mode 100644 pep-9999.rst diff --git a/pep-9999.rst b/pep-9999.rst new file mode 100644 index 00000000000..34406cdb7e3 --- /dev/null +++ b/pep-9999.rst @@ -0,0 +1,743 @@ +PEP: 9999 +Title: Literal types +Author: Michael Lee +Status: Draft +Type: Standards Track +Python-Version: 3.8 +Content-Type: text/x-rst +Created: 19-Nov-2018 +Post-History: 19-Nov-2018 + +Abstract +======== + +This PEP proposes to extend the `PEP 484`_ typing ecosystem by adding +*Literal types*, which indicate that some expression has literally a +specific value. For example, the following function will accept only +expressions that literally have the value "4":: + + from typing import Literal + + def accepts_only_four(x: Literal[4]) -> None: + pass + + accepts_only_four(4) # Type checker accepts + accepts_only_four(8) # Type checker rejects + +**Note:** This PEP should be considered to be very early draft: we plan +on refining it as we add corresponding proof-of-concept implementation +to mypy. + +Motivation and Rationale +======================== + +Python has many APIs that return different types depending on the exact +value of some argument provided. For example: + +- ``open(filename, mode)`` can return an ``IO[bytes]`` or ``IO[Text]`` + depending on whether the second argument is something like “r” or + “rb” +- ``subprocess.check_output(…)`` can also return bytes or text + depending on whether the ``universal_newlines`` keyword argument is + set to ``True`` or not. + +This pattern is also fairly common in many popular 3rd party libraries. +For example, here are just two examples from pandas and numpy respectively: + +- ``pandas.concat(...)`` will return either ``Series`` or + ``DataFrame`` depending on whether the ``axis`` argument is set to + 0 or 1. + +- ``numpy.unique`` will return either a single array or a tuple containing + anywhere from two to four arrays depending on three boolean flag values. + +The typing issue tracker contains some +`additional examples and discussion `. + +There is currently no way of expressing the type signatures of these +functions: PEP 484 does not include any mechanism for writing signatures +where the return type varies depending on the value passed in. +Note that this problem does not go away even if we redesign these APIs to +accept enums instead of string or int literals: the expressions ``MyEnum.FOO`` +and ``MyEnum.BAR`` are both considered to be of type ``MyEnum``.) + +Currently, type checkers work around this limitation by adding ad-hoc +extensions for important builtins and standard library functions. For +example mypy comes bundled with a plugin that attempts to infer more +precise types for ``open(…)``. While this approach works for standard +library functions, it’s unsustainable in general: it’s not reasonable to +expect 3rd party library authors to maintain plugins for N different +type checkers, for example. + +It is, of course, debatable whether these APIs are actually good design: +type checking would be simplified if we we could split functions like +``open(...)`` into two. However, this question is to a large degree +moot: there already exist many widely-used libraries and functions that +vary their behavior based on the provided values. Therefore, we believe that +it is important for the PEP 484 type ecosystem to be updated to match. + +We propose to add *Literal types* to address exactly this gap. + +Specification +============= + +Core Semantics +-------------- + +This section of the doc outlines the baseline behavior of literal types. + +Core behavior +''''''''''''' + +Literal types let us indicate that a variable has a specific and +concrete value. For example, if we define some variable ``foo`` to have +type ``Literal[3]``, we are declaring that ``foo`` must be exactly equal +to ``3`` and no other value. + +Given some value ``V`` that is a member of type ``T``, the type +``Literal[V]`` shall be treated as a subtype of ``T``. For example, +``Literal[3]`` is a subtype of ``int``. + +All of the methods of the parent type will be directly inherited by the +literal type. This means that if we have some variable ``foo`` of type +``Literal[3]``, it’s safe to do things like ``foo + 5`` since ``foo`` +ultimately inherits int’s ``__add__`` method. The resulting type of +``foo + 5`` will be ``int``. + +This “inheriting” behavior is identical to how we handle NewTypes. + +Equivalence of two Literals +''''''''''''''''''''''''''' + +Suppose we have two literal types ``Literal[A]`` and ``Literal[B]``. +Both types are considered equivalent when both of the following +conditions are true: + +1. ``type(A) == type(B)`` +2. ``A == B`` + +So for example, ``Literal[20]`` and ``Literal[0x14]`` would be +equivalent. However, ``Literal[0]`` and ``Literal[False]`` would *not* +be considered equivalent despite the fact that ``0 == False`` evaluates +to ‘true’ at runtime: ``0`` has type ``int`` and ``False`` has type +``bool``. + +Shortening unions of literals +''''''''''''''''''''''''''''' + +Literals may be parameterized with one or more values. When a Literal is +parameterized with more then one value, it is treated as being exactly +equivalent to the union of those values. That is, +``Literal[V1, V2, V3]`` is equivalent to +``Union[Literal[V1], Literal[V2], Literal[V3]]``. + +This shortcut can make writing signatures for functions that can accept +many different literals more ergonomic — for example, functions like +``open(…)``:: + + # Note: this is an over-simplification of the true type signature. + _PathType = Union[str, bytes, int] + + @overload + def open(path: _PathType, + mode: Literal["r", "w", "a", "x", "r+", "w+", "a+", "x+"], + ) -> IO[Text]: ... + @overload + def open(path: _PathType, + mode: Literal["rb", "wb", "ab", "xb", "r+b", "w+b", "a+b", "x+b"], + ) -> IO[bytes]: ... + + # Fallback overload for when the user isn't using literal types + @overload + def open(path: _PathType, mode: str) -> IO[Any]: ... + +**Note:** Literals **must** be parameterized with at least one type. +Types like ``Literal[]`` or ``Literal`` should be rejected by the type +checker. + +Legal and illegal parameterizations +----------------------------------- + +This section of the doc describes exactly which values may or may not +parameterize a ``Literal[…]`` type. + +Legal parameters for ``Literal`` at type check time +''''''''''''''''''''''''''''''''''''''''''''''''''' + +``Literal`` may be parameterized with literal ints, native strings, +bools, Enum values, and ``None``. So for example, all of the following +would be legal:: + + Literal[26] + Literal[0x1A] # Exactly equivalent to Literal[26] + Literal["hello world"] + Literal[True] + Literal[Color.RED] # Assuming Color is some enum + Literal[None] + +**Note 1:** The type ``Literal[None]`` is redundant in that the type +``None`` has only a single inhabitant. We nevertheless allow this mostly +for consistency and ease-of-use. For example, when writing a literal +with multiple parameters, it might look a little cleaner to do +``Literal[1, 2, 3, None]`` instead of ``Optional[Literal[1, 2, 3]]``. + +**Note 2:** The above list contains the bare minimum types necessary to +make literal types useful. We may add more types to this list in the +future based on demand — see below. + +Illegal parameters for ``Literal`` at type check time +''''''''''''''''''''''''''''''''''''''''''''''''''''' + +The following parameters are provisionally disallowed, mostly for +simplicity. We can consider adding these to the above list on a +case-by-case basis based on demand. + +- Explicit byte strings: e.g. ``Literal[b'foo']`` + +- Explicit unicode strings: e.g. ``Literal[u'foo']`` + +- Floats: e.g. ``Literal[3.14]`` Note: if we do decide to allow + floats, we should likely disallow literal infinity and literal NaN. + +- Any: e.g. ``Literal[Any]`` Note: the semantics of what exactly + ``Literal[Any]`` means would need to be clarified first. + +- Literal types themselves (or aliases to literal types) e.g. if we + create a type alias ``BasicIds = Literal[1, 2, 3]``, then perhaps + ``Literal[100, BasicIds]`` should be treated as being equivalent to + ``Literal[100, 1, 2, 3]``. + +The following parameters are intentionally disallowed by design. We will +most likely never add these parameters at a future date, unless somebody +is able to come up with an extremely compelling argument to the +contrary. + +- Arbitrary expressions like ``Literal[3 + 4]`` or + ``Literal["foo".replace("o", "b")]``. Literal types are meant to be a simple and + minimal extension to the PEP 484 typing ecosystem, and requiring type + checkers to interpret potentially expressions inside types adds too + much complexity to this proposal. Also see the + `Rejected or out-of-scope ideas`_ section of this doc below. + +- Complex numbers like ``Literal[4 + 3j]``, ``Literal[-4 + 2j]``, and + ``Literal[5j]``. Types like ``Literal[4 + 3j]`` would violate the + previous rule; it would then be consistent to also not allow types + like ``Literal[4j]``. + +- Tuples containing valid literal types — so ``Literal[(1, "foo", "bar")]`` + would be disallowed. The user could always express this type as + ``Tuple[Literal[1], Literal["foo"], Literal["bar"]]`` instead. Also, + tuples are likely to be confused with the ``Literal[1, 2, 3]`` + shortcut. + +- Mutable literal data structures like dict literals, list literals, or + set literals: literals are always implicitly final and immutable. So, + ``Literal[{"a": "b", "c": "d"}]`` would be disallowed. + +- Any other types. So, things like ``Literal[MyTypedDict]``, or + ``Literal[some_object_instance]`` would be disallowed. + This includes typevars: if ``T`` is a typevar, types like + ``Literal[T]`` is not allowed. Typevars can vary over only types, not + over values. + +Parameters at runtime +''''''''''''''''''''' + +The set of allowable parameters for ``Literal[...]`` is currently +deliberately very strict and limited. However, we may want to extend the +set of allowable parameters in the future or extend the behavior of +``Literal`` in other ways. + +To help us retain this flexibility, the actual implementation of +``typing.Literal`` shall perform *no* checks on any parameters provided +at runtime. For example:: + + def my_function(x: Literal[1 + 2]) -> None: + pass + + x: Literal = 3 + y: Literal[my_function] = my_funcion + +The type checker should reject this program: all three uses of +``Literal`` are *invalid* according to this spec. However, Python itself +should execute this program with no errors. + +Literals, enums, and forward references +''''''''''''''''''''''''''''''''''''''' + +One potential point of ambiguity is between literal strings and forward +references to literal enum members. For example, suppose we have the +type ``Literal["Color.RED"]``. Does this literal type +contain a string literal, or a forward reference to some ``Color.RED`` +enum member? + +In cases like these, we will always assume the user meant to construct a +literal string. If the user wants a forward reference, they must wrap +the entire literal type as a string -- e.g. ``Literal[Color.RED]``. + +The other alternative is to just not allow literal enums and avoid the +ambiguity altogether, but it seems a shame to give them up. + +Literals, enums, and Any +'''''''''''''''''''''''' + +Another point of ambiguity is when the user attempts to use some +expression that is meant to be an enum, but is actually of type ‘Any’. +This can happen when the user, for example, attempts to import an enum +with no type stubs:: + + from typing import Literal + from lib_with_no_types import SomeEnum # SomeEnum has type 'Any'! + + # Signature is equivalent to `func(x: Literal[Any]) -> None` + # due to the bad import + def func(x: Literal[SomeEnum.FOO]) -> None: pass + +In this case, should a type checker report an error with ``func``? On +one hand, it makes sense to allow this: it’s usually safe to substitute +``Any`` anywhere where a type is expected, and we don’t normally report +errors if a function attempts to use a type that was inferred to be +equivalent to ``Any``. + +On the other, ``Literal[…]`` expects a value, not a type, and ``Any`` is +*not* meant to represent a placeholder for any arbitrary *value*. The +semantics of what ``Literal[Any]`` means is also somewhat unclear — so, +we’ve tentatively decided to disallow ``Literal[Any]`` for now. + +So in this example, the type checker should reject the signature of +``func`` for using a bad ``Literal`` type. + +This decision is provisional and may be changed at a future date. + +Inferring types for literal expressions +--------------------------------------- + +This section of the doc describes how to infer the correct type for +literal expressions — e.g. under what circumstances literal expressions +like ``"foo"`` should be assumed to be of type ``Literal["foo"]`` vs of +type ``str``. + +In general, type checkers are expected to infer ``Literal[…]`` +conservatively and only in contexts where a ``Literal[…]`` type is +explicitly requested. See the following subsections for examples of what +this looks like. + +Variable assignment +''''''''''''''''''' + +When assigning a literal expression to an unannotated variable, the +inferred type of the variable should be the original base type, not a +``Literal[…]`` type. For example, consider the following snippet of +code:: + + foo = "hello" + reveal_type(foo) # Revealed type is 'str' + +We want to avoid breaking the semantics of any existing code, so the +inferred type of ``foo`` should be ``str``, **not** +``Literal["hello"]``. + +If the user wants ``foo`` to have a literal type, they must either +explicitly add an annotation:: + + foo: Literal["hello"] = "hello" + reveal_types(foo) # Revealed type is 'Literal["hello"]' + +Or alternatively, use the ``Final`` qualifier:: + + foo: Final = "hello" + reveal_types(foo) # Revealed type is 'Final[Literal["hello"]]' + +The ``Final`` qualifier will automatically infer a ``Literal`` type in +an assignment if the LHS is a literal expression, or an expression of +type ``Literal[…]``. + +**TODO:** Link to the PEP draft for the ``Final`` qualifier once it's ready. + +**Note 1:** A potential third way of declaring a variable to be Literal +might be to also try using ``Literal`` as a qualifier, like so:: + + foo: Literal = "hello" # Illegal! + +Although this spelling looks reasonable, we ultimately decided that type +checkers should *reject* assignments like: constructs like ``Final`` or +``ClassVar`` are *qualifiers* and so infer their parameters, but +``Literal`` is a *type*, and types traditionally substitute in ``Any`` +when their parameters are missing (e.g. ``List`` is the same as +``List[Any]``). In this case, we disallow ``Literal[Any]`` so the type +checker should report an error instead. + +**Note 2:** It may in some cases be possible to use the overall context +of the current scope to determine whether some variable should have a +Literal type or not. For example, in the following function, ``foo`` is +only ever used as an input to a function that expects ``Literal["blah"]`` +which means it’s theoretically possible to infer that foo has type +``Literal["blah"]``:: + + def expects_blah(x: Literal["blah"]) -> None: ... + + def test() -> None: + foo = "blah" + expects_blah(foo) + +Type checkers are **not** expected to handle these cases. It is, +however, an open question as to whether they are permitted to *try* on a +best-effort basis. E.g. is a PEP 484 compliant type checker always +*obligated* to infer that ``foo`` has type ``str``, or is it ok for them +to attempt more complex inference if they so choose? + +Type inference inside calls +''''''''''''''''''''''''''' + +When a literal is used inside of some function call, it will be inferred +as either the original type or the Literal type based on context. For +example, the following code snippet should be legal:: + + def expects_str(x: str) -> None: ... + def expects_literal(x: Literal["foo"]) -> None: ... + + # Legal: "foo" is inferred to be of type 'str' + expects_str("foo") + + # Legal: "foo" is inferred to be of type 'Literal["foo"]' + expects_literal("foo") + +However, other expressions in general will not automatically be inferred +to be literals. For example:: + + def expects_literal(x: Literal["foo"]) -> None: ... + + def runner(my_str: str) -> None: + # ILLEGAL: str is not a subclass of Literal["foo"] + expects_literal(my_str) + +**Note:** If the user wants their API to support accepting both literals +*and* the original type — perhaps for legacy purposes — they should +implement a fallback overload. See the section below on +`Interactions with overloads`_ for more details. + +Miscellaneous interactions +-------------------------- + +This section of the doc discusses how literal types ought to interact +with other aspects of the PEP 484 type system. + +Intelligent indexing of structured data: Interactions with TypedDict, Tuple, NamedTuples, and getattr +''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +The type checker should support "intelligently indexing" into structured +types like TypedDicts, NamedTuple, and classes when using string and int +literal keys. This list is non-exhaustive — there may be other examples +of structured types not included in this list. + +For example, if you try indexing into a TypedDict using a string +Literal, the type checker should return the correct value type if the +key is a member of the TypedDict (and return an error if it isn’t):: + + Foo = TypedDict('Foo', { + 'key1': int, + 'key2': str, + 'key3': List[bool], + }) + + a: Final = "key1" + b: Final = "some other string" + + f: Foo + reveal_type(f[a]) # Revealed type is 'int' + f[b] # Error: 'Foo' does not contain a key named 'some other string' + +We require similar behavior when indexing into a tuple or named tuple:: + + a: Final = 0 + b: Final = 5 + + some_tuple: Tuple[int, str, List[bool]] = (3, "abc", [True, False]) + reveal_type(some_tuple[a]) # Revealed type is 'int' + some_tuple[b] # Error: 5 is not a valid index into the tuple + +...and when using functions like getattr: + + class Test: + def __init__(self, param: int) -> None: + self.myfield = param + + def mymethod(self, val: int) -> str: ... + + a: Literal = "myfield" + b: Literal = "mymethod" + c: Literal = "blah" + + t = Test() + reveal_type(getattr(t, a)) # Revealed type is 'int' + reveal_type(getattr(t, b)) # Revealed type is 'Callable[[int], str]' + getattr(t, c) # Error: 'Test' does not have attribute named 'blah' + +These interactions will most likely need to be added to type checkers on +an ad-hoc basis: e.g. we special-case TypedDicts and NamedTuples, +special-case functions like ``getattr``... + +One potential alternative solution would be to implement something +similar to TypeScript’s `index types ` +and ``keyof`` operator, which lets you encode the idea that some key +(e.g. a literal string) is a member of some object. + +We currently do not plan on adding a similar concept to Python. Python +has many different kinds of structured data beyond just objects +(classes, objects, TypedDict, tuples, NamedTuples…) and it’s unclear +what the ramifications of attempting to unify all these different +concepts using this idea might be. This idea (or a similar one) may +potentially be revisited in the future, but not as a part of this PEP. + +Interactions with overloads +''''''''''''''''''''''''''' + +Literal types and overloads should not need to interact in any +particularly special way: the existing rule for how we handle subtypes +and unions ought to work fine. + +However, one important use case we must make sure will work is the +ability to specify a *fallback* when the user is not using literal +types. For example, consider the ``open`` function from before:: + + # Note: this is an over-simplification of the true type signature. + _PathType = Union[str, bytes, int] + + @overload + def open(path: _PathType, + mode: Literal["r", "w", "a", "x", "r+", "w+", "a+", "x+"], + ) -> IO[Text]: ... + @overload + def open(path: _PathType, + mode: Literal["rb", "wb", "ab", "xb", "r+b", "w+b", "a+b", "x+b"], + ) -> IO[bytes]: ... + + # Fallback overload for when the user isn't using literal types + @overload + def open(path: _PathType, mode: str) -> IO[Any]: ... + +If we changed the signature of ``open`` in typeshed so it uses just the +first two overloads, we could potentially end up breaking some clients +who are not passing in a literal string in as a first argument -- +e.g. clients who do something like this:: + + mode: str = pick_file_mode(...) + with open(path, mode) as f: + # f should continue to be of type IO[Any] here + +A little more broadly: we propose adding a policy to typeshed that +mandates that whenever we add literal types to some existing API, we also +include a fallback overload to let clients who happen to not be using +literal types continue using the API as they were before. Literal types +should always be added to existing APIs in backwards-compatible ways. + +Interactions with generics +'''''''''''''''''''''''''' + +Types like ``Literal[3]`` are meant to be just plain old subclasses of +``int``. Consequently, you can use types like ``Literal[3]`` anywhere +you could use normal types, such as with generics. + +For example, suppose we want to construct a type representing a +2-dimensional Matrix which can be parameterized by two literal ints +representing the number of rows and columns respectively. Such a type +could be built using the existing generics system like so:: + + A = TypeVar('A', bound=int) + B = TypeVar('B', bound=int) + C = TypeVar('C', bound=int) + + # A simplified definition for Matrix[row, column] + class Matrix(Generic[A, B]): + def __init__(self, elements: List[List[int]]) -> None: ... + def __add__(self, other: Matrix[A, B]) -> Matrix[A, B]: ... + def __matmul__(self, other: Matrix[B, C]) -> Matrix[A, C]: ... + def transpose(self) -> Matrix[B, A]: ... + + Foo: Matrix[Literal[2], Literal[3]] = Matrix(...) + Bar: Matrix[Literal[3], Literal[7]] = Matrix(...) + + reveal_type(Foo @ Bar) # Revealed type is Matrix[Literal[2], Literal[7]] + Bar @ Foo # Error, Foo doesn't match expected type Matrix[Literal[7], Literal[int]] + +This class definition is not perfect: it would not prohibit users from +constructing less precise types like ``Matrix[int, int]`` due to the +typevar bound, for example. + +We considered several different proposals for addressing this gap, but +ultimately decided to reject them and defer the problem of integer +generics to a later date. See the `Rejected or out-of-scope ideas`_ +section below. + +Interactions with asserts and other checks +'''''''''''''''''''''''''''''''''''''''''' + +Type checkers should, at the bare minimum, narrow the type of variables +when they are compared directly against other literal types. For +example:: + + def foo(x: str) -> None: + if x == "foo": + # Type checker should narrow 'x' to "foo" here + expects_foo(x) + + # Similarly, type checker should narrow 'x' to "bar" here + assert x == "bar" + expects_bar(x) + +Type checkers may optionally perform additional analysis and narrowing +checks if they so wish. + +**Note:** The exact details of this section may be subject to change. + +Interactions with Final types +''''''''''''''''''''''''''''' + +The interactions between final and literal types were previously +mentioned above, but just to reiterate: if a variable is annotated as +being ``Final``, it should also have an inferred type of ``Literal`` if +the RHS is a literal expression. For example:: + + root_id: Final = 1 + + # Revealed type should be 'Final[Literal[1]]' or something similar + reveal_type(root_id) + + # The types of 'root_id' and 'root_id_2' should be identical + root_id_2: Final[Literal[1]] = 1 + +**TODO:** Cross-link to draft PEP for 'Final' once it's ready + +Rejected or out-of-scope ideas +============================== + +This section of the doc outlines some potential features that are +explicitly out-of-scope. + +True dependent types/integer generics +------------------------------------- + +This proposal is essentially describing adding a very stripped down and +simplified dependent type system to the PEP 484 ecosystem. In contrast, +a full-fledged dependent type system would let users predicate types +based on their values in arbitrary ways. For example, if we had +full-fledged dependent types, it would be possible to write type +signatures like the below:: + + # A vector has length 'n', containing elements of type 'T' + class Vector(Generic[N, T]): ... + + # The type checker will statically verify our function genuinely does + # construct a vector that is equal in length to "len(vec1) + len(vec2)" + # and will throw an error if it does not. + def vector_concat(vec1: Vector[A, T], vec2: Vector[B, T]) -> Vector[A + B, T]: + # ...snip... + +At the very least, it would be useful to add some form of integer +generics. + +There were a few proposals on how this type system might be implemented +— for example, the `Simple dependant types issue ` thread +on the mypy issue tracker suggested compiling Python to Idris. + +Although such a type system could certainly be useful, it’s out-of-scope +for this proposal: it would take a substantial amount of implementation +work, discussion, and research to complete. We also already have a lot +of difficulty as-is trying to exactly keep track of types due to the +``Any`` type and the sheer number of unannotated 3rd party libraries -- +attempting to keep track of values as well seems overly optimistic. + +That said, it’s entirely possible that PEP 484 will acquire a limited +form of dependent types sometime in the future. Specifically, the mypy +team (and other members of the typing ecosystem) would like to add +better support for numpy and similar modules at some point in the +future. This may potentially involve adding some limited form of +dependant typing, along with other features like variadic generics. This +PEP should be seen as a stepping stone towards that goal. + +Adding more concise syntax for literal types +-------------------------------------------- + +One potential downside of this proposal is that having to explicitly +write ``Literal[…]`` can feel verbose. For example, rather then writing:: + + def foobar(arg1: Literal[1], arg2: Literal[True]) -> None: + pass + +...it might be nice to instead write:: + + def foobar(arg1: 1, arg2: True) -> None: + pass + +Unfortunately, these abbreviations simply will not work with the +existing implementation of ``typing`` at runtime. For example, if we try +running the following program using Python 3.7:: + + from typing import Tuple + + # Supposed to accept tuple containing the literals 1 and 2 + def foo(x: Tuple[1, 2]) -> None: + pass + +...we will get the following exception:: + + TypeError: Tuple[t0, t1, …]: each t must be a type. Got 1. + +We don’t want users to have to memorize exactly when and where it’s ok +to omit the ``Literal`` keyword, so we require that ``Literal`` is always +present. + +Backwards compatibility +======================= + +Once this PEP is accepted, the ``Literal`` type will need to be backported for +Python versions that come bundled with older versions of the ``typing`` module. +We plan to do this by adding ``Literal`` to the ``typing_extensions`` 3rd party +module, along with the other backported types. + +There should be no backwards compatibility issues apart from this. + +Related work, references, and footnotes +======================================= + +This proposal was written based on the discussion that took place in the +following threads: + +- `Check that literals belong to/are excluded from a set of values ` + +- `Simple dependent types ` + +- `Typing for multi-dimensional arrays ` + +The overall design of this proposal also ended up converging into +something similar to how +`literal types are handled in TypeScript `. + +.. _PEP 484: https://www.python.org/dev/peps/pep-0484/ + +.. _typing-discussion: https://github.com/python/typing/issues/478 + +.. _mypy-discussion: https://github.com/python/mypy/issues/3062 + +.. _arrays-discussion: https://github.com/python/typing/issues/513 + +.. _typescript-literal-types: https://www.typescriptlang.org/docs/handbook/advanced-types.html#string-literal_types + +.. _typescript-index-types: https://www.typescriptlang.org/docs/handbook/advanced-types.html#index-types + + +Copyright +========= + +This document has been placed in the public domain. + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: + From 0c947fb1657494aacc32cd6551a36741748ce19e Mon Sep 17 00:00:00 2001 From: Michael Lee Date: Mon, 19 Nov 2018 15:53:02 -0800 Subject: [PATCH 02/18] Fix external links (?) --- pep-9999.rst | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 34406cdb7e3..3f32251eb78 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -1,6 +1,6 @@ PEP: 9999 Title: Literal types -Author: Michael Lee +Author: Some subset of the mypy team Status: Draft Type: Standards Track Python-Version: 3.8 @@ -11,7 +11,7 @@ Post-History: 19-Nov-2018 Abstract ======== -This PEP proposes to extend the `PEP 484`_ typing ecosystem by adding +This PEP proposes to extend the PEP 484 typing ecosystem by adding *Literal types*, which indicate that some expression has literally a specific value. For example, the following function will accept only expressions that literally have the value "4":: @@ -52,7 +52,7 @@ For example, here are just two examples from pandas and numpy respectively: anywhere from two to four arrays depending on three boolean flag values. The typing issue tracker contains some -`additional examples and discussion `. +`additional examples and discussion `_. There is currently no way of expressing the type signatures of these functions: PEP 484 does not include any mechanism for writing signatures @@ -478,7 +478,7 @@ an ad-hoc basis: e.g. we special-case TypedDicts and NamedTuples, special-case functions like ``getattr``... One potential alternative solution would be to implement something -similar to TypeScript’s `index types ` +similar to TypeScript’s `index types _` and ``keyof`` operator, which lets you encode the idea that some key (e.g. a literal string) is a member of some object. @@ -637,7 +637,7 @@ At the very least, it would be useful to add some form of integer generics. There were a few proposals on how this type system might be implemented -— for example, the `Simple dependant types issue ` thread +— for example, the `Simple dependent types issue `_ on the mypy issue tracker suggested compiling Python to Idris. Although such a type system could certainly be useful, it’s out-of-scope @@ -697,23 +697,21 @@ module, along with the other backported types. There should be no backwards compatibility issues apart from this. -Related work, references, and footnotes -======================================= +Related work +============ This proposal was written based on the discussion that took place in the following threads: -- `Check that literals belong to/are excluded from a set of values ` +- `Check that literals belong to/are excluded from a set of values `_ -- `Simple dependent types ` +- `Simple dependent types `_ -- `Typing for multi-dimensional arrays ` +- `Typing for multi-dimensional arrays `_ The overall design of this proposal also ended up converging into something similar to how -`literal types are handled in TypeScript `. - -.. _PEP 484: https://www.python.org/dev/peps/pep-0484/ +`literal types are handled in TypeScript `_. .. _typing-discussion: https://github.com/python/typing/issues/478 From ab90bcf2dd9026959a4a985547b08bb3c3eb765b Mon Sep 17 00:00:00 2001 From: Michael Lee Date: Mon, 19 Nov 2018 16:05:49 -0800 Subject: [PATCH 03/18] Tweak and remove some notes --- pep-9999.rst | 37 +++++++++++++++---------------------- 1 file changed, 15 insertions(+), 22 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 3f32251eb78..a38d7e27c39 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -175,16 +175,12 @@ would be legal:: Literal[Color.RED] # Assuming Color is some enum Literal[None] -**Note 1:** The type ``Literal[None]`` is redundant in that the type -``None`` has only a single inhabitant. We nevertheless allow this mostly +**Note:** The type ``Literal[None]`` is redundant in that the type +``None`` has only a single inhabitant. We nevertheless allow this for consistency and ease-of-use. For example, when writing a literal with multiple parameters, it might look a little cleaner to do ``Literal[1, 2, 3, None]`` instead of ``Optional[Literal[1, 2, 3]]``. -**Note 2:** The above list contains the bare minimum types necessary to -make literal types useful. We may add more types to this list in the -future based on demand — see below. - Illegal parameters for ``Literal`` at type check time ''''''''''''''''''''''''''''''''''''''''''''''''''''' @@ -354,18 +350,14 @@ type ``Literal[…]``. **TODO:** Link to the PEP draft for the ``Final`` qualifier once it's ready. -**Note 1:** A potential third way of declaring a variable to be Literal -might be to also try using ``Literal`` as a qualifier, like so:: +**Note 1:** A potential third way of declaring a Literal might be to +try using ``Literal`` as a qualifier:: foo: Literal = "hello" # Illegal! -Although this spelling looks reasonable, we ultimately decided that type -checkers should *reject* assignments like: constructs like ``Final`` or -``ClassVar`` are *qualifiers* and so infer their parameters, but -``Literal`` is a *type*, and types traditionally substitute in ``Any`` -when their parameters are missing (e.g. ``List`` is the same as -``List[Any]``). In this case, we disallow ``Literal[Any]`` so the type -checker should report an error instead. +Although this looks reasonable, we feel type checkers should *reject* +constructs like these: while ``Final`` and ``ClassVar`` are *qualifiers* +and so infer their parameters, ``Literal`` is a *type*. **Note 2:** It may in some cases be possible to use the overall context of the current scope to determine whether some variable should have a @@ -380,11 +372,12 @@ which means it’s theoretically possible to infer that foo has type foo = "blah" expects_blah(foo) -Type checkers are **not** expected to handle these cases. It is, -however, an open question as to whether they are permitted to *try* on a -best-effort basis. E.g. is a PEP 484 compliant type checker always -*obligated* to infer that ``foo`` has type ``str``, or is it ok for them -to attempt more complex inference if they so choose? +This PEP proposes that type checkers are **not** expected to handle these +cases: it is ok to infer that ``foo`` has type ``str``. + +However, it's an open question whether type checkers are permitted to *try* +handling these more complex cases on a best-effort basis. That is, are +type checkers *obligated* to infer that ``foo`` has type ``str``? Type inference inside calls ''''''''''''''''''''''''''' @@ -456,7 +449,7 @@ We require similar behavior when indexing into a tuple or named tuple:: reveal_type(some_tuple[a]) # Revealed type is 'int' some_tuple[b] # Error: 5 is not a valid index into the tuple -...and when using functions like getattr: +...and when using functions like getattr:: class Test: def __init__(self, param: int) -> None: @@ -478,7 +471,7 @@ an ad-hoc basis: e.g. we special-case TypedDicts and NamedTuples, special-case functions like ``getattr``... One potential alternative solution would be to implement something -similar to TypeScript’s `index types _` +similar to TypeScript’s `index types `_ and ``keyof`` operator, which lets you encode the idea that some key (e.g. a literal string) is a member of some object. From 231c104f1f39f168c846e3180254395d07332541 Mon Sep 17 00:00:00 2001 From: Michael Lee Date: Mon, 19 Nov 2018 21:09:56 -0800 Subject: [PATCH 04/18] Prune text --- pep-9999.rst | 354 +++++++++++++++++++++------------------------------ 1 file changed, 148 insertions(+), 206 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index a38d7e27c39..3c2c4edc913 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -11,10 +11,10 @@ Post-History: 19-Nov-2018 Abstract ======== -This PEP proposes to extend the PEP 484 typing ecosystem by adding -*Literal types*, which indicate that some expression has literally a -specific value. For example, the following function will accept only -expressions that literally have the value "4":: +This PEP proposes adding *Literal types* to the PEP 484 ecosystem. +Literal types indicate that some expression has literally a +specific value. For example, the following function will accept +only expressions that have literally the value "4":: from typing import Literal @@ -24,20 +24,20 @@ expressions that literally have the value "4":: accepts_only_four(4) # Type checker accepts accepts_only_four(8) # Type checker rejects -**Note:** This PEP should be considered to be very early draft: we plan -on refining it as we add corresponding proof-of-concept implementation -to mypy. +**Note:** This PEP is still a very early draft: we plan +on refining it as we add work on the corresponding implementation +in mypy. Motivation and Rationale ======================== -Python has many APIs that return different types depending on the exact +Python has many APIs that return different types depending on the value of some argument provided. For example: -- ``open(filename, mode)`` can return an ``IO[bytes]`` or ``IO[Text]`` +- ``open(filename, mode)`` returns either ``IO[bytes]`` or ``IO[Text]`` depending on whether the second argument is something like “r” or “rb” -- ``subprocess.check_output(…)`` can also return bytes or text +- ``subprocess.check_output(…)`` returns either bytes or text depending on whether the ``universal_newlines`` keyword argument is set to ``True`` or not. @@ -57,9 +57,9 @@ The typing issue tracker contains some There is currently no way of expressing the type signatures of these functions: PEP 484 does not include any mechanism for writing signatures where the return type varies depending on the value passed in. -Note that this problem does not go away even if we redesign these APIs to -accept enums instead of string or int literals: the expressions ``MyEnum.FOO`` -and ``MyEnum.BAR`` are both considered to be of type ``MyEnum``.) +Note that this problem persists even if we redesign these APIs to +instead accept enums: ``MyEnum.FOO`` and ``MyEnum.BAR`` are both +considered to be of type ``MyEnum``.) Currently, type checkers work around this limitation by adding ad-hoc extensions for important builtins and standard library functions. For @@ -69,14 +69,7 @@ library functions, it’s unsustainable in general: it’s not reasonable to expect 3rd party library authors to maintain plugins for N different type checkers, for example. -It is, of course, debatable whether these APIs are actually good design: -type checking would be simplified if we we could split functions like -``open(...)`` into two. However, this question is to a large degree -moot: there already exist many widely-used libraries and functions that -vary their behavior based on the provided values. Therefore, we believe that -it is important for the PEP 484 type ecosystem to be updated to match. - -We propose to add *Literal types* to address exactly this gap. +We propose to add *Literal types* to address the gaps listed above. Specification ============= @@ -84,12 +77,12 @@ Specification Core Semantics -------------- -This section of the doc outlines the baseline behavior of literal types. +This section outlines the baseline behavior of literal types. Core behavior ''''''''''''' -Literal types let us indicate that a variable has a specific and +Literal types indicate a variable has a specific and concrete value. For example, if we define some variable ``foo`` to have type ``Literal[3]``, we are declaring that ``foo`` must be exactly equal to ``3`` and no other value. @@ -98,44 +91,40 @@ Given some value ``V`` that is a member of type ``T``, the type ``Literal[V]`` shall be treated as a subtype of ``T``. For example, ``Literal[3]`` is a subtype of ``int``. -All of the methods of the parent type will be directly inherited by the -literal type. This means that if we have some variable ``foo`` of type -``Literal[3]``, it’s safe to do things like ``foo + 5`` since ``foo`` -ultimately inherits int’s ``__add__`` method. The resulting type of -``foo + 5`` will be ``int``. +All methods from the parent type will be directly inherited by the +literal type. So, if we have some variable ``foo`` of type ``Literal[3]`` +it’s safe to do things like ``foo + 5`` since ``foo`` inherits int’s +``__add__`` method. The resulting type of ``foo + 5`` is ``int``. This “inheriting” behavior is identical to how we handle NewTypes. Equivalence of two Literals ''''''''''''''''''''''''''' -Suppose we have two literal types ``Literal[A]`` and ``Literal[B]``. -Both types are considered equivalent when both of the following -conditions are true: +Two types ``Literal[A]`` and ``Literal[B]`` are equivalent when +both of the following conditions are true: 1. ``type(A) == type(B)`` 2. ``A == B`` -So for example, ``Literal[20]`` and ``Literal[0x14]`` would be -equivalent. However, ``Literal[0]`` and ``Literal[False]`` would *not* -be considered equivalent despite the fact that ``0 == False`` evaluates -to ‘true’ at runtime: ``0`` has type ``int`` and ``False`` has type -``bool``. +For example, ``Literal[20]`` and ``Literal[0x14]`` are equivalent. +However, ``Literal[0]`` and ``Literal[False]`` is *not* equivalent +despite that ``0 == False`` evaluates to 'true' at runtime: ``0`` +has type ``int`` and ``False`` has type ``bool``. Shortening unions of literals ''''''''''''''''''''''''''''' -Literals may be parameterized with one or more values. When a Literal is -parameterized with more then one value, it is treated as being exactly -equivalent to the union of those values. That is, -``Literal[V1, V2, V3]`` is equivalent to -``Union[Literal[V1], Literal[V2], Literal[V3]]``. +Literals are parameterized with one or more value. When a Literal is +parameterized with more then one value, it's treated as exactly equivalent +to the union of those types. That is, ``Literal[V1, V2, V3]`` is equivalent +to ``Union[Literal[V1], Literal[V2], Literal[V3]]``. -This shortcut can make writing signatures for functions that can accept +This shortcut helps make writing signatures for functions that accept many different literals more ergonomic — for example, functions like -``open(…)``:: +``open(...)``:: - # Note: this is an over-simplification of the true type signature. + # Note: this is a simplification of the true type signature. _PathType = Union[str, bytes, int] @overload @@ -152,14 +141,13 @@ many different literals more ergonomic — for example, functions like def open(path: _PathType, mode: str) -> IO[Any]: ... **Note:** Literals **must** be parameterized with at least one type. -Types like ``Literal[]`` or ``Literal`` should be rejected by the type -checker. +Types like ``Literal[]`` or ``Literal`` are illegal. Legal and illegal parameterizations ----------------------------------- -This section of the doc describes exactly which values may or may not -parameterize a ``Literal[…]`` type. +This section describes exactly which values may or may not parameterize +a ``Literal[...]`` type. Legal parameters for ``Literal`` at type check time ''''''''''''''''''''''''''''''''''''''''''''''''''' @@ -184,44 +172,40 @@ with multiple parameters, it might look a little cleaner to do Illegal parameters for ``Literal`` at type check time ''''''''''''''''''''''''''''''''''''''''''''''''''''' -The following parameters are provisionally disallowed, mostly for +The following are provisionally disallowed, mostly for simplicity. We can consider adding these to the above list on a case-by-case basis based on demand. -- Explicit byte strings: e.g. ``Literal[b'foo']`` +- Explicit byte strings: e.g. ``Literal[b'foo']``. -- Explicit unicode strings: e.g. ``Literal[u'foo']`` +- Explicit unicode strings: e.g. ``Literal[u'foo']``. -- Floats: e.g. ``Literal[3.14]`` Note: if we do decide to allow +- Floats: e.g. ``Literal[3.14]``. Note: if we do decide to allow floats, we should likely disallow literal infinity and literal NaN. - Any: e.g. ``Literal[Any]`` Note: the semantics of what exactly ``Literal[Any]`` means would need to be clarified first. -- Literal types themselves (or aliases to literal types) e.g. if we +- Literal types themselves (or aliases to literal types). For example, if we create a type alias ``BasicIds = Literal[1, 2, 3]``, then perhaps ``Literal[100, BasicIds]`` should be treated as being equivalent to ``Literal[100, 1, 2, 3]``. -The following parameters are intentionally disallowed by design. We will -most likely never add these parameters at a future date, unless somebody -is able to come up with an extremely compelling argument to the -contrary. +The following parameters are intentionally disallowed by design: - Arbitrary expressions like ``Literal[3 + 4]`` or - ``Literal["foo".replace("o", "b")]``. Literal types are meant to be a simple and - minimal extension to the PEP 484 typing ecosystem, and requiring type + ``Literal["foo".replace("o", "b")]``. Literal types are meant to be a + minimal extension to the PEP 484 typing ecosystem and requiring type checkers to interpret potentially expressions inside types adds too - much complexity to this proposal. Also see the - `Rejected or out-of-scope ideas`_ section of this doc below. + much complexity. Also see `Rejected or out-of-scope ideas`_. - Complex numbers like ``Literal[4 + 3j]``, ``Literal[-4 + 2j]``, and ``Literal[5j]``. Types like ``Literal[4 + 3j]`` would violate the - previous rule; it would then be consistent to also not allow types + previous rule; it would then be consistent to also disallow types like ``Literal[4j]``. -- Tuples containing valid literal types — so ``Literal[(1, "foo", "bar")]`` - would be disallowed. The user could always express this type as +- Tuples containing valid literal types like ``Literal[(1, "foo", "bar")]``. + The user could always express this type as ``Tuple[Literal[1], Literal["foo"], Literal["bar"]]`` instead. Also, tuples are likely to be confused with the ``Literal[1, 2, 3]`` shortcut. @@ -230,23 +214,19 @@ contrary. set literals: literals are always implicitly final and immutable. So, ``Literal[{"a": "b", "c": "d"}]`` would be disallowed. -- Any other types. So, things like ``Literal[MyTypedDict]``, or - ``Literal[some_object_instance]`` would be disallowed. - This includes typevars: if ``T`` is a typevar, types like - ``Literal[T]`` is not allowed. Typevars can vary over only types, not - over values. +- Any other types: for example, ``Literal[MyTypedDict]``, or + ``Literal[some_object_instance]`` are disallowed. + This includes typevars: if ``T`` is a typevar, ``Literal[T]`` is + not allowed. Typevars can vary over only types, never over values. Parameters at runtime ''''''''''''''''''''' -The set of allowable parameters for ``Literal[...]`` is currently -deliberately very strict and limited. However, we may want to extend the -set of allowable parameters in the future or extend the behavior of -``Literal`` in other ways. - -To help us retain this flexibility, the actual implementation of -``typing.Literal`` shall perform *no* checks on any parameters provided -at runtime. For example:: +The set of allowable parameters for ``Literal[...]`` is currently intentionally +very small. However, we may want to extend the ways in which we can use +``Literal[...]`` in the future. To help us retain this flexibility, the +actual implementation of ``typing.Literal`` will perform *no* checks on +any parameters provided at runtime. For example:: def my_function(x: Literal[1 + 2]) -> None: pass @@ -261,15 +241,15 @@ should execute this program with no errors. Literals, enums, and forward references ''''''''''''''''''''''''''''''''''''''' -One potential point of ambiguity is between literal strings and forward +One potential ambiguity is between literal strings and forward references to literal enum members. For example, suppose we have the type ``Literal["Color.RED"]``. Does this literal type -contain a string literal, or a forward reference to some ``Color.RED`` +contain a string literal or a forward reference to some ``Color.RED`` enum member? -In cases like these, we will always assume the user meant to construct a +In cases like these, we always assume the user meant to construct a literal string. If the user wants a forward reference, they must wrap -the entire literal type as a string -- e.g. ``Literal[Color.RED]``. +the entire literal type in a string -- e.g. ``Literal[Color.RED]``. The other alternative is to just not allow literal enums and avoid the ambiguity altogether, but it seems a shame to give them up. @@ -277,10 +257,9 @@ ambiguity altogether, but it seems a shame to give them up. Literals, enums, and Any '''''''''''''''''''''''' -Another point of ambiguity is when the user attempts to use some -expression that is meant to be an enum, but is actually of type ‘Any’. -This can happen when the user, for example, attempts to import an enum -with no type stubs:: +Another ambiguity is when the user attempts to use some expression that +is meant to be an enum but is actually of type ‘Any’. For example, +suppose a user attempts to import an enum from a package with no type hints:: from typing import Literal from lib_with_no_types import SomeEnum # SomeEnum has type 'Any'! @@ -289,51 +268,41 @@ with no type stubs:: # due to the bad import def func(x: Literal[SomeEnum.FOO]) -> None: pass -In this case, should a type checker report an error with ``func``? On -one hand, it makes sense to allow this: it’s usually safe to substitute -``Any`` anywhere where a type is expected, and we don’t normally report -errors if a function attempts to use a type that was inferred to be -equivalent to ``Any``. +Normally, the type checker would be fine with ``func``: it's usually safe to +substitute ``Any`` anywhere a type is expected. -On the other, ``Literal[…]`` expects a value, not a type, and ``Any`` is -*not* meant to represent a placeholder for any arbitrary *value*. The -semantics of what ``Literal[Any]`` means is also somewhat unclear — so, -we’ve tentatively decided to disallow ``Literal[Any]`` for now. - -So in this example, the type checker should reject the signature of -``func`` for using a bad ``Literal`` type. +However, in this case the type checker should report an error: types like +``Literal[Any]`` are currently considered illegal. Although ``Any`` can +serve as a placeholder for any arbitrary *type*, it is **not** allowed to +serve as a placeholder for any arbitrary *value*. This decision is provisional and may be changed at a future date. Inferring types for literal expressions --------------------------------------- -This section of the doc describes how to infer the correct type for -literal expressions — e.g. under what circumstances literal expressions -like ``"foo"`` should be assumed to be of type ``Literal["foo"]`` vs of -type ``str``. +This section describes how to infer the correct type for literal expressions. +E.g. under what circumstances should literal expressions like ``"foo"`` +have an inferred type of ``Literal["foo"]`` vs ``str``? -In general, type checkers are expected to infer ``Literal[…]`` -conservatively and only in contexts where a ``Literal[…]`` type is -explicitly requested. See the following subsections for examples of what -this looks like. +In general, type checkers are expected to be conservative and bias +towards inferring standard types like ``str``. Type checkers should +infer ``Literal[...]`` only in context where a Literal type is +explicitly requested. Variable assignment ''''''''''''''''''' When assigning a literal expression to an unannotated variable, the -inferred type of the variable should be the original base type, not a -``Literal[…]`` type. For example, consider the following snippet of -code:: +inferred type of the variable is the original base type, not ``Literal[...]``. +For example:: foo = "hello" reveal_type(foo) # Revealed type is 'str' -We want to avoid breaking the semantics of any existing code, so the -inferred type of ``foo`` should be ``str``, **not** -``Literal["hello"]``. +This helps ensure we don't break the semantics of any existing code. -If the user wants ``foo`` to have a literal type, they must either +If the user wants ``foo`` to have a literal type, they must explicitly add an annotation:: foo: Literal["hello"] = "hello" @@ -357,14 +326,13 @@ try using ``Literal`` as a qualifier:: Although this looks reasonable, we feel type checkers should *reject* constructs like these: while ``Final`` and ``ClassVar`` are *qualifiers* -and so infer their parameters, ``Literal`` is a *type*. +and so infer their parameters, ``Literal`` is a *type* and so should not. -**Note 2:** It may in some cases be possible to use the overall context -of the current scope to determine whether some variable should have a -Literal type or not. For example, in the following function, ``foo`` is -only ever used as an input to a function that expects ``Literal["blah"]`` -which means it’s theoretically possible to infer that foo has type -``Literal["blah"]``:: +**Note 2:** It may sometimes be possible to use the broader context +to determine whether some variable should have a Literal type or not. +For example, in the following function, ``foo`` is only ever used as +an input to a function that expects ``Literal["blah"]`` which means +it’s theoretically possible to infer that foo has type ``Literal["blah"]``:: def expects_blah(x: Literal["blah"]) -> None: ... @@ -382,9 +350,9 @@ type checkers *obligated* to infer that ``foo`` has type ``str``? Type inference inside calls ''''''''''''''''''''''''''' -When a literal is used inside of some function call, it will be inferred +When a literal is used inside of a function call, it will be inferred as either the original type or the Literal type based on context. For -example, the following code snippet should be legal:: +example, the following snippet should be legal:: def expects_str(x: str) -> None: ... def expects_literal(x: Literal["foo"]) -> None: ... @@ -395,7 +363,7 @@ example, the following code snippet should be legal:: # Legal: "foo" is inferred to be of type 'Literal["foo"]' expects_literal("foo") -However, other expressions in general will not automatically be inferred +However, non-literal expressions in general will not automatically be inferred to be literals. For example:: def expects_literal(x: Literal["foo"]) -> None: ... @@ -405,27 +373,24 @@ to be literals. For example:: expects_literal(my_str) **Note:** If the user wants their API to support accepting both literals -*and* the original type — perhaps for legacy purposes — they should -implement a fallback overload. See the section below on -`Interactions with overloads`_ for more details. +*and* the original type -- perhaps for legacy purposes -- they should +implement a fallback overload. See `Interactions with overloads`_. Miscellaneous interactions -------------------------- -This section of the doc discusses how literal types ought to interact -with other aspects of the PEP 484 type system. +This section discusses how literal types interact with other existing types. Intelligent indexing of structured data: Interactions with TypedDict, Tuple, NamedTuples, and getattr ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' The type checker should support "intelligently indexing" into structured types like TypedDicts, NamedTuple, and classes when using string and int -literal keys. This list is non-exhaustive — there may be other examples -of structured types not included in this list. +literal keys. (This is not an exhaustive list.) -For example, if you try indexing into a TypedDict using a string -Literal, the type checker should return the correct value type if the -key is a member of the TypedDict (and return an error if it isn’t):: +For example, type checkers should infer the correct value type when +indexing into a TypedDict using a string literal that corresponds to +one of the available keys: Foo = TypedDict('Foo', { 'key1': int, @@ -457,9 +422,9 @@ We require similar behavior when indexing into a tuple or named tuple:: def mymethod(self, val: int) -> str: ... - a: Literal = "myfield" - b: Literal = "mymethod" - c: Literal = "blah" + a: Final = "myfield" + b: Final = "mymethod" + c: Final = "blah" t = Test() reveal_type(getattr(t, a)) # Revealed type is 'int' @@ -467,33 +432,29 @@ We require similar behavior when indexing into a tuple or named tuple:: getattr(t, c) # Error: 'Test' does not have attribute named 'blah' These interactions will most likely need to be added to type checkers on -an ad-hoc basis: e.g. we special-case TypedDicts and NamedTuples, -special-case functions like ``getattr``... - -One potential alternative solution would be to implement something -similar to TypeScript’s `index types `_ -and ``keyof`` operator, which lets you encode the idea that some key -(e.g. a literal string) is a member of some object. +an ad-hoc basis. This is a little unfortunate: it would have been nice to +unify these interactions by adding something like TypeScript’s +`index types `_ and ``keyof`` operator, which lets +you encode the idea that some key (e.g. a literal string) is a member of +some object. We currently do not plan on adding a similar concept to Python. Python has many different kinds of structured data beyond just objects (classes, objects, TypedDict, tuples, NamedTuples…) and it’s unclear what the ramifications of attempting to unify all these different -concepts using this idea might be. This idea (or a similar one) may -potentially be revisited in the future, but not as a part of this PEP. +concepts using this idea might be. It may be worth attempting to +unify this behavior in the future, but probably not as a part of this PEP. Interactions with overloads ''''''''''''''''''''''''''' -Literal types and overloads should not need to interact in any -particularly special way: the existing rule for how we handle subtypes -and unions ought to work fine. +Literal types and overloads do not need to interact in any +particularly special way: the existing rules should work fine. -However, one important use case we must make sure will work is the -ability to specify a *fallback* when the user is not using literal -types. For example, consider the ``open`` function from before:: +However, one important use case type checkers must take care to +support is the ability a *fallback* when the user is not using literal +types. For example, consider ``open``:: - # Note: this is an over-simplification of the true type signature. _PathType = Union[str, bytes, int] @overload @@ -509,10 +470,9 @@ types. For example, consider the ``open`` function from before:: @overload def open(path: _PathType, mode: str) -> IO[Any]: ... -If we changed the signature of ``open`` in typeshed so it uses just the -first two overloads, we could potentially end up breaking some clients -who are not passing in a literal string in as a first argument -- -e.g. clients who do something like this:: +If we change the signature of ``open`` to use just the first two overloads, +we would break any code that does not pass in a literal string expression. +For example, code like this would be broken:: mode: str = pick_file_mode(...) with open(path, mode) as f: @@ -520,15 +480,13 @@ e.g. clients who do something like this:: A little more broadly: we propose adding a policy to typeshed that mandates that whenever we add literal types to some existing API, we also -include a fallback overload to let clients who happen to not be using -literal types continue using the API as they were before. Literal types -should always be added to existing APIs in backwards-compatible ways. +always include a fallback overload to maintain backwards-compatibility. Interactions with generics '''''''''''''''''''''''''' Types like ``Literal[3]`` are meant to be just plain old subclasses of -``int``. Consequently, you can use types like ``Literal[3]`` anywhere +``int``. This means you can use types like ``Literal[3]`` anywhere you could use normal types, such as with generics. For example, suppose we want to construct a type representing a @@ -557,17 +515,15 @@ This class definition is not perfect: it would not prohibit users from constructing less precise types like ``Matrix[int, int]`` due to the typevar bound, for example. -We considered several different proposals for addressing this gap, but -ultimately decided to reject them and defer the problem of integer -generics to a later date. See the `Rejected or out-of-scope ideas`_ -section below. +We considered several different proposals for addressing this gap +but ultimately rejected all of them and decided to defer the problem +of integer generics to a later date. See `Rejected or out-of-scope ideas`_ Interactions with asserts and other checks '''''''''''''''''''''''''''''''''''''''''' -Type checkers should, at the bare minimum, narrow the type of variables -when they are compared directly against other literal types. For -example:: +Type checkers should narrow the type of variables when they are compared +directly against other literal types. For example:: def foo(x: str) -> None: if x == "foo": @@ -578,8 +534,7 @@ example:: assert x == "bar" expects_bar(x) -Type checkers may optionally perform additional analysis and narrowing -checks if they so wish. +Type checkers may optionally perform additional analysis and narrowing. **Note:** The exact details of this section may be subject to change. @@ -587,9 +542,9 @@ Interactions with Final types ''''''''''''''''''''''''''''' The interactions between final and literal types were previously -mentioned above, but just to reiterate: if a variable is annotated as -being ``Final``, it should also have an inferred type of ``Literal`` if -the RHS is a literal expression. For example:: +mentioned above, but to reiterate: if a variable is annotated as +``Final`` and has a literal expression on the RHS, the inferred type +should be Literal:: root_id: Final = 1 @@ -604,18 +559,16 @@ the RHS is a literal expression. For example:: Rejected or out-of-scope ideas ============================== -This section of the doc outlines some potential features that are -explicitly out-of-scope. +This section outlines some potential features that are explicitly out-of-scope. True dependent types/integer generics ------------------------------------- -This proposal is essentially describing adding a very stripped down and -simplified dependent type system to the PEP 484 ecosystem. In contrast, -a full-fledged dependent type system would let users predicate types -based on their values in arbitrary ways. For example, if we had -full-fledged dependent types, it would be possible to write type -signatures like the below:: +This proposal is essentially describing adding a very simplified +dependent type system to the PEP 484 ecosystem. One obvious extension +is to implement a full-fledged dependent type system that let users +predicate types based on their values in arbitrary ways. This would +let us write signatures like the below:: # A vector has length 'n', containing elements of type 'T' class Vector(Generic[N, T]): ... @@ -626,45 +579,35 @@ signatures like the below:: def vector_concat(vec1: Vector[A, T], vec2: Vector[B, T]) -> Vector[A + B, T]: # ...snip... -At the very least, it would be useful to add some form of integer -generics. +At the very least, it would be useful to add some form of integer generics. -There were a few proposals on how this type system might be implemented -— for example, the `Simple dependent types issue `_ -on the mypy issue tracker suggested compiling Python to Idris. +Although such a type system would certainly be useful, it’s out-of-scope +for this PEP: it would require a far more substantial amount of implementation work, +discussion, and research to complete compared to the current proposal. -Although such a type system could certainly be useful, it’s out-of-scope -for this proposal: it would take a substantial amount of implementation -work, discussion, and research to complete. We also already have a lot -of difficulty as-is trying to exactly keep track of types due to the -``Any`` type and the sheer number of unannotated 3rd party libraries -- -attempting to keep track of values as well seems overly optimistic. +It's entirely possible that we'll circle back and revisit this topic in the future: +we very likely will need some form of dependent typing along with other extensions +like variadic generics to support popular libraries like numpy. -That said, it’s entirely possible that PEP 484 will acquire a limited -form of dependent types sometime in the future. Specifically, the mypy -team (and other members of the typing ecosystem) would like to add -better support for numpy and similar modules at some point in the -future. This may potentially involve adding some limited form of -dependant typing, along with other features like variadic generics. This -PEP should be seen as a stepping stone towards that goal. +This PEP should be seen as one of the stepping stones towards this goal. Adding more concise syntax for literal types -------------------------------------------- -One potential downside of this proposal is that having to explicitly -write ``Literal[…]`` can feel verbose. For example, rather then writing:: +One objection to this PEP is that having to explicitly write ``Literal[...]`` feels +verbose. For example, instead of writing:: def foobar(arg1: Literal[1], arg2: Literal[True]) -> None: pass -...it might be nice to instead write:: +...it would be nice to instead write:: def foobar(arg1: 1, arg2: True) -> None: pass Unfortunately, these abbreviations simply will not work with the -existing implementation of ``typing`` at runtime. For example, if we try -running the following program using Python 3.7:: +existing implementation of ``typing`` at runtime. For example, the +following snippet crashes when run using Python 3.7:: from typing import Tuple @@ -672,13 +615,12 @@ running the following program using Python 3.7:: def foo(x: Tuple[1, 2]) -> None: pass -...we will get the following exception:: +Running this yields the following exception:: TypeError: Tuple[t0, t1, …]: each t must be a type. Got 1. -We don’t want users to have to memorize exactly when and where it’s ok -to omit the ``Literal`` keyword, so we require that ``Literal`` is always -present. +We don’t want users to have to memorize exactly when it’s ok to elide ``Literal``, +so we require ``Literal`` to always be present. Backwards compatibility ======================= From 82105bac55e0b4517f5523e01e446656a6fac869 Mon Sep 17 00:00:00 2001 From: Michael Lee Date: Mon, 19 Nov 2018 21:10:55 -0800 Subject: [PATCH 05/18] Fix code formatting --- pep-9999.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-9999.rst b/pep-9999.rst index 3c2c4edc913..2af3a50b3a0 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -390,7 +390,7 @@ literal keys. (This is not an exhaustive list.) For example, type checkers should infer the correct value type when indexing into a TypedDict using a string literal that corresponds to -one of the available keys: +one of the available keys:: Foo = TypedDict('Foo', { 'key1': int, From b4678571b9019d85c9c68fcc7c7faae4c9b50cfb Mon Sep 17 00:00:00 2001 From: Michael Lee Date: Mon, 19 Nov 2018 21:17:20 -0800 Subject: [PATCH 06/18] More wording tweaks --- pep-9999.rst | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 2af3a50b3a0..29f03a2923d 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -69,7 +69,7 @@ library functions, it’s unsustainable in general: it’s not reasonable to expect 3rd party library authors to maintain plugins for N different type checkers, for example. -We propose to add *Literal types* to address the gaps listed above. +We propose adding *Literal types* to address these gaps. Specification ============= @@ -326,7 +326,7 @@ try using ``Literal`` as a qualifier:: Although this looks reasonable, we feel type checkers should *reject* constructs like these: while ``Final`` and ``ClassVar`` are *qualifiers* -and so infer their parameters, ``Literal`` is a *type* and so should not. +and so infer their parameters, ``Literal`` is a *type* and should not. **Note 2:** It may sometimes be possible to use the broader context to determine whether some variable should have a Literal type or not. @@ -352,7 +352,7 @@ Type inference inside calls When a literal is used inside of a function call, it will be inferred as either the original type or the Literal type based on context. For -example, the following snippet should be legal:: +example, the following snippet is legal:: def expects_str(x: str) -> None: ... def expects_literal(x: Literal["foo"]) -> None: ... @@ -384,9 +384,8 @@ This section discusses how literal types interact with other existing types. Intelligent indexing of structured data: Interactions with TypedDict, Tuple, NamedTuples, and getattr ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' -The type checker should support "intelligently indexing" into structured -types like TypedDicts, NamedTuple, and classes when using string and int -literal keys. (This is not an exhaustive list.) +Literals can be used to "intelligently index" into structured types +TypedDicts, NamedTuple, and classes. (This is not an exhaustive list). For example, type checkers should infer the correct value type when indexing into a TypedDict using a string literal that corresponds to @@ -395,7 +394,6 @@ one of the available keys:: Foo = TypedDict('Foo', { 'key1': int, 'key2': str, - 'key3': List[bool], }) a: Final = "key1" @@ -405,7 +403,7 @@ one of the available keys:: reveal_type(f[a]) # Revealed type is 'int' f[b] # Error: 'Foo' does not contain a key named 'some other string' -We require similar behavior when indexing into a tuple or named tuple:: +We require similar behavior when indexing into a tuple or NamedTuple:: a: Final = 0 b: Final = 5 @@ -448,8 +446,8 @@ unify this behavior in the future, but probably not as a part of this PEP. Interactions with overloads ''''''''''''''''''''''''''' -Literal types and overloads do not need to interact in any -particularly special way: the existing rules should work fine. +Literal types and overloads do not need to interact in a special +way: the existing rules work fine. However, one important use case type checkers must take care to support is the ability a *fallback* when the user is not using literal @@ -544,7 +542,7 @@ Interactions with Final types The interactions between final and literal types were previously mentioned above, but to reiterate: if a variable is annotated as ``Final`` and has a literal expression on the RHS, the inferred type -should be Literal:: +is Literal:: root_id: Final = 1 @@ -585,11 +583,12 @@ Although such a type system would certainly be useful, it’s out-of-scope for this PEP: it would require a far more substantial amount of implementation work, discussion, and research to complete compared to the current proposal. -It's entirely possible that we'll circle back and revisit this topic in the future: +It's entirely possible we'll circle back and revisit this topic in the future: we very likely will need some form of dependent typing along with other extensions like variadic generics to support popular libraries like numpy. -This PEP should be seen as one of the stepping stones towards this goal. +This PEP should be seen as a stepping stones towards this goal, +rather then an attempt at providing a comprehensive solution. Adding more concise syntax for literal types -------------------------------------------- From 12c6f44b2d3fba40801664c3c3af2bd3241f779b Mon Sep 17 00:00:00 2001 From: Michael Lee Date: Mon, 26 Nov 2018 21:33:22 -0800 Subject: [PATCH 07/18] General cleanup, discuss negative numbers and enums, allow nested literals This commit: 1. Performs a bunch of cleanup suggested by Ivan 2. Adds a note clarifying that negative numbers are *allowed* 3. Adds a note reiterating that PEP 484 expects type checkers to understand enum values when performing type inference (and suggests this can be implemented by treating enums as roughly equal to the union of their types) 4. Moves "nested literals" into the "supported" category -- as I discovered in https://github.com/python/mypy/pull/5947, implementing support for this is not as bad as I thought. 5. Adds an explicit warning to the "literals and generics" section. 6. Modifies some text to become more firm about disallowing 'Literal[Any]' and related constructs. 7. Deletes discussion about TypeScript's "index types" and "keyof" operator -- I mostly included that only because I got some feedback earlier to discuss TypeScript. It felt pretty shoehorned in, anyways. --- pep-9999.rst | 403 ++++++++++++++++++++++++++++----------------------- 1 file changed, 224 insertions(+), 179 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 29f03a2923d..6c2767025b4 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -1,6 +1,6 @@ PEP: 9999 Title: Literal types -Author: Some subset of the mypy team +Author: TODO Status: Draft Type: Standards Track Python-Version: 3.8 @@ -21,8 +21,8 @@ only expressions that have literally the value "4":: def accepts_only_four(x: Literal[4]) -> None: pass - accepts_only_four(4) # Type checker accepts - accepts_only_four(8) # Type checker rejects + accepts_only_four(4) # Ok + accepts_only_four(19) # Rejected **Note:** This PEP is still a very early draft: we plan on refining it as we add work on the corresponding implementation @@ -35,9 +35,9 @@ Python has many APIs that return different types depending on the value of some argument provided. For example: - ``open(filename, mode)`` returns either ``IO[bytes]`` or ``IO[Text]`` - depending on whether the second argument is something like “r” or - “rb” -- ``subprocess.check_output(…)`` returns either bytes or text + depending on whether the second argument is something like ``r`` or + ``rb``. +- ``subprocess.check_output(...)`` returns either bytes or text depending on whether the ``universal_newlines`` keyword argument is set to ``True`` or not. @@ -59,36 +59,33 @@ functions: PEP 484 does not include any mechanism for writing signatures where the return type varies depending on the value passed in. Note that this problem persists even if we redesign these APIs to instead accept enums: ``MyEnum.FOO`` and ``MyEnum.BAR`` are both -considered to be of type ``MyEnum``.) +considered to be of type ``MyEnum``. Currently, type checkers work around this limitation by adding ad-hoc extensions for important builtins and standard library functions. For example mypy comes bundled with a plugin that attempts to infer more -precise types for ``open(…)``. While this approach works for standard +precise types for ``open(...)``. While this approach works for standard library functions, it’s unsustainable in general: it’s not reasonable to expect 3rd party library authors to maintain plugins for N different type checkers, for example. We propose adding *Literal types* to address these gaps. -Specification -============= - Core Semantics --------------- +============== This section outlines the baseline behavior of literal types. Core behavior -''''''''''''' +------------- Literal types indicate a variable has a specific and concrete value. For example, if we define some variable ``foo`` to have type ``Literal[3]``, we are declaring that ``foo`` must be exactly equal to ``3`` and no other value. -Given some value ``V`` that is a member of type ``T``, the type -``Literal[V]`` shall be treated as a subtype of ``T``. For example, +Given some value ``v`` that is a member of type ``T``, the type +``Literal[v]`` shall be treated as a subtype of ``T``. For example, ``Literal[3]`` is a subtype of ``int``. All methods from the parent type will be directly inherited by the @@ -96,16 +93,17 @@ literal type. So, if we have some variable ``foo`` of type ``Literal[3]`` it’s safe to do things like ``foo + 5`` since ``foo`` inherits int’s ``__add__`` method. The resulting type of ``foo + 5`` is ``int``. -This “inheriting” behavior is identical to how we handle NewTypes. +This "inheriting" behavior is identical to how we +`handle NewTypes. `_. Equivalence of two Literals -''''''''''''''''''''''''''' +--------------------------- -Two types ``Literal[A]`` and ``Literal[B]`` are equivalent when +Two types ``Literal[v1]`` and ``Literal[v2]`` are equivalent when both of the following conditions are true: -1. ``type(A) == type(B)`` -2. ``A == B`` +1. ``type(v1) == type(v2)`` +2. ``v1 == v2`` For example, ``Literal[20]`` and ``Literal[0x14]`` are equivalent. However, ``Literal[0]`` and ``Literal[False]`` is *not* equivalent @@ -113,12 +111,12 @@ despite that ``0 == False`` evaluates to 'true' at runtime: ``0`` has type ``int`` and ``False`` has type ``bool``. Shortening unions of literals -''''''''''''''''''''''''''''' +----------------------------- Literals are parameterized with one or more value. When a Literal is -parameterized with more then one value, it's treated as exactly equivalent -to the union of those types. That is, ``Literal[V1, V2, V3]`` is equivalent -to ``Union[Literal[V1], Literal[V2], Literal[V3]]``. +parameterized with more than one value, it's treated as exactly equivalent +to the union of those types. That is, ``Literal[v1, v2, v3]`` is equivalent +to ``Union[Literal[v1], Literal[v2], Literal[v3]]``. This shortcut helps make writing signatures for functions that accept many different literals more ergonomic — for example, functions like @@ -140,69 +138,88 @@ many different literals more ergonomic — for example, functions like @overload def open(path: _PathType, mode: str) -> IO[Any]: ... -**Note:** Literals **must** be parameterized with at least one type. +The provided values do not all have to be members of the same type. +For example, ``Literal[42, "foo", True]`` is a legal type. + +However, Literal **must** be parameterized with at least one type. Types like ``Literal[]`` or ``Literal`` are illegal. + Legal and illegal parameterizations ------------------------------------ +=================================== + +This section describes what exactly constitutes a legal ``Literal[...]`` type: +what values may and may not be used as parameters. + +In short, a ``Literal[...]`` type may be parameterized by one or more literal expressions, +and nothing else. -This section describes exactly which values may or may not parameterize -a ``Literal[...]`` type. Legal parameters for ``Literal`` at type check time -''''''''''''''''''''''''''''''''''''''''''''''''''' +--------------------------------------------------- ``Literal`` may be parameterized with literal ints, native strings, -bools, Enum values, and ``None``. So for example, all of the following -would be legal:: +bools, Enum values and ``None``. So for example, all of +the following would be legal:: Literal[26] Literal[0x1A] # Exactly equivalent to Literal[26] + Literal[-4] Literal["hello world"] Literal[True] Literal[Color.RED] # Assuming Color is some enum Literal[None] -**Note:** The type ``Literal[None]`` is redundant in that the type -``None`` has only a single inhabitant. We nevertheless allow this -for consistency and ease-of-use. For example, when writing a literal -with multiple parameters, it might look a little cleaner to do -``Literal[1, 2, 3, None]`` instead of ``Optional[Literal[1, 2, 3]]``. +**Note:** Since the type ``None`` is inhabited by just a single +value, the types ``None`` and ``Literal[None]`` are exactly equivalent. +Type checkers may simplify ``Literal[None]`` into just ``None``. -Illegal parameters for ``Literal`` at type check time -''''''''''''''''''''''''''''''''''''''''''''''''''''' +``Literal`` may also be parameterized by other literal types, or type aliases +to other literal types. For example, the following is legal:: -The following are provisionally disallowed, mostly for -simplicity. We can consider adding these to the above list on a -case-by-case basis based on demand. + ReadOnlyMode = Literal["r", "r+"] + WriteAndTruncateMode = Literal["w", "w+", "wt", "w+t"] + WriteNoTruncateMode = Literal["r+", "r+t"] + AppendMode = Literal["a", "a+", "at", "a+t"] -- Explicit byte strings: e.g. ``Literal[b'foo']``. + AllModes = Literal[ReadOnlyMode, WriteAndTruncateMode, WriteNoTruncateMode, AppendMode] -- Explicit unicode strings: e.g. ``Literal[u'foo']``. +This feature is again intended to help make using and reusing literal types +more ergonomic. -- Floats: e.g. ``Literal[3.14]``. Note: if we do decide to allow - floats, we should likely disallow literal infinity and literal NaN. +**Note:** As a consequence of the above rules, type checkers are also expected +to support types that look like the following:: -- Any: e.g. ``Literal[Any]`` Note: the semantics of what exactly - ``Literal[Any]`` means would need to be clarified first. + Literal[Literal[Literal[1, 2, 3], "foo"], 5, 5, 5, None] + +This should be exactly equivalent to the following type:: -- Literal types themselves (or aliases to literal types). For example, if we - create a type alias ``BasicIds = Literal[1, 2, 3]``, then perhaps - ``Literal[100, BasicIds]`` should be treated as being equivalent to - ``Literal[100, 1, 2, 3]``. + Literal[1, 2, 3, "foo", 5, None] + +...and also to the following type:: + + Optional[Literal[1, 2, 3, "foo", 5]] + + +Illegal parameters for ``Literal`` at type check time +----------------------------------------------------- The following parameters are intentionally disallowed by design: -- Arbitrary expressions like ``Literal[3 + 4]`` or - ``Literal["foo".replace("o", "b")]``. Literal types are meant to be a - minimal extension to the PEP 484 typing ecosystem and requiring type - checkers to interpret potentially expressions inside types adds too - much complexity. Also see `Rejected or out-of-scope ideas`_. +- Arbitrary expressions like ``Literal[3 + 4]`` or + ``Literal["foo".replace("o", "b")]``. + + - Rationale: Literal types are meant to be a + minimal extension to the PEP 484 typing ecosystem and requiring type + checkers to interpret potentially expressions inside types adds too + much complexity. Also see `Rejected or out-of-scope ideas`_. + + - As a consequence, complex numbers like ``Literal[4 + 3j]`` and ``Literal[-4 + 2j]`` + are also prohibited. For consistency, literals like ``Literal[4j]`` that contain + just a single complex number are also prohibited. -- Complex numbers like ``Literal[4 + 3j]``, ``Literal[-4 + 2j]``, and - ``Literal[5j]``. Types like ``Literal[4 + 3j]`` would violate the - previous rule; it would then be consistent to also disallow types - like ``Literal[4j]``. + - The only exception to this rule is the unary ``-`` (minus) for ints: types + like ``Literal[-5]`` are *accepted*. - Tuples containing valid literal types like ``Literal[(1, "foo", "bar")]``. The user could always express this type as @@ -212,34 +229,48 @@ The following parameters are intentionally disallowed by design: - Mutable literal data structures like dict literals, list literals, or set literals: literals are always implicitly final and immutable. So, - ``Literal[{"a": "b", "c": "d"}]`` would be disallowed. + ``Literal[{"a": "b", "c": "d"}]`` is illegal. - Any other types: for example, ``Literal[MyTypedDict]``, or - ``Literal[some_object_instance]`` are disallowed. + ``Literal[some_object_instance]`` are illegal. This includes typevars: if ``T`` is a typevar, ``Literal[T]`` is not allowed. Typevars can vary over only types, never over values. +The following are provisionally disallowed for simplicity. We can consider allowing +them on a case-by-case basis based on demand. + +- Explicit byte strings: e.g. ``Literal[b'foo']``. + +- Explicit unicode strings: e.g. ``Literal[u'foo']``. + +- Floats: e.g. ``Literal[3.14]``. Note: if we do decide to allow + floats, we should likely disallow literal infinity and literal NaN. + +- Any: e.g. ``Literal[Any]`` Note: the semantics of what exactly + ``Literal[Any]`` means would need to be clarified first. + Parameters at runtime -''''''''''''''''''''' +--------------------- -The set of allowable parameters for ``Literal[...]`` is currently intentionally -very small. However, we may want to extend the ways in which we can use -``Literal[...]`` in the future. To help us retain this flexibility, the -actual implementation of ``typing.Literal`` will perform *no* checks on -any parameters provided at runtime. For example:: +Although the set of parameters ``Literal[...]`` may contain at type-check time +is very small, the actual implementation of ``typing.Literal`` will not perform +any checks at runtime. For example:: - def my_function(x: Literal[1 + 2]) -> None: - pass + def my_function(x: Literal[1 + 2]) -> int: + return x * 3 x: Literal = 3 - y: Literal[my_function] = my_funcion + y: Literal[my_function] = my_function The type checker should reject this program: all three uses of ``Literal`` are *invalid* according to this spec. However, Python itself should execute this program with no errors. +This helps us preserve flexibility in case we want to expand the scope of +what ``Literal`` can be used for in the future. + Literals, enums, and forward references -''''''''''''''''''''''''''''''''''''''' +--------------------------------------- One potential ambiguity is between literal strings and forward references to literal enum members. For example, suppose we have the @@ -249,73 +280,68 @@ enum member? In cases like these, we always assume the user meant to construct a literal string. If the user wants a forward reference, they must wrap -the entire literal type in a string -- e.g. ``Literal[Color.RED]``. - -The other alternative is to just not allow literal enums and avoid the -ambiguity altogether, but it seems a shame to give them up. +the entire literal type in a string -- e.g. ``"Literal[Color.RED]"``. Literals, enums, and Any -'''''''''''''''''''''''' +------------------------ Another ambiguity is when the user attempts to use some expression that -is meant to be an enum but is actually of type ‘Any’. For example, +is meant to be an enum but is actually of type ``Any``. For example, suppose a user attempts to import an enum from a package with no type hints:: from typing import Literal from lib_with_no_types import SomeEnum # SomeEnum has type 'Any'! - # Signature is equivalent to `func(x: Literal[Any]) -> None` - # due to the bad import - def func(x: Literal[SomeEnum.FOO]) -> None: pass + # x has type `Literal[Any]` due to the bad import + x: Literal[SomeEnum.FOO] -Normally, the type checker would be fine with ``func``: it's usually safe to -substitute ``Any`` anywhere a type is expected. +Because ``Literal`` may not be parameterized by ``Any``, this program +is *illegal*: the type checker should report an error with the last line. -However, in this case the type checker should report an error: types like -``Literal[Any]`` are currently considered illegal. Although ``Any`` can -serve as a placeholder for any arbitrary *type*, it is **not** allowed to -serve as a placeholder for any arbitrary *value*. +In short, while ``Any`` may effectively be used as a placeholder for any +arbitrary *type*, it is currently **not** allowed to serve as a placeholder +for any arbitrary *value*. -This decision is provisional and may be changed at a future date. Inferring types for literal expressions ---------------------------------------- +======================================= -This section describes how to infer the correct type for literal expressions. -E.g. under what circumstances should literal expressions like ``"foo"`` -have an inferred type of ``Literal["foo"]`` vs ``str``? +This section describes under what circumstances some expression should have +an inferred Literal type. -In general, type checkers are expected to be conservative and bias -towards inferring standard types like ``str``. Type checkers should -infer ``Literal[...]`` only in context where a Literal type is -explicitly requested. +For example, under what circumstances should literal expressions like ``"blue"`` +have an inferred type of ``Literal["blue"]`` vs ``str``? + +In short, type checkers are expected to be conservative and bias towards +inferring standard types like ``str``. Type checkers should infer ``Literal[...]`` +only in contexts where a Literal type is explicitly requested. Variable assignment -''''''''''''''''''' +------------------- When assigning a literal expression to an unannotated variable, the inferred type of the variable is the original base type, not ``Literal[...]``. For example:: - foo = "hello" - reveal_type(foo) # Revealed type is 'str' + border_color = "blue" + reveal_type(border_color) # Revealed type is 'str' This helps ensure we don't break the semantics of any existing code. If the user wants ``foo`` to have a literal type, they must explicitly add an annotation:: - foo: Literal["hello"] = "hello" - reveal_types(foo) # Revealed type is 'Literal["hello"]' + border_color: Literal["blue"] = "blue" + reveal_types(border_color) # Revealed type is 'Literal["blue"]' Or alternatively, use the ``Final`` qualifier:: - foo: Final = "hello" - reveal_types(foo) # Revealed type is 'Final[Literal["hello"]]' + border_color: Final = "blue" + reveal_types(border_color) # Revealed type is 'Final[Literal["blue"]]' The ``Final`` qualifier will automatically infer a ``Literal`` type in an assignment if the LHS is a literal expression, or an expression of -type ``Literal[…]``. +type ``Literal[...]``. **TODO:** Link to the PEP draft for the ``Final`` qualifier once it's ready. @@ -324,31 +350,32 @@ try using ``Literal`` as a qualifier:: foo: Literal = "hello" # Illegal! -Although this looks reasonable, we feel type checkers should *reject* -constructs like these: while ``Final`` and ``ClassVar`` are *qualifiers* -and so infer their parameters, ``Literal`` is a *type* and should not. +Type checkers should *reject* lines like these. Unlike ``Final`` and ``ClassVar``, +``Literal`` is a *type*, not a *qualifier*. Only qualifiers should infer their +parameters. -**Note 2:** It may sometimes be possible to use the broader context -to determine whether some variable should have a Literal type or not. -For example, in the following function, ``foo`` is only ever used as -an input to a function that expects ``Literal["blah"]`` which means -it’s theoretically possible to infer that foo has type ``Literal["blah"]``:: +**Note 2:** Type checkers are only expected to use the context available +to them within the current statement to infer the type of the variable. +They may *optionally* use additional context to infer more precise types. +For example:: - def expects_blah(x: Literal["blah"]) -> None: ... + def expects_blah(x: Literal[4]) -> None: ... def test() -> None: - foo = "blah" + foo = 4 expects_blah(foo) -This PEP proposes that type checkers are **not** expected to handle these -cases: it is ok to infer that ``foo`` has type ``str``. +In this program, it is theoretically possible for a type checker to deduce that +``foo`` is only ever used as input to a function that expects ``Literal[4]`` +and so infer that ``foo`` must have type ``Literal[4]``. -However, it's an open question whether type checkers are permitted to *try* -handling these more complex cases on a best-effort basis. That is, are -type checkers *obligated* to infer that ``foo`` has type ``str``? +While type checkers *may* perform this kind of analysis, they are not obligated +to do so. It is ok to infer that ``foo`` has type ``int``, since there is no +context present in the assignment statement itself that would suggest that ``foo`` +should be a literal type. Type inference inside calls -''''''''''''''''''''''''''' +--------------------------- When a literal is used inside of a function call, it will be inferred as either the original type or the Literal type based on context. For @@ -376,34 +403,19 @@ to be literals. For example:: *and* the original type -- perhaps for legacy purposes -- they should implement a fallback overload. See `Interactions with overloads`_. -Miscellaneous interactions --------------------------- +Interactions with other types and features +========================================== -This section discusses how literal types interact with other existing types. +This section discusses how Literal types interact with other existing types. -Intelligent indexing of structured data: Interactions with TypedDict, Tuple, NamedTuples, and getattr -''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' +Intelligent indexing of structured data +--------------------------------------- -Literals can be used to "intelligently index" into structured types -TypedDicts, NamedTuple, and classes. (This is not an exhaustive list). +Literals can be used to "intelligently index" into structured types like +NamedTuple, classes, and TypedDict. (Note: this is not an exhaustive list). For example, type checkers should infer the correct value type when -indexing into a TypedDict using a string literal that corresponds to -one of the available keys:: - - Foo = TypedDict('Foo', { - 'key1': int, - 'key2': str, - }) - - a: Final = "key1" - b: Final = "some other string" - - f: Foo - reveal_type(f[a]) # Revealed type is 'int' - f[b] # Error: 'Foo' does not contain a key named 'some other string' - -We require similar behavior when indexing into a tuple or NamedTuple:: +indexing into a tuple using an int key that corresponds a valid index:: a: Final = 0 b: Final = 5 @@ -412,7 +424,7 @@ We require similar behavior when indexing into a tuple or NamedTuple:: reveal_type(some_tuple[a]) # Revealed type is 'int' some_tuple[b] # Error: 5 is not a valid index into the tuple -...and when using functions like getattr:: +We expect similar behavior when using functions like getattr:: class Test: def __init__(self, param: int) -> None: @@ -430,21 +442,12 @@ We require similar behavior when indexing into a tuple or NamedTuple:: getattr(t, c) # Error: 'Test' does not have attribute named 'blah' These interactions will most likely need to be added to type checkers on -an ad-hoc basis. This is a little unfortunate: it would have been nice to -unify these interactions by adding something like TypeScript’s -`index types `_ and ``keyof`` operator, which lets -you encode the idea that some key (e.g. a literal string) is a member of -some object. - -We currently do not plan on adding a similar concept to Python. Python -has many different kinds of structured data beyond just objects -(classes, objects, TypedDict, tuples, NamedTuples…) and it’s unclear -what the ramifications of attempting to unify all these different -concepts using this idea might be. It may be worth attempting to -unify this behavior in the future, but probably not as a part of this PEP. +an ad-hoc basis. + +**TODO:** Link to the PEP for TypedDict once it's ready. Interactions with overloads -''''''''''''''''''''''''''' +--------------------------- Literal types and overloads do not need to interact in a special way: the existing rules work fine. @@ -481,16 +484,14 @@ mandates that whenever we add literal types to some existing API, we also always include a fallback overload to maintain backwards-compatibility. Interactions with generics -'''''''''''''''''''''''''' +-------------------------- Types like ``Literal[3]`` are meant to be just plain old subclasses of ``int``. This means you can use types like ``Literal[3]`` anywhere you could use normal types, such as with generics. -For example, suppose we want to construct a type representing a -2-dimensional Matrix which can be parameterized by two literal ints -representing the number of rows and columns respectively. Such a type -could be built using the existing generics system like so:: +This means that it is legal to parameterize generic functions or +classes using Literal types:: A = TypeVar('A', bound=int) B = TypeVar('B', bound=int) @@ -498,27 +499,36 @@ could be built using the existing generics system like so:: # A simplified definition for Matrix[row, column] class Matrix(Generic[A, B]): - def __init__(self, elements: List[List[int]]) -> None: ... def __add__(self, other: Matrix[A, B]) -> Matrix[A, B]: ... def __matmul__(self, other: Matrix[B, C]) -> Matrix[A, C]: ... def transpose(self) -> Matrix[B, A]: ... - Foo: Matrix[Literal[2], Literal[3]] = Matrix(...) - Bar: Matrix[Literal[3], Literal[7]] = Matrix(...) + foo: Matrix[Literal[2], Literal[3]] = Matrix(...) + bar: Matrix[Literal[3], Literal[7]] = Matrix(...) - reveal_type(Foo @ Bar) # Revealed type is Matrix[Literal[2], Literal[7]] - Bar @ Foo # Error, Foo doesn't match expected type Matrix[Literal[7], Literal[int]] + baz = foo @ bar + reveal_type(baz) # Revealed type is 'Matrix[Literal[2], Literal[7]]' -This class definition is not perfect: it would not prohibit users from -constructing less precise types like ``Matrix[int, int]`` due to the -typevar bound, for example. +Similarly, it is legal to construct TypeVars with value restrictions +or bounds involving Literal types:: -We considered several different proposals for addressing this gap -but ultimately rejected all of them and decided to defer the problem -of integer generics to a later date. See `Rejected or out-of-scope ideas`_ + T = TypeVar('T', Literal["a"], Literal["b"], Literal["c"]) + S = TypeVar('S', bound=Literal["foo"]) + +...although it is unclear when it would ever be useful to do so. + +**Note:** Literal types and generics deliberately interact in only very +basic and limited ways. In particular, libraries that want to typecheck +code containing an heavy amount of numeric or numpy-style manipulation will +almost certainly likely find Literal types as proposed in this PEP to be +insufficient for their needs. + +We considered several different proposals for fixing this, but ultimately +decided to defer the problem of integer generics to a later date. See +`Rejected or out-of-scope ideas`_ for more details. Interactions with asserts and other checks -'''''''''''''''''''''''''''''''''''''''''' +------------------------------------------ Type checkers should narrow the type of variables when they are compared directly against other literal types. For example:: @@ -532,12 +542,43 @@ directly against other literal types. For example:: assert x == "bar" expects_bar(x) -Type checkers may optionally perform additional analysis and narrowing. - -**Note:** The exact details of this section may be subject to change. +This includes with enums. For example, the type checker should be capable +of inferring that the final ``else`` statement in the following function +is unreachable:: + + class Status(Enum): + SUCCESS = 0 + PARSE_ERROR = 1 + INVALID_DATA = 2 + FATAL_ERROR = 3 + + def parse_status(status: Status) -> None: + if status is Status.SUCCESS: + print("Success!") + elif status is Status.PARSE_ERROR: + print("Unable to deserialize data") + elif status is Status.INVALID_DATA: + print("The given data is invalid because...") + elif status is Status.FATAL_ERROR: + print("Unexpected fatal error...") + else: + # Error should not be reported by type checkers that + # ignore errors in unreachable blocks + print("Nonsense" + 100) + +This behavior is technically not new: this behavior is +`already codified within PEP 484 `_. However, many type +checkers (such as mypy) do not yet implement this behavior. Once literal +types are introduced, it will become easier to do so: we can model +enums as being approximately equal to the union of their values. So, +``Status`` would be treated as being approximately equal to +``Literal[Status.SUCCESS, Status.PARSE_ERROR, Status.INVALID_DATA, Status.FATAL_ERROR]``. + +Type checkers may optionally perform additional analysis and narrowing +beyond what is described above. Interactions with Final types -''''''''''''''''''''''''''''' +----------------------------- The interactions between final and literal types were previously mentioned above, but to reiterate: if a variable is annotated as @@ -616,7 +657,7 @@ following snippet crashes when run using Python 3.7:: Running this yields the following exception:: - TypeError: Tuple[t0, t1, …]: each t must be a type. Got 1. + TypeError: Tuple[t0, t1, ...]: each t must be a type. Got 1. We don’t want users to have to memorize exactly when it’s ok to elide ``Literal``, so we require ``Literal`` to always be present. @@ -657,6 +698,10 @@ something similar to how .. _typescript-index-types: https://www.typescriptlang.org/docs/handbook/advanced-types.html#index-types +.. _newtypes: https://www.python.org/dev/peps/pep-0484/#newtype-helper-function + +.. _pep-484-enums: https://www.python.org/dev/peps/pep-0484/#support-for-singleton-types-in-unions + Copyright ========= From 554a2dfc45fead4e8055046a6e68d7bf3bd59355 Mon Sep 17 00:00:00 2001 From: Michael Lee Date: Mon, 26 Nov 2018 21:51:39 -0800 Subject: [PATCH 08/18] Add an additional reason why we do not perform runtime checks --- pep-9999.rst | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 6c2767025b4..90bff644526 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -266,8 +266,11 @@ The type checker should reject this program: all three uses of ``Literal`` are *invalid* according to this spec. However, Python itself should execute this program with no errors. -This helps us preserve flexibility in case we want to expand the scope of -what ``Literal`` can be used for in the future. +This is partly to help us preserve flexibility in case we want to expand the +scope of what ``Literal`` can be used for in the future, and partly because +it is not possible to detect all illegal parameters at runtime to begin with. +For example, it is impossible to ditinguish between ``Literal[1 + 2]`` and +``Literal[3]`` at runtime. Literals, enums, and forward references --------------------------------------- From 003afd632cc64405c9dbb2acb73deb619637b398 Mon Sep 17 00:00:00 2001 From: Michael Lee Date: Thu, 13 Dec 2018 19:34:05 -0800 Subject: [PATCH 09/18] Respond to rchen152's comments: fix typo, clarify aside about typevars --- pep-9999.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 90bff644526..3c681b2885f 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -269,7 +269,7 @@ should execute this program with no errors. This is partly to help us preserve flexibility in case we want to expand the scope of what ``Literal`` can be used for in the future, and partly because it is not possible to detect all illegal parameters at runtime to begin with. -For example, it is impossible to ditinguish between ``Literal[1 + 2]`` and +For example, it is impossible to distinguish between ``Literal[1 + 2]`` and ``Literal[3]`` at runtime. Literals, enums, and forward references @@ -518,7 +518,8 @@ or bounds involving Literal types:: T = TypeVar('T', Literal["a"], Literal["b"], Literal["c"]) S = TypeVar('S', bound=Literal["foo"]) -...although it is unclear when it would ever be useful to do so. +...although it is unclear when it would ever be useful to construct a +TypeVar with a Literal bound. **Note:** Literal types and generics deliberately interact in only very basic and limited ways. In particular, libraries that want to typecheck From a8b6f16951216793b239877b792b4cb41e0e43da Mon Sep 17 00:00:00 2001 From: Michael Lee Date: Thu, 13 Dec 2018 19:59:53 -0800 Subject: [PATCH 10/18] Move byte and unicode strings into the 'allowed' list --- pep-9999.rst | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 3c681b2885f..f655cb54ca5 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -158,7 +158,7 @@ and nothing else. Legal parameters for ``Literal`` at type check time --------------------------------------------------- -``Literal`` may be parameterized with literal ints, native strings, +``Literal`` may be parameterized with literal ints, byte and unicode strings, bools, Enum values and ``None``. So for example, all of the following would be legal:: @@ -166,6 +166,8 @@ the following would be legal:: Literal[0x1A] # Exactly equivalent to Literal[26] Literal[-4] Literal["hello world"] + Literal[b"hello world"] + Literal[u"hello world"] Literal[True] Literal[Color.RED] # Assuming Color is some enum Literal[None] @@ -190,7 +192,7 @@ more ergonomic. **Note:** As a consequence of the above rules, type checkers are also expected to support types that look like the following:: - Literal[Literal[Literal[1, 2, 3], "foo"], 5, 5, 5, None] + Literal[Literal[Literal[1, 2, 3], "foo"], 5, None] This should be exactly equivalent to the following type:: @@ -200,6 +202,15 @@ This should be exactly equivalent to the following type:: Optional[Literal[1, 2, 3, "foo", 5]] +**Note:** String literal types like ``Literal["foo"]`` should subtype either bytes or +unicode in the same way regular string literals do at runtime. + +For example, in Python 3, the type ``Literal["foo"]`` is equivalent to ``Literal[u"foo"]``, +since ``"foo"`` is equivalent to ``u"foo"`` in Python 3. + +Similarly, in Python 2, the type ``Literal["foo"]`` is equivalent to ``Literal[b"foo"]`` -- +unless the file includes a ``from __future__ import unicode_literals`` import, in which case +it would be equivalent to ``Literal[u"foo"]``. Illegal parameters for ``Literal`` at type check time ----------------------------------------------------- @@ -239,10 +250,6 @@ The following parameters are intentionally disallowed by design: The following are provisionally disallowed for simplicity. We can consider allowing them on a case-by-case basis based on demand. -- Explicit byte strings: e.g. ``Literal[b'foo']``. - -- Explicit unicode strings: e.g. ``Literal[u'foo']``. - - Floats: e.g. ``Literal[3.14]``. Note: if we do decide to allow floats, we should likely disallow literal infinity and literal NaN. From 088f492030e1c2abeec412205821d90a7a039ff3 Mon Sep 17 00:00:00 2001 From: Michael Lee Date: Wed, 13 Mar 2019 15:55:19 -0700 Subject: [PATCH 11/18] Relax sections regarding inference, mention the 'custom type guard' idea --- pep-9999.rst | 252 +++++++++++++++++++++++++-------------------------- 1 file changed, 124 insertions(+), 128 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index f655cb54ca5..6ffff5b9148 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -1,12 +1,13 @@ PEP: 9999 Title: Literal types -Author: TODO +Author: Michael Lee , Ivan Levkivskyi , Jukka Lehtosalo Status: Draft Type: Standards Track Python-Version: 3.8 Content-Type: text/x-rst -Created: 19-Nov-2018 -Post-History: 19-Nov-2018 +Created: 13-Mar-2018 +Post-History: 13-Mar-2018 +Discussions-To: Typing-Sig Abstract ======== @@ -21,13 +22,9 @@ only expressions that have literally the value "4":: def accepts_only_four(x: Literal[4]) -> None: pass - accepts_only_four(4) # Ok + accepts_only_four(4) # Ok accepts_only_four(19) # Rejected -**Note:** This PEP is still a very early draft: we plan -on refining it as we add work on the corresponding implementation -in mypy. - Motivation and Rationale ======================== @@ -67,7 +64,7 @@ example mypy comes bundled with a plugin that attempts to infer more precise types for ``open(...)``. While this approach works for standard library functions, it’s unsustainable in general: it’s not reasonable to expect 3rd party library authors to maintain plugins for N different -type checkers, for example. +type checkers. We propose adding *Literal types* to address these gaps. @@ -259,7 +256,7 @@ them on a case-by-case basis based on demand. Parameters at runtime --------------------- -Although the set of parameters ``Literal[...]`` may contain at type-check time +Although the set of parameters ``Literal[...]`` may contain at type check time is very small, the actual implementation of ``typing.Literal`` will not perform any checks at runtime. For example:: @@ -312,96 +309,68 @@ In short, while ``Any`` may effectively be used as a placeholder for any arbitrary *type*, it is currently **not** allowed to serve as a placeholder for any arbitrary *value*. +Type inference +============== -Inferring types for literal expressions -======================================= - -This section describes under what circumstances some expression should have -an inferred Literal type. - -For example, under what circumstances should literal expressions like ``"blue"`` -have an inferred type of ``Literal["blue"]`` vs ``str``? - -In short, type checkers are expected to be conservative and bias towards -inferring standard types like ``str``. Type checkers should infer ``Literal[...]`` -only in contexts where a Literal type is explicitly requested. - -Variable assignment -------------------- - -When assigning a literal expression to an unannotated variable, the -inferred type of the variable is the original base type, not ``Literal[...]``. -For example:: - - border_color = "blue" - reveal_type(border_color) # Revealed type is 'str' - -This helps ensure we don't break the semantics of any existing code. - -If the user wants ``foo`` to have a literal type, they must -explicitly add an annotation:: - - border_color: Literal["blue"] = "blue" - reveal_types(border_color) # Revealed type is 'Literal["blue"]' - -Or alternatively, use the ``Final`` qualifier:: - - border_color: Final = "blue" - reveal_types(border_color) # Revealed type is 'Final[Literal["blue"]]' +This section describes a few rules regarding type inference and +literals, along with some examples. -The ``Final`` qualifier will automatically infer a ``Literal`` type in -an assignment if the LHS is a literal expression, or an expression of -type ``Literal[...]``. +Backwards compatibility +----------------------- -**TODO:** Link to the PEP draft for the ``Final`` qualifier once it's ready. +When type checkers add support for Literal, it's important they do so +in a way that preserves backwards-compatibility. Code that used to +type check **must** continue to type check after support for Literal +is added. -**Note 1:** A potential third way of declaring a Literal might be to -try using ``Literal`` as a qualifier:: +This is particularly important when performing type inference. For +example, given the statement ``x = "blue"``, should the inferred +type of ``x`` be ``str`` or ``Literal["blue"]``? - foo: Literal = "hello" # Illegal! +This PEP does not require any particular strategy for cases like this, +apart from requiring that backwards compatibility is maintained. -Type checkers should *reject* lines like these. Unlike ``Final`` and ``ClassVar``, -``Literal`` is a *type*, not a *qualifier*. Only qualifiers should infer their -parameters. +For example, one valid strategy might be to analyze the surrounding code +to deduce the appropriate type for ``x``. Another valid strategy would +be to assume that ``x`` is always of type ``str`` and require users to +manually annotate it if they want it to have a Literal type. A third valid +strategy would be to use a hybrid of these two approaches: infer +``Literal["blue"]`` if it's clear from context that's an appropriate type, +and fall back to using ``str`` otherwise. -**Note 2:** Type checkers are only expected to use the context available -to them within the current statement to infer the type of the variable. -They may *optionally* use additional context to infer more precise types. -For example:: +Type checkers should take care to avoid being too over-zealous when performing +type inference. For example, one strategy that does *not* work is always +assuming expressions are Literal types. This naive strategy would cause +programs like the following to start failing when they previously did not:: - def expects_blah(x: Literal[4]) -> None: ... + var = 3 # If we infer 'var' has type Literal[3]... + my_list = [var] # ...and if we infer my_list has type List[Literal[3]]... + my_list.append(4) # ...this call would be a type-error. - def test() -> None: - foo = 4 - expects_blah(foo) +Another example of when this strategy would fail is when setting fields +in objects:: -In this program, it is theoretically possible for a type checker to deduce that -``foo`` is only ever used as input to a function that expects ``Literal[4]`` -and so infer that ``foo`` must have type ``Literal[4]``. + class MyObject: + def __init__(self) -> None: + self.field = 3 # If we infer MyObject.field has type Literal[3]... -While type checkers *may* perform this kind of analysis, they are not obligated -to do so. It is ok to infer that ``foo`` has type ``int``, since there is no -context present in the assignment statement itself that would suggest that ``foo`` -should be a literal type. + m = MyObject() + m.field = 4 # This assignment would no longer type check -Type inference inside calls ---------------------------- +Using non-Literals in Literal contexts +-------------------------------------- -When a literal is used inside of a function call, it will be inferred -as either the original type or the Literal type based on context. For -example, the following snippet is legal:: +Literal types follow the existing rules regarding subtyping with no additional +special-casing. For example, programs like the following are type safe:: def expects_str(x: str) -> None: ... - def expects_literal(x: Literal["foo"]) -> None: ... - - # Legal: "foo" is inferred to be of type 'str' - expects_str("foo") + var: Literal["foo"] = "foo" - # Legal: "foo" is inferred to be of type 'Literal["foo"]' - expects_literal("foo") + # Legal: Literal["foo"] is a subtype of str + expects_str(var) -However, non-literal expressions in general will not automatically be inferred -to be literals. For example:: +This also means non-Literal expressions in general should not automatically +inferred to be Literal. For example:: def expects_literal(x: Literal["foo"]) -> None: ... @@ -422,7 +391,7 @@ Intelligent indexing of structured data --------------------------------------- Literals can be used to "intelligently index" into structured types like -NamedTuple, classes, and TypedDict. (Note: this is not an exhaustive list). +tuples, NamedTuple, and classes. (Note: this is not an exhaustive list). For example, type checkers should infer the correct value type when indexing into a tuple using an int key that corresponds a valid index:: @@ -451,11 +420,6 @@ We expect similar behavior when using functions like getattr:: reveal_type(getattr(t, b)) # Revealed type is 'Callable[[int], str]' getattr(t, c) # Error: 'Test' does not have attribute named 'blah' -These interactions will most likely need to be added to type checkers on -an ad-hoc basis. - -**TODO:** Link to the PEP for TypedDict once it's ready. - Interactions with overloads --------------------------- @@ -526,7 +490,9 @@ or bounds involving Literal types:: S = TypeVar('S', bound=Literal["foo"]) ...although it is unclear when it would ever be useful to construct a -TypeVar with a Literal bound. +TypeVar with a Literal upper bound. For example, the ``S`` TypeVar in +the above example is essentially pointless: we can get equivalent behavior +by using ``S = Literal["foo"]`` instead. **Note:** Literal types and generics deliberately interact in only very basic and limited ways. In particular, libraries that want to typecheck @@ -538,24 +504,13 @@ We considered several different proposals for fixing this, but ultimately decided to defer the problem of integer generics to a later date. See `Rejected or out-of-scope ideas`_ for more details. -Interactions with asserts and other checks ------------------------------------------- +Interactions with type narrowing +-------------------------------- -Type checkers should narrow the type of variables when they are compared -directly against other literal types. For example:: - - def foo(x: str) -> None: - if x == "foo": - # Type checker should narrow 'x' to "foo" here - expects_foo(x) - - # Similarly, type checker should narrow 'x' to "bar" here - assert x == "bar" - expects_bar(x) - -This includes with enums. For example, the type checker should be capable -of inferring that the final ``else`` statement in the following function -is unreachable:: +Type checkers should be capable of performing exhaustibility checks when +working Literal types that have a closed number of variants, such as +enums. For example, the type checker should be capable of inferring that the +final ``else`` statement in the following function is unreachable:: class Status(Enum): SUCCESS = 0 @@ -579,7 +534,7 @@ is unreachable:: This behavior is technically not new: this behavior is `already codified within PEP 484 `_. However, many type -checkers (such as mypy) do not yet implement this behavior. Once literal +checkers (such as mypy) do not yet implement this behavior. Once Literal types are introduced, it will become easier to do so: we can model enums as being approximately equal to the union of their values. So, ``Status`` would be treated as being approximately equal to @@ -588,23 +543,42 @@ enums as being approximately equal to the union of their values. So, Type checkers may optionally perform additional analysis and narrowing beyond what is described above. -Interactions with Final types ------------------------------ +For example, it may be useful to perform narrowing based on things like +containment or equality checks:: -The interactions between final and literal types were previously -mentioned above, but to reiterate: if a variable is annotated as -``Final`` and has a literal expression on the RHS, the inferred type -is Literal:: + def parse_status(status: str) -> None: + if status in ("MALFORMED", "ABORTED"): + # Type checker could narrow 'status' to type + # Literal["MALFORMED", "ABORTED"] here. + return expects_bad_status(status) + + # Similarly, type checker could narrow 'x' to Literal["PENDING"] + if status == "PENDING": + expects_pending_status(status) - root_id: Final = 1 +It may also be useful to perform narrowing taking into account expressions +involving Literal bools. For example, we can combine ``Literal[True]``, +``Literal[False]``, and overloads to construct "custom type guards":: - # Revealed type should be 'Final[Literal[1]]' or something similar - reveal_type(root_id) + @overload + def is_int_like(x: Union[int, List[int]]) -> Literal[True]: ... + @overload + def is_int_like(x: Union[str, List[str]]) -> Literal[False]: ... + def is_int_like(x): ... - # The types of 'root_id' and 'root_id_2' should be identical - root_id_2: Final[Literal[1]] = 1 + vector: List[int] = [1, 2, 3] + if is_int_like(vector): + vector.append(3) + else: + vector.append("bad") # This branch is inferred to be unreachable + + scalar: Union[int, str] + if is_int_like(scalar): + scalar += 3 # Type checks: type of 'scalar' is narrowed to 'int' + else: + scalar += "foo" # Type checks: type of 'scalar' is narrowed to 'str' + -**TODO:** Cross-link to draft PEP for 'Final' once it's ready Rejected or out-of-scope ideas ============================== @@ -642,8 +616,8 @@ like variadic generics to support popular libraries like numpy. This PEP should be seen as a stepping stones towards this goal, rather then an attempt at providing a comprehensive solution. -Adding more concise syntax for literal types --------------------------------------------- +Adding more concise syntax +-------------------------- One objection to this PEP is that having to explicitly write ``Literal[...]`` feels verbose. For example, instead of writing:: @@ -670,18 +644,29 @@ Running this yields the following exception:: TypeError: Tuple[t0, t1, ...]: each t must be a type. Got 1. -We don’t want users to have to memorize exactly when it’s ok to elide ``Literal``, -so we require ``Literal`` to always be present. +We don’t want users to have to memorize exactly when it’s ok to elide +``Literal``, so we require ``Literal`` to always be present. -Backwards compatibility -======================= +A little more broadly, we feel overhauling the syntax of types in +Python is not within the scope of this PEP: it would be best to have +that discussion in a separate PEP, instead of attaching it to this one. +So, this PEP deliberately does not try and innovate Python's type syntax. + +Backporting the ``Literal`` type +================================ Once this PEP is accepted, the ``Literal`` type will need to be backported for Python versions that come bundled with older versions of the ``typing`` module. We plan to do this by adding ``Literal`` to the ``typing_extensions`` 3rd party -module, along with the other backported types. +module, which contains a variety of other backported types. + +Implementation +============== + +The mypy type checker currently has implemented a large subset of the behavior +described in this spec, with the exception of enum Literals and some of the +more complex narrowing interactions described above. -There should be no backwards compatibility issues apart from this. Related work ============ @@ -714,6 +699,17 @@ something similar to how .. _pep-484-enums: https://www.python.org/dev/peps/pep-0484/#support-for-singleton-types-in-unions +Acknowledgements +================ + +Thanks to Mark Mendoza, Ran Benita, Rebecca Chen, and the other members of +typing-sig for their comments on this PEP. + +Additional thanks to the various participants in the mypy and typing issue +tracker, who helped provide a lot of the motivation and reasoning behind +this PEP. + + Copyright ========= From 79253ddd13eafbe4675461baa674a19526ca6ba2 Mon Sep 17 00:00:00 2001 From: Michael Lee Date: Wed, 13 Mar 2019 16:09:10 -0700 Subject: [PATCH 12/18] Make lines fit under 79 chars; tweak some wording --- pep-9999.rst | 107 ++++++++++++++++++++++++++++----------------------- 1 file changed, 58 insertions(+), 49 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 6ffff5b9148..663a645c8a6 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -148,8 +148,8 @@ Legal and illegal parameterizations This section describes what exactly constitutes a legal ``Literal[...]`` type: what values may and may not be used as parameters. -In short, a ``Literal[...]`` type may be parameterized by one or more literal expressions, -and nothing else. +In short, a ``Literal[...]`` type may be parameterized by one or more literal +expressions, and nothing else. Legal parameters for ``Literal`` at type check time @@ -181,7 +181,8 @@ to other literal types. For example, the following is legal:: WriteNoTruncateMode = Literal["r+", "r+t"] AppendMode = Literal["a", "a+", "at", "a+t"] - AllModes = Literal[ReadOnlyMode, WriteAndTruncateMode, WriteNoTruncateMode, AppendMode] + AllModes = Literal[ReadOnlyMode, WriteAndTruncateMode, + WriteNoTruncateMode, AppendMode] This feature is again intended to help make using and reusing literal types more ergonomic. @@ -199,15 +200,16 @@ This should be exactly equivalent to the following type:: Optional[Literal[1, 2, 3, "foo", 5]] -**Note:** String literal types like ``Literal["foo"]`` should subtype either bytes or -unicode in the same way regular string literals do at runtime. +**Note:** String literal types like ``Literal["foo"]`` should subtype either +bytes or unicode in the same way regular string literals do at runtime. -For example, in Python 3, the type ``Literal["foo"]`` is equivalent to ``Literal[u"foo"]``, -since ``"foo"`` is equivalent to ``u"foo"`` in Python 3. +For example, in Python 3, the type ``Literal["foo"]`` is equivalent to +``Literal[u"foo"]``, since ``"foo"`` is equivalent to ``u"foo"`` in Python 3. -Similarly, in Python 2, the type ``Literal["foo"]`` is equivalent to ``Literal[b"foo"]`` -- -unless the file includes a ``from __future__ import unicode_literals`` import, in which case -it would be equivalent to ``Literal[u"foo"]``. +Similarly, in Python 2, the type ``Literal["foo"]`` is equivalent to +``Literal[b"foo"]`` -- unless the file includes a +``from __future__ import unicode_literals`` import, in which case it would be +equivalent to ``Literal[u"foo"]``. Illegal parameters for ``Literal`` at type check time ----------------------------------------------------- @@ -222,9 +224,10 @@ The following parameters are intentionally disallowed by design: checkers to interpret potentially expressions inside types adds too much complexity. Also see `Rejected or out-of-scope ideas`_. - - As a consequence, complex numbers like ``Literal[4 + 3j]`` and ``Literal[-4 + 2j]`` - are also prohibited. For consistency, literals like ``Literal[4j]`` that contain - just a single complex number are also prohibited. + - As a consequence, complex numbers like ``Literal[4 + 3j]`` and + ``Literal[-4 + 2j]`` are also prohibited. For consistency, literals like + ``Literal[4j]`` that contain just a single complex number are also + prohibited. - The only exception to this rule is the unary ``-`` (minus) for ints: types like ``Literal[-5]`` are *accepted*. @@ -244,8 +247,8 @@ The following parameters are intentionally disallowed by design: This includes typevars: if ``T`` is a typevar, ``Literal[T]`` is not allowed. Typevars can vary over only types, never over values. -The following are provisionally disallowed for simplicity. We can consider allowing -them on a case-by-case basis based on demand. +The following are provisionally disallowed for simplicity. We can consider +allowing them on a case-by-case basis based on demand. - Floats: e.g. ``Literal[3.14]``. Note: if we do decide to allow floats, we should likely disallow literal infinity and literal NaN. @@ -333,29 +336,34 @@ apart from requiring that backwards compatibility is maintained. For example, one valid strategy might be to analyze the surrounding code to deduce the appropriate type for ``x``. Another valid strategy would be to assume that ``x`` is always of type ``str`` and require users to -manually annotate it if they want it to have a Literal type. A third valid -strategy would be to use a hybrid of these two approaches: infer -``Literal["blue"]`` if it's clear from context that's an appropriate type, -and fall back to using ``str`` otherwise. - -Type checkers should take care to avoid being too over-zealous when performing -type inference. For example, one strategy that does *not* work is always -assuming expressions are Literal types. This naive strategy would cause +manually annotate it if they want it to have a Literal type. A third strategy +might be to use a hybrid of the previous two. + +However, type checkers should take care to avoid being too over-zealous +when performing inference. For example, one strategy that does *not* work is +always assuming expressions are Literal types. This naive strategy would cause programs like the following to start failing when they previously did not:: - var = 3 # If we infer 'var' has type Literal[3]... - my_list = [var] # ...and if we infer my_list has type List[Literal[3]]... - my_list.append(4) # ...this call would be a type-error. + # If we infer 'var' has type Literal[3] + # and my_list has type List[Literal[3]]... + var = 3 + my_list = [var] + + # ...this call would be a type-error. + my_list.append(4) Another example of when this strategy would fail is when setting fields in objects:: class MyObject: def __init__(self) -> None: - self.field = 3 # If we infer MyObject.field has type Literal[3]... + # If we infer MyObject.field has type Literal[3]... + self.field = 3 m = MyObject() - m.field = 4 # This assignment would no longer type check + + # ...this assignment would no longer type check + m.field = 4 Using non-Literals in Literal contexts -------------------------------------- @@ -418,7 +426,7 @@ We expect similar behavior when using functions like getattr:: t = Test() reveal_type(getattr(t, a)) # Revealed type is 'int' reveal_type(getattr(t, b)) # Revealed type is 'Callable[[int], str]' - getattr(t, c) # Error: 'Test' does not have attribute named 'blah' + getattr(t, c) # Error: No attribute named 'blah' in Test Interactions with overloads --------------------------- @@ -514,18 +522,15 @@ final ``else`` statement in the following function is unreachable:: class Status(Enum): SUCCESS = 0 - PARSE_ERROR = 1 - INVALID_DATA = 2 - FATAL_ERROR = 3 + INVALID_DATA = 1 + FATAL_ERROR = 2 - def parse_status(status: Status) -> None: - if status is Status.SUCCESS: + def parse_status(s: Status) -> None: + if s is Status.SUCCESS: print("Success!") - elif status is Status.PARSE_ERROR: - print("Unable to deserialize data") - elif status is Status.INVALID_DATA: + elif s is Status.INVALID_DATA: print("The given data is invalid because...") - elif status is Status.FATAL_ERROR: + elif s is Status.FATAL_ERROR: print("Unexpected fatal error...") else: # Error should not be reported by type checkers that @@ -536,9 +541,13 @@ This behavior is technically not new: this behavior is `already codified within PEP 484 `_. However, many type checkers (such as mypy) do not yet implement this behavior. Once Literal types are introduced, it will become easier to do so: we can model -enums as being approximately equal to the union of their values. So, -``Status`` would be treated as being approximately equal to -``Literal[Status.SUCCESS, Status.PARSE_ERROR, Status.INVALID_DATA, Status.FATAL_ERROR]``. +enums as being approximately equal to the union of their values and +take advantage of any existing logic regarding unions, exhaustibility, +and type narrowing. + +So here, ``Status`` could be treated as being approximately equal to +``Literal[Status.SUCCESS, Status.INVALID_DATA, Status.FATAL_ERROR]`` +and the type of ``s`` narrowed accordingly. Type checkers may optionally perform additional analysis and narrowing beyond what is described above. @@ -600,18 +609,18 @@ let us write signatures like the below:: # The type checker will statically verify our function genuinely does # construct a vector that is equal in length to "len(vec1) + len(vec2)" # and will throw an error if it does not. - def vector_concat(vec1: Vector[A, T], vec2: Vector[B, T]) -> Vector[A + B, T]: + def concat(vec1: Vector[A, T], vec2: Vector[B, T]) -> Vector[A + B, T]: # ...snip... At the very least, it would be useful to add some form of integer generics. Although such a type system would certainly be useful, it’s out-of-scope -for this PEP: it would require a far more substantial amount of implementation work, -discussion, and research to complete compared to the current proposal. +for this PEP: it would require a far more substantial amount of implementation +work, discussion, and research to complete compared to the current proposal. It's entirely possible we'll circle back and revisit this topic in the future: -we very likely will need some form of dependent typing along with other extensions -like variadic generics to support popular libraries like numpy. +we very likely will need some form of dependent typing along with other +extensions like variadic generics to support popular libraries like numpy. This PEP should be seen as a stepping stones towards this goal, rather then an attempt at providing a comprehensive solution. @@ -619,8 +628,8 @@ rather then an attempt at providing a comprehensive solution. Adding more concise syntax -------------------------- -One objection to this PEP is that having to explicitly write ``Literal[...]`` feels -verbose. For example, instead of writing:: +One objection to this PEP is that having to explicitly write ``Literal[...]`` +feels verbose. For example, instead of writing:: def foobar(arg1: Literal[1], arg2: Literal[True]) -> None: pass @@ -706,7 +715,7 @@ Thanks to Mark Mendoza, Ran Benita, Rebecca Chen, and the other members of typing-sig for their comments on this PEP. Additional thanks to the various participants in the mypy and typing issue -tracker, who helped provide a lot of the motivation and reasoning behind +trackers, who helped provide a lot of the motivation and reasoning behind this PEP. From fb9ada2022cc4ecbf176f6a677b11d88917eb43f Mon Sep 17 00:00:00 2001 From: Michael Lee Date: Wed, 13 Mar 2019 16:11:35 -0700 Subject: [PATCH 13/18] Remove references to Final and TypedDict --- pep-9999.rst | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 663a645c8a6..1a3c22246a5 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -242,10 +242,10 @@ The following parameters are intentionally disallowed by design: set literals: literals are always implicitly final and immutable. So, ``Literal[{"a": "b", "c": "d"}]`` is illegal. -- Any other types: for example, ``Literal[MyTypedDict]``, or - ``Literal[some_object_instance]`` are illegal. - This includes typevars: if ``T`` is a typevar, ``Literal[T]`` is - not allowed. Typevars can vary over only types, never over values. +- Any other types: for example, ``Literal[Path]``, or + ``Literal[some_object_instance]`` are illegal. This includes typevars: if + ``T`` is a typevar, ``Literal[T]`` is not allowed. Typevars can vary over + only types, never over values. The following are provisionally disallowed for simplicity. We can consider allowing them on a case-by-case basis based on demand. @@ -404,8 +404,8 @@ tuples, NamedTuple, and classes. (Note: this is not an exhaustive list). For example, type checkers should infer the correct value type when indexing into a tuple using an int key that corresponds a valid index:: - a: Final = 0 - b: Final = 5 + a: Literal[0] = 0 + b: Literal[5] = 5 some_tuple: Tuple[int, str, List[bool]] = (3, "abc", [True, False]) reveal_type(some_tuple[a]) # Revealed type is 'int' @@ -419,9 +419,9 @@ We expect similar behavior when using functions like getattr:: def mymethod(self, val: int) -> str: ... - a: Final = "myfield" - b: Final = "mymethod" - c: Final = "blah" + a: Literal["myfield"] = "myfield" + b: Literal["mymethod"] = "mymethod" + c: Literal["blah"] = "blah" t = Test() reveal_type(getattr(t, a)) # Revealed type is 'int' From 3ddaa0ce6c72c112ae189084b5fe158ea138aa63 Mon Sep 17 00:00:00 2001 From: Michael Lee Date: Wed, 13 Mar 2019 19:24:07 -0700 Subject: [PATCH 14/18] Respond to Mark's feedback --- pep-9999.rst | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 1a3c22246a5..0d8da6a47ef 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -333,18 +333,20 @@ type of ``x`` be ``str`` or ``Literal["blue"]``? This PEP does not require any particular strategy for cases like this, apart from requiring that backwards compatibility is maintained. -For example, one valid strategy might be to analyze the surrounding code -to deduce the appropriate type for ``x``. Another valid strategy would -be to assume that ``x`` is always of type ``str`` and require users to -manually annotate it if they want it to have a Literal type. A third strategy -might be to use a hybrid of the previous two. - -However, type checkers should take care to avoid being too over-zealous -when performing inference. For example, one strategy that does *not* work is -always assuming expressions are Literal types. This naive strategy would cause -programs like the following to start failing when they previously did not:: - - # If we infer 'var' has type Literal[3] +For example, one simple strategy for meeting this requirement would be +to always assume expressions are *not* Literal types unless they are +explicitly annotated otherwise. A type checker using this strategy would +always infer that ``x`` is of type ``str`` in the above example. + +If type checkers choose to use more sophisticated inference strategies +(e.g. using the surrounding context to infer the appropriate type for ``x``), +they should avoid being too over-zealous while doing so. + +For example, one strategy that does *not* work is always assuming expressions +are Literal types. This naive strategy would cause programs like the +following to start failing when they previously did not:: + + # If a type checker infers 'var' has type Literal[3] # and my_list has type List[Literal[3]]... var = 3 my_list = [var] @@ -357,7 +359,7 @@ in objects:: class MyObject: def __init__(self) -> None: - # If we infer MyObject.field has type Literal[3]... + # If a type checker infers MyObject.field has type Literal[3]... self.field = 3 m = MyObject() From 8006152785ecbf6ca0d1aa8a599a02837cae1bb9 Mon Sep 17 00:00:00 2001 From: Michael Lee Date: Wed, 13 Mar 2019 20:58:00 -0700 Subject: [PATCH 15/18] Remove one e.g. --- pep-9999.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 0d8da6a47ef..cc3ce23558e 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -338,8 +338,7 @@ to always assume expressions are *not* Literal types unless they are explicitly annotated otherwise. A type checker using this strategy would always infer that ``x`` is of type ``str`` in the above example. -If type checkers choose to use more sophisticated inference strategies -(e.g. using the surrounding context to infer the appropriate type for ``x``), +If type checkers choose to use more sophisticated inference strategies, they should avoid being too over-zealous while doing so. For example, one strategy that does *not* work is always assuming expressions From e6d7c1540804017ac74cb9f28e6feea814660c29 Mon Sep 17 00:00:00 2001 From: Michael Lee Date: Thu, 14 Mar 2019 14:01:29 -0700 Subject: [PATCH 16/18] Rebase; change PEP number --- pep-9999.rst => pep-0586.rst | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename pep-9999.rst => pep-0586.rst (100%) diff --git a/pep-9999.rst b/pep-0586.rst similarity index 100% rename from pep-9999.rst rename to pep-0586.rst From 9d46b3c158c6f4a91945b13d8304aee76c947661 Mon Sep 17 00:00:00 2001 From: Michael Lee Date: Thu, 14 Mar 2019 14:05:11 -0700 Subject: [PATCH 17/18] Update headers --- pep-0586.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/pep-0586.rst b/pep-0586.rst index cc3ce23558e..b494f9c3d97 100644 --- a/pep-0586.rst +++ b/pep-0586.rst @@ -1,13 +1,13 @@ -PEP: 9999 -Title: Literal types +PEP: 586 +Title: Literal Types Author: Michael Lee , Ivan Levkivskyi , Jukka Lehtosalo +Discussions-To: Typing-Sig Status: Draft Type: Standards Track -Python-Version: 3.8 Content-Type: text/x-rst -Created: 13-Mar-2018 -Post-History: 13-Mar-2018 -Discussions-To: Typing-Sig +Created: 14-Mar-2018 +Python-Version: 3.8 +Post-History: 14-Mar-2018 Abstract ======== From 32582e652261d658fcae6f0baac1ad68631108b8 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 14 Mar 2019 15:18:32 -0700 Subject: [PATCH 18/18] Apply suggestions from Jelle's code review Co-Authored-By: Michael0x2a --- pep-0586.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/pep-0586.rst b/pep-0586.rst index b494f9c3d97..f5aedcc60e9 100644 --- a/pep-0586.rst +++ b/pep-0586.rst @@ -110,7 +110,7 @@ has type ``int`` and ``False`` has type ``bool``. Shortening unions of literals ----------------------------- -Literals are parameterized with one or more value. When a Literal is +Literals are parameterized with one or more values. When a Literal is parameterized with more than one value, it's treated as exactly equivalent to the union of those types. That is, ``Literal[v1, v2, v3]`` is equivalent to ``Union[Literal[v1], Literal[v2], Literal[v3]]``. @@ -436,7 +436,7 @@ Literal types and overloads do not need to interact in a special way: the existing rules work fine. However, one important use case type checkers must take care to -support is the ability a *fallback* when the user is not using literal +support is the ability to use a *fallback* when the user is not using literal types. For example, consider ``open``:: _PathType = Union[str, bytes, int] @@ -516,7 +516,7 @@ decided to defer the problem of integer generics to a later date. See Interactions with type narrowing -------------------------------- -Type checkers should be capable of performing exhaustibility checks when +Type checkers should be capable of performing exhaustiveness checks when working Literal types that have a closed number of variants, such as enums. For example, the type checker should be capable of inferring that the final ``else`` statement in the following function is unreachable::