Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PEP 646: Update draft #1856

Merged
merged 1 commit into from
Mar 3, 2021
Merged

PEP 646: Update draft #1856

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
163 changes: 71 additions & 92 deletions pep-0646.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -476,38 +476,14 @@ Type variable tuples can also be used in the arguments section of a
Process(target=func, args=(0, 'foo')) # Valid
Process(target=func, args=('foo', 0)) # Error

However, note that as of this PEP, if a type variable tuple does appear in
the arguments section of a ``Callable``, it must appear alone.
That is, `Type Concatenation`_ is not supported in the context of ``Callable``.
(Use cases where this might otherwise be desirable are likely covered through use
of either a ``ParamSpec`` from PEP 612, or a type variable tuple in the ``__call__``
signature of a callback protocol from PEP 544.)

Type Variable Tuples with ``Union``
-----------------------------------

Type variable tuples can also be used with ``Union``:

::

def f(*args: *Ts) -> Union[*Ts]:
return random.choice(args)

f(1, 'foo') # Inferred type is Union[int, str]

Here, if the type variable tuple is empty (e.g. if we had ``*args: *Ts``
and didn't pass any arguments), the type checker should
raise an error on the ``Union`` (matching the behaviour of ``Union``
at runtime, which requires at least one type argument).

Other types can also be included in the ``Union``:
Other types and normal type variables can also be prefixed/suffixed
to the type variable tuple:

::

def f(*args :*Ts) -> Union[int, str, *Ts]: ...
T = TypeVar('T')

However, note that as elsewhere, only a single type variable tuple
may occur within a ``Union``.
def foo(f: Callable[[int, *Ts, T], Tuple[T, *Ts]]): ...

Aliases
-------
Expand All @@ -518,33 +494,65 @@ a similar way to regular type variables:
::

IntTuple = Tuple[int, *Ts]
NamedArray = Tuple[str, Array[*Ts]]

IntTuple[float, bool] # Equivalent to Tuple[int, float, bool]
NamedArray[Height] # Equivalent to Tuple[str, Array[Height]]

As this example shows, all type parameters passed to the alias are
bound to the type variable tuple. If no type parameters are given,
or if an explicitly empty list of type parameters are given,
type variable tuple in the alias is simply ignored:
bound to the type variable tuple.

Importantly for our ``Array`` example (see `Summary Examples`_), this
allows us to define convenience aliases for arrays of a fixed shape
or datatype:

::

# Both equivalent to Tuple[int]
IntTuple
IntTuple[()]
Shape = TypeVarTuple('Shape')
DType = TypeVar('DType')
class Array(Generic[DType, *Shape]):

Normal ``TypeVar`` instances can also be used in such aliases:
# E.g. Float32Array[Height, Width, Channels]
Float32Array = Array[np.float32, *Shape]

# E.g. Array1D[np.uint8]
Array1D = Array[DType, Any]

If an explicitly empty type parameter list is given, the type variable
tuple in the alias is set empty:

::

T = TypeVar('T')
Foo = Tuple[T, *Ts]
IntTuple[()] # Equivalent to Tuple[int]
NamedArray[()] # Equivalent to Tuple[str, Array[()]]

If the type parameter list is omitted entirely, the alias is
compatible with arbitrary type parameters:

::

# T is bound to `int`; Ts is bound to `bool, str`
Foo[int, bool, str]
def takes_float_array_of_any_shape(x: Float32Array): ...
x: Float32Array[Height, Width] = Array()
takes_float_array_of_any_shape(x) # Valid

Note that the same rules for `Type Concatenation`_ apply for aliases.
In particular, only one ``TypeVarTuple`` may occur within an alias,
and the ``TypeVarTuple`` must be at the end of the alias.
def takes_float_array_with_specific_shape(y: Float32Array[Height, Width]): ...
y: Float32Array = Array()
takes_float_array_with_specific_shape(y) # Valid

Normal ``TypeVar`` instances can also be used in such aliases:

::

T = TypeVar('T')
Foo = Tuple[*Ts, T]

# Ts bound to Tuple[int], T to int
Foo[str, int]
# Ts bound to Tuple[()], T to int
Foo[int]
# T bound to Any, Ts to an arbitrary number of Any
Foo

Overloads for Accessing Individual Types
----------------------------------------

Expand Down Expand Up @@ -574,51 +582,6 @@ overloads for each possible rank is, of course, a rather cumbersome
solution. However, it's the best we can do without additional type
manipulation mechanisms, which are beyond the scope of this PEP.)

An Ideal Array Type: One Possible Example
=========================================

Type variable tuples allow us to make significant progress on the
typing of arrays. However, the array class we have sketched
out in this PEP is still missing some desirable features. [#typing-ideas]_

The most crucial feature missing is the ability to specify
the data type (e.g. ``np.float32`` or ``np.uint8``). This is important
because some numerical computing libraries will silently cast
types, which can easily lead to hard-to-diagnose bugs.

Additionally, it might be useful to be able to specify the rank
instead of the full shape. This could be useful for cases where
axes don't have obvious semantic meaning like 'height' or 'width',
or where the array is very high-dimensional and writing out all
the axes would be too verbose.

Here is one possible example of how these features might be implemented
in a complete array type.

::

# E.g. Ndim[Literal[3]]
Integer = TypeVar('Integer')
class Ndim(Generic[Integer]): ...

# E.g. Shape[Height, Width]
# (Where Height and Width are custom types)
Axes = TypeVarTuple('Axes')
class Shape(Generic[*Axes]): ...

DataType = TypeVar('DataType')
ShapeType = TypeVar('ShapeType', Ndim, Shape)

# The most verbose type
# E.g. Array[np.float32, Ndim[Literal[3]]
# Array[np.uint8, Shape[Height, Width, Channels]]
class Array(Generic[DataType, ShapeType]): ...

# Type aliases for less verbosity
# E.g. Float32Array[Height, Width, Channels]
Float32Array = Array[np.float32, Shape[*Axes]]
# E.g. Array1D[np.uint8]
Array1D = Array[DataType, Ndim[Literal[1]]]

Rationale and Rejected Ideas
============================
Expand Down Expand Up @@ -710,20 +673,32 @@ in how much of their code they wish to annotate, and to enable compatibility
between old unannotated code and new versions of libraries which do use
these type annotations.


Backwards Compatibility
=======================

TODO
The ``Unpack`` version of the PEP should be back-portable to previous
versions of Python.

* ``Tuple`` needs to be upgraded to support parameterization with a
type variable tuple.
Gradual typing is enabled by the fact that unparameterised variadic classes
are compatible with an arbitrary number of type parameters. This means
that if existing classes are made generic, a) all existing (unparameterised)
uses of the class will still work, and b) parameterised and unparameterised
versions of the class can be used together (relevant if, for example, library
code is updated to use parameters while user code is not, or vice-versa).


Reference Implementation
========================

Two reference implementations exist: one in Pyre, as of TODO, and one in
Pyright, as of v1.1.108.
Two reference implementations of type-checking functionality exist:
one in Pyre, as of TODO, and one in Pyright, as of v1.1.108.

A preliminary implementation of the ``Unpack`` version of the PEP in CPython
is available in `cpython/23527`_. A preliminary version of the version
using the star operator, based on an early implementation of PEP 637,
is also available at `mrahtz/cpython/pep637+646`_.


Footnotes
==========
Expand Down Expand Up @@ -758,6 +733,10 @@ References

.. [#arbitrary_len] Discussion on Python typing-sig mailing list: https://mail.python.org/archives/list/[email protected]/thread/SQVTQYWIOI4TIO7NNBTFFWFMSMS2TA4J/

.. _cpython/23527: https://github.com/python/cpython/pull/24527

.. _mrahtz/cpython/pep637+646: https://github.com/mrahtz/cpython/tree/pep637%2B646


Acknowledgements
================
Expand Down