Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add info about classes and types to 'getting started' docs #6557

2 changes: 2 additions & 0 deletions docs/source/class_basics.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
.. _class-basics:

Class basics
============

Expand Down
154 changes: 122 additions & 32 deletions docs/source/getting_started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Getting started
===============

This chapter introduces some core concepts of mypy, including function
annotations, the :py:mod:`typing` module, library stubs, and more.
annotations, the :py:mod:`typing` module, stub files, and more.

Be sure to read this chapter carefully, as the rest of the documentation
may not make much sense otherwise.
Expand Down Expand Up @@ -319,37 +319,136 @@ syntax like so:

.. _stubs-intro:
97littleleaf11 marked this conversation as resolved.
Show resolved Hide resolved

Library stubs and typeshed
**************************
Types and classes
*****************

Mypy uses library *stubs* to type check code interacting with library
modules, including the Python standard library. A library stub defines
a skeleton of the public interface of the library, including classes,
variables and functions, and their types. Mypy ships with stubs for
the standard library from the `typeshed
<https://github.com/python/typeshed>`_ project, which contains library
stubs for the Python builtins, the standard library, and selected
third-party packages.
So far, we've only seen examples of pre-existing types like the ``int``
or ``float`` builtins, or generic types from ``collections.abc`` and
``typing``, such as ``Iterable``. However, these aren't the only types you can
use: in fact, you can use any Python class as a type!

For example, consider this code:
For example, suppose you've defined a custom class representing a bank account:

.. code-block:: python

x = chr(4)
class BankAccount:
# Note: It is ok to omit type hints for the "self" parameter.
# Mypy will infer the correct type.

Without a library stub, mypy would have no way of inferring the type of ``x``
and checking that the argument to :py:func:`chr` has a valid type.
def __init__(self, account_name: str, initial_balance: int = 0) -> None:
# Note: Mypy will infer the correct types of your fields
# based on the types of the parameters.
self.account_name = account_name
self.balance = initial_balance

Mypy complains if it can't find a stub (or a real module) for a
library module that you import. Some modules ship with stubs or inline
annotations that mypy can automatically find, or you can install
additional stubs using pip (see :ref:`fix-missing-imports` and
:ref:`installed-packages` for the details). For example, you can install
the stubs for the ``requests`` package like this:
def deposit(self, amount: int) -> None:
self.balance += amount

def withdraw(self, amount: int) -> None:
self.balance -= amount

def overdrawn(self) -> bool:
return self.balance < 0

You can declare that a function will accept any instance of your class
by simply annotating the parameters with ``BankAccount``:

.. code-block:: python

def transfer(src: BankAccount, dst: BankAccount, amount: int) -> None:
src.withdraw(amount)
dst.deposit(amount)

account_1 = BankAccount('Alice', 400)
account_2 = BankAccount('Bob', 200)
transfer(account_1, account_2, 50)

In fact, the ``transfer`` function we wrote above can accept more then just
instances of ``BankAccount``: it can also accept any instance of a *subclass*
of ``BankAccount``. For example, suppose you write a new class that looks like this:

.. code-block:: python

class AuditedBankAccount(BankAccount):
def __init__(self, account_name: str, initial_balance: int = 0) -> None:
super().__init__(account_name, initial_balance)

# In this case, mypy can't infer the exact type of this field
# based on the information available in this constructor, so we
# need to add a type annotation.
self.audit_log: List[str] = []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not massively keen on an example that requires a three-line comment, especially since this is a beginner's tutorial. Can we think of a different example here, that doesn't involve having to instantiate an empty list? (I'm also trying to think of one.)

In any event, we should use PEP 585 syntax here :)

Suggested change
# In this case, mypy can't infer the exact type of this field
# based on the information available in this constructor, so we
# need to add a type annotation.
self.audit_log: List[str] = []
# In this case, mypy can't infer the exact type of this field
# based on the information available in this constructor, so we
# need to add a type annotation.
self.audit_log: list[str] = []

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a transaction count as an int?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that idea!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO it's not really a big deal since typing an empty list is a frequent case (personal feeling).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mypy would be able to infer an int

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to keep the list[str] but remove the comments since the rule has been introduced in previous section


def deposit(self, amount: int) -> None:
self.audit_log.append(f"Deposited {amount}")
self.balance += amount

def withdraw(self, amount: int) -> None:
self.audit_log.append(f"Withdrew {amount}")
self.balance -= amount

Since ``AuditedBankAccount`` is a subclass of ``BankAccount``, we can directly pass
in instances of it into our ``transfer`` function:

.. code-block:: python

audited = AuditedBankAccount('Charlie', 300)
transfer(account_1, audited, 100) # Type checks!

This behavior is actually a fundamental aspect of the PEP 484 type system: when
we annotate some variable with a type ``T``, we are actually telling mypy that
variable can be assigned an instance of ``T``, or an instance of a *subclass* of ``T``.
The same rule applies to type hints on parameters or fields.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we mention that mypy enforces LSP to make sure this is actually safe (maybe in a footnote)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to mention that in this article, since this is a beginner's tutorial.


See :ref:`class-basics` to learn more about how to work with code involving classes.


.. _stubs-intro:

Stubs files and typeshed
************************

Mypy also understands how to work with classes found in the standard library.
For example, here is a function which uses the ``Path`` object from the
`pathlib standard library module <https://docs.python.org/3/library/pathlib.html>`_:

.. code-block:: python

from pathlib import Path

def load_template(template_path: Path, name: str) -> str:
# Mypy understands that 'file_path.read_text()' returns a str...
template = template_path.read_text()

# ...so understands this line type checks.
return template.replace('USERNAME', name)

This behavior may surprise you if you're familiar with how
Python internally works. The standard library does not use type hints
anywhere, so how did mypy know that ``Path.read_text()`` returns a ``str``,
or that ``str.replace(...)`` accepts exactly two ``str`` arguments?

The answer is that mypy comes bundled with *stub files* from the
the `typeshed <https://github.com/python/typeshed>`_ project.

A *stub file* is a file containing a skeleton of the public interface
of that Python module, including classes, variables, functions -- and
most importantly, their types. Typeshed is a collection of stub files
for modules in the standard library and select third party libraries
and is what mypy used to identify the correct types for our example above.

Mypy can understand third party libraries in the same way as long as those
libraries come bundled with stub files or have inline annotations that mypy
can automatically find. (see :ref:`installed-packages` for the details).
However, if the third party library does *not* come bundled with type hints,
mypy will not try and guess what the types are: it'll assume the entire library
is :ref:`dynamically typed <dynamic-typing>` and report an error whenever you
import the library.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JukkaL I tried to merge the conflicts. But I am not sure about this parts.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this section (from lines 452-487) should be moved to running_mypy.rst, or maybe just cut altogether. With the changes made in this PR, it's a very different topic to the rest of the article, and I don't think we need it here since it's covered in depth elsewhere.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the changes by Michael. But I keep the current example since it's quite useful for beginners IMHO.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd vote for cutting this section entirely. At this point, typeshed is good enough that users don't need to know at all about typeshed and beginner users don't really need to know about stubs to get started

You can install the stubs for third-party packages like this:

.. code-block:: shell

python3 -m pip install types-requests
$ python3 -m pip install types-requests

The stubs are usually packaged in a distribution named
``types-<distribution>``. Note that the distribution name may be
Expand All @@ -363,17 +462,8 @@ often suggest the name of the stub distribution:
prog.py:1: note: Hint: "python3 -m pip install types-PyYAML"
...

.. note::

Starting in mypy 0.900, most third-party package stubs must be
installed explicitly. This decouples mypy and stub versioning,
allowing stubs to updated without updating mypy. This also allows
stubs not originally included with mypy to be installed. Earlier
mypy versions included a fixed set of stubs for third-party
packages.

You can also :ref:`create
stubs <stub-files>` easily. We discuss ways of silencing complaints
stubs <stub-files>` easily. We discuss strategies for handling errors
about missing stubs in :ref:`ignore-missing-imports`.

Configuring mypy
Expand Down
9 changes: 9 additions & 0 deletions docs/source/installed_packages.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,15 @@ you can create such packages.
example), it is recommended that you also pin the versions of all
your stub package dependencies.

.. note::

Starting in mypy 0.900, most third-party package stubs must be
installed explicitly. This decouples mypy and stub versioning,
allowing stubs to updated without updating mypy. This also allows
stubs not originally included with mypy to be installed. Earlier
mypy versions included a fixed set of stubs for third-party
packages.

Using installed packages with mypy (PEP 561)
********************************************

Expand Down
6 changes: 4 additions & 2 deletions docs/source/running_mypy.rst
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,9 @@ If you are getting this error, try:

2. Searching to see if there is a :ref:`PEP 561 compliant stub package <installed-packages>`.
corresponding to your third party library. Stub packages let you install
type hints independently from the library itself.
type hints independently from the library itself. For example, if you want
type hints for the `sqlalchemy <https://www.sqlalchemy.org/>`_ library,
you can install `sqlalchemy-stubs <https://pypi.org/project/sqlalchemy-stubs/>`_.

For example, if you want type hints for the ``django`` library, you can
install the `django-stubs <https://pypi.org/project/django-stubs/>`_ package.
Expand Down Expand Up @@ -322,7 +324,7 @@ this error, try:
In some rare cases, you may get the "Cannot find implementation or library
stub for module" error even when the module is installed in your system.
This can happen when the module is both missing type hints and is installed
on your system in a unconventional way.
on your system in an unconventional way.

In this case, follow the steps above on how to handle
:ref:`missing type hints in third party libraries <missing-type-hints-for-third-party-library>`.
Expand Down