-
-
Notifications
You must be signed in to change notification settings - Fork 2.9k
Creating Stubs For Python Modules
Note: this page may be outdated. If you want to create/use stubs for 3rd party packages, you may want to look at PEP 561 packages.
The type checker only needs a stubs file to let programs access a Python module. There is no need to port the entire module to mypy. A stubs file is also a good starting point for porting an entire Python module to mypy. They can also highlight potential areas of improvement in the mypy type system.
A stubs file only contains a description of the public interface of the module without any implementations. Stubs can be dynamically typed, statically typed or a mixture of both. Based on preliminary results, it seems that over 95% of the names defined in the Python standard library can be given static types. The rest will be dynamically typed (use type 'Any') or have 'Any' components, but that is fine, since mypy supports seamless mixing of dynamic and static types in programs.
Mypy uses stubs from https://github.com/python/typeshed -- all stub changes should be contributed there. Mypy developers will periodically pull latest changes from typeshed to be included with mypy. There are stubs for both Python 2.7 and 3.x, though not every stub supports both 2 and 3. Some stubs can be shared between Python 2.7 and 3.x.
The file stubs/builtins.py is the stubs file for Python builtins. Here are a few examples from the stubs file (module global variables and a function):
True: bool
False: bool
...
def chr(code: int) -> str: ...
The unittest module stubs contain a good example of a stub for a class:
class TestSuite(Testable):
def __init__(self, tests: Iterable[Testable] = None) -> None: ...
def addTest(self, test: Testable) -> None: ...
def addTests(self, tests: Iterable[Testable]) -> None: ...
def run(self, result: TestResult) -> None: ...
def debug(self) -> None: ...
def countTestCases(self) -> int: ...
Follow these steps to be able to use your newly created stubs file:
- Create a directory named 'stubs' (it can be anywhere, but the name has to be exactly 'stubs').
- Set the environment variable MYPYPATH to refer to the above directory. For example:
$ export MYPYPATH=/home/jukka/work/stubs
- Add the stub file to the stubs directory. Use the normal Python file name conventions for modules, but use
.pyi
as the extension -- e.g. 'csv.pyi' for module 'csv'; use subdirectory with__init__.pyi
for packages. - That's it! Now you can access the module in mypy programs and test that it works correctly.
Use the Python interactive interpreter to introspect modules and try them interactively. You should also consult the Python library reference, of course, since that is more definitive than doc strings. Example:
>>> import os
>>> dir(os)
[..., 'abort', 'access', ...] # List of names defined in os
>>> help(os)
Help on module os:
... (contents omitted)
>>> help(os.abort)
Help on built-in function abort in module Posix:
abort(...):
abort() -> does not return!
...
You can use help() with functions, methods, classes and modules.
Often it's easier to start with dynamically typed stubs for a module. These still let mypy code to access the module, but static checking and type inference will be limited.
Here is a small example:
from typing import Any
x: Any # variable
def f(x, y=None): ... # function
class C:
def __init__(self, foo): ... # method
If you introduce helpers not part of the actual module interface in your stubs, such as type variables or type aliases, prefix them with an underscore. This way if someone uses from m import *
it doesn't pollute the namespace with the internal helpers. Generally you don't need to define __all__
, since it's mostly redundant as stubs mostly have public functionality.
Sometimes it may not be obvious how to add static types to some parts of a Python module. This section covers some of the more common issues.
You can using typing.NamedTuple
to define named tuples with statically typed items. Here is an example:
from typing import NamedTuple
ClassName = NamedTuple('ClassName', [('item', int), ('another', str)])
Sometimes the type of a function return value depends on the value of a variable or other external state. This case generally can't be modelled statically (or it's not worth the complexity). However, this is not a problem per se: just use type 'Any' for the return value. Alternatively, you can use a type with some 'Any' components, such as 'Tuple[int, Any]'. Accompany these with a comment explaining your reasoning.
The Python documentation sometimes fails to properly explain the type of some function arguments or return values. In this case you can always leave these types as 'Any', and add a TODO comment describing why. Somebody can later figure out the precise type. If you are more persistent, you can try doing some of the following:
- Try calling the function with various argument types and see what works. This has the potential problem that sometimes a function accepts certain values by accident, and it might not be portable across Python implementations.
- Browse the module source code (C or Python) or unit tests within the CPython source package and see what's going on. This is obviously only realistic if you are already somewhat familiar with the CPython source code structure or motivated enough to learn.
Sometimes a function return an instance, and only the structure of the instance is described in the Python documentation -- the class has no documented name in Python.
In this case, you can introduce a new class or interface
with a short and descriptive name, and use this for the return value type. Explain prominently in comments that the class name is not derived from the Python implementation. The
mypy-to-Python translator has to be modified to remove references to these synthetic class names during translation to Python, since they are not valid in Python code (e.g. from m import MyClassName
would be an error in Python code). You don't have to do this; just mention the issue in a commit message and somebody else can deal with it.
(TODO document how this is done)
Alternatively, the class may actually have a name, but perhaps that name is undocumented. You can use Python introspection to figure this out: just use print(type(ret_val))
and see if
the result looks like something you could use (if the name is _xvf_internal_wrapper
or similar you should not use it in the stubs; the name is likely to change between Python versions and
is not intended to be a part of the public interface).
If a function accepts arguments of various types, you generally need an overloaded function stub or a generic function stub.
TODO add examples
Often a functions expects an argument that implements a protocol such as Iterable
.
Mypy builtins (import these from typing) include interfaces that match common Python protocols. Here is a partial list:
-
Iterable[t]
(this is a generic type -- you can replace t with any type, e.g.Iterable[int]
,Iterable[Any]
) Iterator[t]
Sequence[t]
-
Sized
(can be used with len)