Skip to content

2023 03 07 Session on typing

Jump to bottom
Richard Stromer edited this page Mar 8, 2023 · 1 revision

2023-03-07 Python Typing

https://docs.python.org/3/library/typing.html

First introduced in Python 3.5

PEP 484 and PEP 483

Challenges (w/ NeuralProphet)

  1. Wrong types are worse than no types
  2. Not homogenous ecosystem and environments (there are multiple type checkers, python versions, operating systems, etc)
  3. Precision vs. simplicity (how ‘correct’ do we want to define our types: str vs. Literal["multiplicative", "additive"])
  4. Default parameters and None plus our structure of nested dataclasses
  5. Typing heavily relies on good typing (especially with 3rd party projects)

Package related

  1. numpy (has different types, goes beyond regular python type system)
  2. pandas (needs stubs, wraps a lot of data, hard to know and define internal structure of dataframes)
  3. dataclasses (are not very type friendly, make it difficult to deal with default parameters)

Tipps and recommendations

  1. Start early with typing (easy to do it on the fly, so much extra effort to start with it later)
  2. Be pragmatic about typing
    1. Do not make it overly complex (str is good enough often, specific choices might be nice, but are not essential)
  3. Create custom types to avoid repetition and secure typing across the project

Benefits

  1. Typing helps to prevent or catch errors earlier
  2. Nice ecosystem around type validation (can save you a lot of manual checks)

Open questions for myself

  1. How to best combine docstrings with typings (heavy overlap for parameters)?
    1. Sphinx docs

Questions

  • 1-2 standard examples (e.g. where you use Optional or so)
  • What specific troubles did you run into when adding typing to excisting code? E.g. using None for a str
  • Any hacks how to adapt code most efficiently?
  • What problems did you run into with runing pyright on the refactored code? What were often seen pyright error (gab es typische Strukturen in unserem Code, die sich mühsam refactoren liesen? ... du hattest die internen Code Failures angesproche ... vielleicht hast du ja 1-2 examples.

Reading material

Code sample

a = 10
print(a)

a = "str"
print(a)

# Avoiding circular imports with TYPE_CHECKING
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from sample2 import multiple
from sample import add

from pydantic import validate_arguments, ValidationError
from neuralprophet import NeuralProphet

@validate_arguments
def add(x: DataFrame, y: multiple) -> int:
    # if isinstance(x, int):
    #     raise ValueError("x must be int")
    # if isinstance(y, int):
    #     raise ValueError("x must be int")
    # Could be replaced by https://docs.pydantic.dev/

    return x + y

add(a, 5)

add("str", [1, 2, 3])

add(1, 2)

NeuralProphet(
    yearly_seasonality="auto"
)
Clone this wiki locally