(🎁) multiple platform/version compatible mode

[KotlinIsland]

Feature

I want mypy to show errors for all platforms and versions, not just the current/selected one

import sys
if sys.version_info < (3, 11):
    print(21 + "sus")  # very funny, but is a type error 😢
if sys.platorm == "win32":
    print(21 + "sus")  # very funny, but is a type error 😢

actual:

> mypy test.py
no errors 😳

Instead I have to run mypy many times to get a comprehensive output:

> mypy --python-version 3.11 --platform lignux test.py
no errors 😳

> mypy --python-version 3.11 --platform darwin test.py
no errors 😳

> mypy --python-version 3.11 --platform win32 test.py
test.py:5: error: very funny, but that's a type error  [😳]
Found 1 error in 1 file (checked 1 source file)

> mypy --python-version 3.10 --platform lignux test.py
test.py:3: error: very funny, but that's a type error  [😳]
Found 1 error in 1 file (checked 1 source file)

> mypy --python-version 3.10 --platform win32 test.py
test.py:3: error: very funny, but that's a type error  [😳]
test.py:5: error: very funny, but that's a type error  [😳]
Found 2 errors in 1 file (checked 1 source file)

... etc, etc, etc ...

+ additional steps to curate the output into a readable format. (+ ratio?)

I would imagine that most libraries would want type checking applied for their supported python versions and platforms, and most python applications would probably want multiple platforms supported.

So almost everyone using mypy would want some level of cross support, and the current workaround includes manually building a solution involving running mypy concurrently and collecting / filtering the results. I would much rather this very common use-case be supported natively.

Perhaps some way of specifying a list of desired selections, and mypy could pass over the code with a matrix of those options. Perhaps mypy could run multiple processes to improve performance, or alternatively incorporate some complicated logic where it only does one run over the code but analyses the different paths/types correctly. The latter is the preferred solution in my mind, as it would be a lot more efficient and allow some additional benefits (#8823/#5940 comes to mind)

I would also expect mypy to show a note when an error is only detected under a diverging path, to make resolution easier.

pyproject.toml:

[tool.mypy]
platforms = ["lignux", "darwin", "win32"]
python_versions = [3.11, 3.10, 3.9, 3.8, 3.7]

> mypy test.py
test.py:3: error: very funny, but that's a type error  [😳]
test.py:3: note: error found with python-version < 3.11
test.py:5: error: very funny, but that's a type error  [😳]
test.py:5: note: error found with platform "win32"
Found 2 errors in 1 file (checked 1 source file)

[A5rocks]

+1

To add a concrete example, I'm writing a library and I would really appreciate being able to test all these versions without (presumably -- I assume most work is duplicated but the caches are per python version, I believe?) taking 12x the time (windows, macos, linux * 3.8, 3.9, 3.10, 3.11).

[jhance]

You can run them in parallel and it won't take 12x the time. If anything, doing it in one process will be slower because its python and the GIL exists.

(By the way, this is how we did python2+3 at dropbox without taking 2x the time. Even daemon mode you can just have two daemons).

[KotlinIsland]

@jhance
That's a functional workaround, but still involves a lot of reinventing the wheel to collect and filter the results. I imagine this use-case would be desired by almost all libraries and applications, so native support would be highly appreciated.

[JukkaL]

One option could be to have an official/documented script in the mypy repo (and possibly installed with mypy) that does this -- run mypy in parallel (separate processes) using different options, making sure caches don't conflict, merge duplicate messages, etc. This would be fairly simple to implement and wouldn't complicate the maintenance of mypy itself. Anybody want to prototype this? The initial version could be a bit limited, and once it's in the repo, perhaps other contributors would help polish it!

[KotlinIsland]

@JukkaL That sounds awesome! What is your opinion on upgrading the current analysis to incorporate the diverging paths of version/platform checks? As mentioned in the OP there would be additional benefits from it.

[JelleZijlstra]

What is your opinion on upgrading the current analysis to incorporate the diverging paths of version/platform checks?

I'm not Jukka, but that sounds like a big project. It means code throughout the whole type checker needs to have a concept of types that are different depending on the Python version.

[NeilGirdhar]

I'm not Jukka, but that sounds like a big project. It means code throughout the whole type checker needs to have a concept of types that are different depending on the Python version.

Also, different versions have different imports, different members in typing, etc. That sounds way harder than the parallel solution.

[henryiii]

Another feature that this might be able to provide is the ability to use warn_unreachable & used ignores together in a multiplatform codebase. Simply placing a sys.platform or check breaks warn_unreachable - you can't type ignore it in a way that doesn't trigger unused ignores on some platforms. But the "combined" parallel run could possibly handle those properly. (Same thing happens with multiple Python versions, but I'm more used to locking the version there - this would allow several versions, also fixing that).

[Nothing4You]

duplicates/extends #10747

[TiansuYu]

If not supported natively, consider one option to combine with tox to run against multiple python versions.