Implement flake8-future-annotations FA100

[TylerYep]

This PR tackles the first part of #3072, which is to raise a lint error when a rewrite-able type is used without from __future__ import annotations present.

Test cases taken from: https://github.com/TylerYep/flake8-future-annotations

Some of the test cases I added (prefixed by no_future) do not raise lint warnings now (as expected), but will raise lint warnings once FA102 is implemented. I am adding them to ensure they do not raise extraneous errors to begin with.

This lint rule also found some available fixes in scripts/check_ecosystem.py, which tells me there's some value to having this around :)

The next two PRs will add the autofix functionality for FA100 and FA101 and/or FA102. Thanks for the review!

[github-actions]

PR Check Results

Ecosystem

✅ ecosystem check detected no changes.

Benchmark

Linux

group                                      main                                   pr
-----                                      ----                                   --
linter/all-rules/large/dataset.py          1.01     16.2±0.29ms     2.5 MB/sec    1.00     16.1±0.24ms     2.5 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.00      4.0±0.07ms     4.2 MB/sec    1.00      4.0±0.06ms     4.2 MB/sec
linter/all-rules/numpy/globals.py          1.00    488.4±9.48µs     6.0 MB/sec    1.00    487.9±9.25µs     6.0 MB/sec
linter/all-rules/pydantic/types.py         1.01      6.8±0.09ms     3.8 MB/sec    1.00      6.7±0.13ms     3.8 MB/sec
linter/default-rules/large/dataset.py      1.00      7.8±0.12ms     5.2 MB/sec    1.01      7.9±0.08ms     5.1 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.01  1723.3±13.22µs     9.7 MB/sec    1.00  1708.6±19.59µs     9.7 MB/sec
linter/default-rules/numpy/globals.py      1.00    187.7±3.36µs    15.7 MB/sec    1.00    187.9±3.59µs    15.7 MB/sec
linter/default-rules/pydantic/types.py     1.00      3.6±0.05ms     7.1 MB/sec    1.00      3.6±0.05ms     7.1 MB/sec
parser/large/dataset.py                    1.00      6.3±0.11ms     6.5 MB/sec    1.01      6.3±0.09ms     6.4 MB/sec
parser/numpy/ctypeslib.py                  1.00  1225.3±17.85µs    13.6 MB/sec    1.00  1225.1±24.36µs    13.6 MB/sec
parser/numpy/globals.py                    1.00    124.7±2.50µs    23.7 MB/sec    1.00    124.2±2.27µs    23.8 MB/sec
parser/pydantic/types.py                   1.00      2.7±0.05ms     9.5 MB/sec    1.01      2.7±0.05ms     9.4 MB/sec

Windows

group                                      main                                   pr
-----                                      ----                                   --
linter/all-rules/large/dataset.py          1.02     21.0±0.63ms  1985.7 KB/sec    1.00     20.6±0.66ms  2025.1 KB/sec
linter/all-rules/numpy/ctypeslib.py        1.00      5.3±0.20ms     3.1 MB/sec    1.03      5.5±0.25ms     3.0 MB/sec
linter/all-rules/numpy/globals.py          1.00   630.1±26.83µs     4.7 MB/sec    1.02  644.2±108.66µs     4.6 MB/sec
linter/all-rules/pydantic/types.py         1.00      8.8±0.30ms     2.9 MB/sec    1.03      9.0±0.35ms     2.8 MB/sec
linter/default-rules/large/dataset.py      1.01     10.5±0.38ms     3.9 MB/sec    1.00     10.3±0.36ms     3.9 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.00      2.2±0.07ms     7.7 MB/sec    1.01      2.2±0.10ms     7.6 MB/sec
linter/default-rules/numpy/globals.py      1.00   256.8±13.91µs    11.5 MB/sec    1.08   278.2±19.02µs    10.6 MB/sec
linter/default-rules/pydantic/types.py     1.00      4.6±0.13ms     5.6 MB/sec    1.07      4.9±0.26ms     5.2 MB/sec
parser/large/dataset.py                    1.00      8.5±0.16ms     4.8 MB/sec    1.04      8.8±0.24ms     4.6 MB/sec
parser/numpy/ctypeslib.py                  1.00  1659.1±49.63µs    10.0 MB/sec    1.02  1698.0±70.38µs     9.8 MB/sec
parser/numpy/globals.py                    1.00    169.7±9.08µs    17.4 MB/sec    1.02    172.8±8.51µs    17.1 MB/sec
parser/pydantic/types.py                   1.00      3.7±0.17ms     6.8 MB/sec    1.01      3.8±0.12ms     6.7 MB/sec

[charliermarsh]

(Really sorry for the delay here, I will try to get to this soon, I just got distracted between launch and PyCon and some travel.)

[TylerYep]

I'm not sure why tests are failing - running cargo test locally passes cleanly for me.

Any idea @charliermarsh ?

[charliermarsh]

@TylerYep - Just needed to merge, we recently hit a limit on a fixed-size vector that stores all the rules and it had to be increased.

[TylerYep]

I went back to my original plugin and remembered why DefaultDict and Deque were originally included in FA100:

The main goal of this plugin is to signal to users when a type annotation could be rewritten in a more concise way when from __future__ import annotations is added.

1. This code could be simplified (and is currently autofixed by Pyupgrade):

from typing import List, Optional, Union

a: List[int] = []
b: Optional[Union[int, str]] = None

from __future__ import annotations

a: list[int] = []
b: int | str | None = None

2. This code could be simplified (but is not currently autofixed by Pyupgrade):

from collections import defaultdict, deque
from typing import DefaultDict, Deque

a: DefaultDict[int] = defaultdict(int)
b: Deque[int] = deque()

from __future__ import annotations

from collections import defaultdict, deque

a: defaultdict[int] = defaultdict(int)
b: deque[int] = deque()

asottile/pyupgrade#799 provides context on why collection types are not rewritten using Pyupgrade (import aliasing is hard). It might be possible to extend the autofix functionality to rewrite all of them correctly, but that is out of scope for now.

Afaik, DefaultDict and Deque are the only two code changes included in PEP585, improved using from __future__ import annotations to use the common lowercase name, but not autofixed by Pyupgrade.

3. This code cannot be simplified, but can use a different import. These cases are not intended to be raised by this flake8 rule because the type annotation doesn't need to be changed by adding from __future__ import annotations:

from typing import AsyncIterator, Mapping

a: Mapping[int, str] = {}
b: AsyncIterator = ...

from collections.abc import AsyncIterator, Mapping

a: Mapping[int, str] = {}
b: AsyncIterator = ...

My view is that both 1 and 2 should be raised by this plugin, but not 3, because the code itself is not improved by adding from __future__ import annotations.

I reverted 4373751 to reflect the above explanation.

[charliermarsh]

Sorry, last question: what's the benefit in flagging both the usages and the imports? Would flagging the import alone be insufficient?

[TylerYep]

If the user uses import typing, we can only flag typing.List at the usage. If they use from typing import List, then flagging the import alone is sufficient.

[charliermarsh]

Will merge this after cutting the release.

[charliermarsh]

These should only be activated for minimum-version < Python 3.9, right? Since PEP 585 was part of Python 3.9 (so future annotations aren't required to use the standard-library variants in 3.9 and later).

[TylerYep]

In practice, this works on Python 3.7+.

[charliermarsh]

But if your minimum-supported Python version is Python 3.9, you don't need __future__ annotations to use the standard-library generics. So these errors would already be detected and fixed by the existing pyupgrade rules, right?

[TylerYep]

Ah, you're asking about the upper version.

Unions like str | None didn't start working until Python 3.10, which are also covered by this plugin.

After 3.10, the future annotations import is not as useful anymore.

[calumy]

Will merge this after cutting the release.

I thought it might be worth flagging that the changes to the ecosystem check seem to be causing it to error. I don't have time to investigate now but thought it would be worth saying something before this is merged.

[TylerYep]

What is the error? I see check_ecosystem imports Self, which must mean it's running on Python 3.11+, so none of my changes should impact it.

[calumy]

What is the error? I see check_ecosystem imports Self, which must mean it's running on Python 3.11+, so none of my changes should impact it.

Error in the ecosystem check above:

'async_generator' object does not support the asynchronous context manager protocol

[TylerYep]

Was that caused by removing the asynccontextmanager import? It was an unused import that pre-commit told me to remove.

If that import is required due to side effects, then we probably need to add a # noqa to that line.

Update: fixed the issue.

[charliermarsh]

I made a few changes here that I think are improvements based on some of the capabilities we have in Ruff, but if you disagree, just let me know and we can of course discuss further -- want to merge to keep momentum.

1. I removed the diagnostics that trigger on-import, and instead, I'm more closely following our pyupgrade rules. We flag all usages, regardless of whether the user does import typing or from typing import List -- when the user writes typing.List or List, we resolve it to typing.List. So it's always sufficient to just flag usages, given that we have enough binding tracking and semantic analysis to resolve those usages, regardless of how they're imported.
2. These rules now only flag if your minimum version is below Python 3.9 (or Python 3.10, for Optional and Union), since adding __future__ on those versions has no effect with regards to simplifying the imports. (You can already use the standard library generics on those versions.)
3. We weren't flagging Optional or Union, so I added that too.

[charliermarsh]

This condition is a lot to read, I'd like to clean this up... But the idea is: mirror the logic below for Rule::NonPEP585Annotation, but flip the !self.ctx.future_annotations(). So we're asking: would we trigger here if __future__ annotations were enabled?