Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[flake8-pyi] Add autofix for PYI058 #9355

Merged
merged 6 commits into from
Jan 3, 2024

Conversation

AlexWaygood
Copy link
Member

@AlexWaygood AlexWaygood commented Jan 2, 2024

Summary

This PR adds an autofix for the newly added PYI058 rule (added in #9313). The PR's current implementation is that the fix is only available if the fully qualified name of Generator or AsyncGenerator is being used:

  • -> typing.Generator is converted to -> typing.Iterator;
  • -> collections.abc.AsyncGenerator[str, Any] is converted to -> collections.abc.AsyncIterator[str];
  • but -> Generator is not converted to -> Iterator. (It would require more work to figure out if Iterator was already imported or not. And if it wasn't, where should we import it from? typing, typing_extensions, or collections.abc? It seems much more complicated.)

The fix is marked as always safe for __iter__ or __aiter__ methods in .pyi files, but unsafe for all such methods in .py files that have more than one statement in the method body.

This felt slightly fiddly to accomplish, but I couldn't see any utilities in https://github.com/astral-sh/ruff/tree/main/crates/ruff_linter/src/fix that would have made it simpler to implement. Lmk if I'm missing something, though -- my first time implementing an autofix! :)

Test Plan

cargo test / cargo insta review.

Copy link
Contributor

github-actions bot commented Jan 2, 2024

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Typing,
TypingExtensions,
CollectionsAbc,
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AlexWaygood - I did some light refactoring here to use enums for all the different values rather than strings. You might be interested in taking a look. I think it helps enforce some of the invariants that are expected throughout the code and make it clear which values are valid in various places.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this is nice, thanks! Fewer magic strings everywhere

from collections.abc import AsyncGenerator, Generator
from typing import Any
def scope():
from collections.abc import Generator
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, I had to scope each test here into its own function so that they can reason about the imports independently.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, that makes sense, was wondering if that would be an issue

checker.generator().expr(yield_type_info.expr),
yield_type_info.range(),
)
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AlexWaygood - I tried to generalize this so that it doesn't need different logic for attribute vs. name.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I didn't realise how powerful get_or_import_symbol was

29 30 |
30 31 | class IteratorReturningSimpleGenerator5:
31 |- def __iter__(self, /) -> collections.abc.Generator[str, None, typing.Any]: ... # PYI058 (use `Iterator`)
32 |+ def __iter__(self, /) -> Iterator[str]: ... # PYI058 (use `Iterator`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bug in get_or_import_symbol (it doesn't know that it can reuse collections.abc because it's a two-part module). Need to fix.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll fix this separately, since the current fix isn't "wrong", it's just suboptimal.


#[derive(Debug)]
struct YieldTypeInfo<'a> {
expr: &'a ast::Expr,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AlexWaygood - A bit advanced but I changed this to use a lifetime so that it doesn't need to clone the expression. We know the expression will "live" long enough, so it's okay for us to use a reference.

Copy link
Member Author

@AlexWaygood AlexWaygood Jan 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks! I've read about things being generic over lifetimes but have yet to successfully use the feature :)

"__iter__" => "Iterator",
"__aiter__" => "AsyncIterator",
_ => return,
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AlexWaygood - I removed this since it's enforced in let (method, module, member) = { below.

better_return_type: String,
method_name: String,
return_type: Iterator,
method: Method,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AlexWaygood - I changed this to use the enums, which implement std::fmt::Display, so we can still inline them in the format! call and automatically get the expected formatting.

Copy link
Member

@charliermarsh charliermarsh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent work!

@charliermarsh charliermarsh merged commit 1ffc738 into astral-sh:main Jan 3, 2024
17 checks passed
@charliermarsh charliermarsh added fixes Related to suggested fixes for violations rule Implementing or modifying a lint rule labels Jan 3, 2024
@AlexWaygood AlexWaygood deleted the pyi058-autofix branch January 3, 2024 16:18
Comment on lines +289 to +293
impl Ranged for YieldTypeInfo<'_> {
fn range(&self) -> TextRange {
self.range
}
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my education: what's the advantage of implementing the trait here rather than just reading the attribute?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's really minor, more of a consistency thing... If you implement Ranged, then you get methods like .start() and .end() instead of having to do .range.start().

@AlexWaygood
Copy link
Member Author

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fixes Related to suggested fixes for violations rule Implementing or modifying a lint rule
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants