-
-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
implement autofix for 91x #158
Conversation
0bf4add
to
b700100
Compare
Exciting! |
b700100
to
49166b6
Compare
@property | ||
def artificial_errors(self) -> set[cst.Return | cst.FunctionDef | cst.Yield]: | ||
return self.loop_state.artificial_errors | ||
|
||
@artificial_errors.setter | ||
def artificial_errors(self, value: set[cst.Return | cst.FunctionDef | cst.Yield]): | ||
self.loop_state.artificial_errors = value # pragma: no cover |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
randomly got irritated by this one as I had to modify the typing on it so I removed it, it was barely used anyway and you might not even noticed the diff access below.
for statement in self.uncheckpointed_statements: | ||
self.error_91x(node, statement) | ||
self.restore_state(original_node) | ||
return updated_node # noqa: R504 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
R504 irritated me so much, but realized it wasn't just me writing esoteric code and opened an issue afonasev/flake8-return#133
# TODO: generate an error in these two if transforming+visiting is done in a single | ||
# pass and emit-error-on-transform can be enabled/disabled. The error can't be | ||
# generated in the yield/return since it doesn't know if it will be autofixed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
unlikely I'll go with that design, transform+visit in single pass I'm pretty sure would guarantee that errors are output with incorrect line markers so I think you have to first transform, then revisit without transform. But would be substantial runtime reduction if you can do in single pass ofc, so will have to play around a bit with it.
@@ -116,3 +119,22 @@ def visit_Import(self, node: ast.Import): | |||
name = alias.name | |||
if name in ("trio", "anyio") and alias.asname is None: | |||
self.add_library(name) | |||
|
|||
|
|||
@utility_visitor_cst |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pretty much just a copy of the ast version
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remaining todo that [selectively] disabling autofix works, but now testing autofix for both trio and anyio.
@@ -494,10 +589,31 @@ def leave_While_orelse(self, node: cst.While | cst.For): | |||
|
|||
# reset break & continue in case of nested loops | |||
self.outer[node]["uncheckpointed_statements"] = self.uncheckpointed_statements | |||
self.restore_state(node) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
needed to be moved to leave_While
since it has to be done after inserting checkpoints in the loop body.
49166b6
to
6f20d79
Compare
# returns a bool indicating if any real (i.e. not artificial) errors were raised | ||
# so caller can insert checkpoint before statement (if yield/return) or at end | ||
# of body (functiondef) | ||
def check_function_exit( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
moved all artificial statement logic from error_91x to check_function_exit
@@ -394,6 +493,8 @@ def visit_While(self, node: cst.While | cst.For): | |||
self.loop_state = LoopState() | |||
self.infinite_loop = self.body_guaranteed_once = False | |||
|
|||
visit_For = visit_While |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ooooops, pretty nasty line to have forgotten about. Added a test that would've caught it.
This is *slightly* different enough from what the utility visitor does | ||
that it's added here, but the functionality is maybe better placed in there. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it'd probably just need another bit saved in self.library on whether it's been explicitly imported, or just assumed/specified with --anyio
. But I'll procrastinate on that with the on-the-spot-made-up-reasoning that there's ast visitors that use the current self.library functionality and I neither wanna update both nor have different library logic between them.
async def foo_while_endless_3(): | ||
while True: | ||
... | ||
yield # type: ignore[unreachable] | ||
await foo() | ||
|
||
|
||
async def foo_while_endless_4(): | ||
await foo() | ||
while True: | ||
yield | ||
while True: | ||
await foo() | ||
yield |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't remember why I added these 😇
Exciting indeed! Although not sure how you can be excited about reviewing modifications to a visitor that's up to 800 lines 😱 And that's before adding remove-unnecessary-checkpoint-logic!!! |
### autofix files | ||
Checks that have autofixing can have a file in the `tests/autofix_files` directory matching the filename in `tests/eval_files`. The result of running the checker on the eval file with autofix enabled will then be compared to the content of the autofix file and will print a diff (if `-s` is on) and assert that the content is the same. `--generate-autofix` is added as a pytest flag to ease development, which will print a diff (with `-s`) and overwrite the content of the autofix file. Also see the magic line marker `pass # AUTOFIX_LINE ` below |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of spitting out the resulting .py
files, can we save the diff between before and after as a .py.diff
? It's much easier to get a sense of "what it's doing" in this format, and I'm concerned that without that code review is not going to be effective.
(although instead of constructing a three-way diff, I'd just let Pytest's assertion helper show the diff between the actual and expected diffs)
((diff diff diff, it hardly seems to diff mean anything anymore. diff)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the generated files for checking that they're valid and passes linters, and when developing they're easier to read and throw into cst.parse_module
, but I'll generate diff files as well 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, yep, both sounds good!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wohoo, bigger commits! 😁
They're still not perfect, as they're not showing e.g. where checkpoints aren't inserted - but that will be covered by future tests that autofixed files don't generate any errors. (and will have to figure out a solution for errors that are expected not to get autofixed)
flake8_trio/runner.py
Outdated
@@ -112,7 +125,7 @@ def __init__(self, options: Namespace, module: Module): | |||
def run(self) -> Iterable[Error]: | |||
if not self.visitors: | |||
return | |||
for v in self.visitors: | |||
for v in chain(self.utility_visitors, self.visitors): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for v in chain(self.utility_visitors, self.visitors): | |
for v in (*self.utility_visitors, *self.visitors): |
Could also use +
because they're both tuples, but the "anything iterable" syntax is visibly safe.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thought the chain
was nice and explicit, but don't mind either variant. The plus solution might also be slower, no?
6f20d79
to
249eb2b
Compare
fixed comments |
249eb2b
to
cc43270
Compare
except I had untracked files ... updated my aliases so that won't happen again ^^ alias untracked="git ls-files -o --directory --exclude-standard | sed q1 > /dev/null"
alias gfpush='untracked && pytest --quiet -x && git commit -a --amend --no-edit && git push -f' |
Oh and I realized that the |
ahahaha, |
cc43270
to
220c405
Compare
I'm writing the extra tests now, and this implementation still does some dumb stuff (yield inside boolops and stuff), but I'll leave (not) handling those for the next PR. |
lint all the things 😜 |
Ooh, Well, there's no linter - other than flake8-trio itself, that catches unawaited |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Whew! Big diff to review, but this looks great. Thanks again @jakkdl!
I'll try to get the diffs to be smaller, but it's hard to avoid at this stage - and especially when working with the monster that is 91x 😅 |
Oh yeah, that's not a criticism! Just an expression of relief that I made it through 😋 |
Haha, just meant it's worth trying for. If I don't keep it in the back of my mind the scope creep of PR's is very real 😇 |
does still not finish #70 😅 It "merely" inserts checkpoints wherever needed.
TODO:
And added a couple more tasks to #124 before this is reasonably usable by an end user. That includes checks for whether autofix toggling actually is toggled, which I kinda know isn't done properly by this anyway (but atm it mostly doesn't matter cause the new file doesn't get written without the autofix flag). But I'll leave those for another PR.