Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[partially defined] implement support for try statements #14114

Merged
merged 17 commits into from
Dec 16, 2022

Conversation

ilinum
Copy link
Collaborator

@ilinum ilinum commented Nov 16, 2022

This adds support for try/except/finally/else check.

The implementation ended up pretty complicated because it had to handle jumps different for finally. It took me a few iterations to get to this solution and that's the cleanest one I could come up with.

Let me know if you have suggestions on how to structure the code differently.

Closes #13928.

@github-actions

This comment has been minimized.

Copy link
Collaborator

@JukkaL JukkaL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, nested try statements seem to be potentially quite inefficient due to exponential behavior? I'll try to come up with a way to avoid analyzing the same nodes multiple times.

# Conflicts:
#	mypy/partially_defined.py
@ilinum
Copy link
Collaborator Author

ilinum commented Nov 23, 2022

Hmm, nested try statements seem to be potentially quite inefficient due to exponential behavior? I'll try to come up with a way to avoid analyzing the same nodes multiple times.

So right now we process the body for every except type. We could probably cache the result of processing the body and just reuse that. What do you think about that?

Note that we will still need to process it twice (once with disabling jumps and once without). That one is potentially more difficult to fix.

@github-actions

This comment has been minimized.

self.tracker.next_branch()
self.tracker.end_branch_statement()
o.handlers[i].accept(self)
for v in o.vars:
Copy link
Collaborator Author

@ilinum ilinum Nov 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This handling is not correct. I think they should be defined conditionally for the handler.

# Conflicts:
#	mypy/partially_defined.py
@github-actions

This comment has been minimized.

@JukkaL
Copy link
Collaborator

JukkaL commented Nov 28, 2022

Here's a quick idea about how we might be able to avoid exponential behavior. I haven't fully thought this out -- there maybe some obvious thing why this doesn't work. Anyway, I wonder if we could analyze a try/except like these statements (and just once):

  if ...:
      try_body
  if ...:
      except 1
  elif ...:
      except 2
  elif ...:
      except 3
  else:
      else_block
  finally_block

The target try statement would look like this:

try:
    try_body
<except blocks 1..3>
else:
    else_block
finally:
    finally_block

@ilinum
Copy link
Collaborator Author

ilinum commented Nov 29, 2022

I implemented your proposal (which was pretty easy). Here's an example where this doesn't work:

def f1(div: int) -> int:
    try:
        x = 100 // div
    except ArithmeticError:
        x = 0
    return x  # False positive: "x" is always defined but we get an error here.

This is happening because the body of the try is conditional, independently of the except clause.

Now, I also implemented the following:

if ...:
   try_body
   else_block
elif ...:  # Note the change from if to elif here.
   except 1
elif ...:
   except 2
finally_block

However, in this case we don't detect undefined variables inside finally itself.

def f0(div: int) -> int:
    try:
        x = 100 // div
    except ArithmeticError:
        return 0
    finally:
        # "x" is not defined in the except ArithmeticError branch, which returns. 
        # Because it returns, the PartiallyDefined check considers this branch "skipped"
        # and doesn't think it's undefined.
        y = x  # False negative: this should be an error but mypy reports nothing. 
    return y

So to implement it 100% correct, we need to disable branch skipping. I have updated this PR with a version of the check that goes through every try/except/finally 2 times (the previous implementation was doing some unnecessary work). I believe this is the best we can do, unless we're willing to trade off correctness.

I've also found an unrelated bug in nested branch statements that get skipped. It's fixed and I added tests.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

JukkaL pushed a commit that referenced this pull request Nov 30, 2022
…kipped branch (#14221)

The goal of partially-defined check is to detect variables that could be
undefined but semantic analyzer doesn't detect them as undefined. In
this case, a variable was defined in a branch that returns, so semantic
analyzer considered it defined when it was not.

I've discovered this when testing support for try/except statements
(#14114).
@github-actions

This comment has been minimized.

Copy link
Collaborator

@JukkaL JukkaL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Apologies for the slow review.

[builtins fixtures/exception.pyi]

[case testTryElse]
# flags: --enable-error-code partially-defined
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error codes need to be updated.

@github-actions
Copy link
Contributor

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

@ilinum ilinum merged commit 96ac8b3 into python:master Dec 16, 2022
@ilinum ilinum deleted the partial/try branch December 16, 2022 23:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support try/except in partially defined check
2 participants