Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

handle frame in stack trace that doesn't have a module #1741

Merged
merged 19 commits into from
Aug 6, 2021

Conversation

Jasha10
Copy link
Collaborator

@Jasha10 Jasha10 commented Jul 30, 2021

Closes #1739

Not sure how to test this.
Currently there are not any unit tests of the run_and_report function.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 30, 2021
@omry
Copy link
Collaborator

omry commented Jul 30, 2021

Yeah, it's a hard one to test.

I am actually not sure that the best handling is to break.
Maybe we should revert to printing the full stacktrace in such a case?

@omry
Copy link
Collaborator

omry commented Jul 30, 2021

you can test by triggering the code path to see what happens.

@omry
Copy link
Collaborator

omry commented Jul 30, 2021

I would say that in any case there is any exception at all while trying print a nice exception in that branch we should revert to printing the full exception.

@Jasha10 Jasha10 marked this pull request as draft July 31, 2021 14:47
Jasha10 added 9 commits July 31, 2021 10:05
Previously `typing.Any` was used for the type of some Optional
tracebacks in `hydra._internal.run_and_report`. This commit makes the
typing more precise, using `types.TracebackType` and
`Optional[types.TracebackType]` where appropriate.

Additionally, a new type assertion is introduced based on feedback from
mypy.
@Jasha10 Jasha10 marked this pull request as ready for review July 31, 2021 17:24
@Jasha10
Copy link
Collaborator Author

Jasha10 commented Jul 31, 2021

I wrapped the nice-exception logic in a try/except block.

hydra/_internal/utils.py Outdated Show resolved Hide resolved
@omry
Copy link
Collaborator

omry commented Aug 2, 2021

Can you check the CI failures?

@Jasha10 Jasha10 marked this pull request as draft August 2, 2021 21:22
@Jasha10
Copy link
Collaborator Author

Jasha10 commented Aug 2, 2021

Can you check the CI failures?

Ok, I've fixed most of the failures.
There are still a few ray_launcher failures, but I think they're unrelated.

@Jasha10 Jasha10 marked this pull request as ready for review August 2, 2021 23:17
Comment on lines 70 to 72
class TestRunAndReport:
class DemoFunctions:
"""Demo inputs for `run_and_report`"""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain the testing logic?
The test is surprisingly large.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test is surprisingly large.

Sorry about that.
This is not just testing the new functionality. It is also testing the pre-existing run_and_report behavior.

The run_and_report function has the following signature:

def run_and_report(func: Any) -> Any: ...

There are three test methods:

  • test_success tests when run_and_report runs func successfully.
  • test_failure tests when func raises an exception and run_and_report is able to print a nicely-formatted error message.
  • test_simplified_traceback_failure tests when func raises an exception and run_and_report is not able to print a nicely-formatted error message -- this is the case where the original exception gets re-raised.

There are four @staticmethod definitions. These are passed in to run_and_report as the func argument. They are designed to trigger different logic code paths inside of run_and_report:

  • success_func raises no exception
  • simple_error raises an AssertionError
  • run_job_wrapper triggers the special logic in run_and_report that looks for a function called "run_job" in the stack, and strips away the leading stack frames
  • omegaconf_job_wrapper triggers the special logic in run_and_report that looks for the omegaconf module in the stack, and strips away the bottom stack frames

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha.
Thanks for adding this. Can you add a comment describing this in the test class?
(does not need to be as detailed).

Copy link
Collaborator

@omry omry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good.
See comment.

Comment on lines 70 to 72
class TestRunAndReport:
class DemoFunctions:
"""Demo inputs for `run_and_report`"""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha.
Thanks for adding this. Can you add a comment describing this in the test class?
(does not need to be as detailed).

@omry
Copy link
Collaborator

omry commented Aug 5, 2021

Looking good, don't forget a bugfix news fragment.

@Jasha10 Jasha10 merged commit 9445eac into facebookresearch:master Aug 6, 2021
@Jasha10 Jasha10 deleted the closes1739 branch August 6, 2021 01:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] failure while sanitiainzg exception stack trace in some cases
3 participants