Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use only pytest #51

Merged
merged 15 commits into from
Oct 5, 2022
Merged

Use only pytest #51

merged 15 commits into from
Oct 5, 2022

Conversation

15r10nk
Copy link
Collaborator

@15r10nk 15r10nk commented Sep 20, 2022

I was able to execute all tests with pytest.
However assertion rewriting is still disabled for test_main.py.

But this has already several benefits:

  • tox -e py310 -- --sw works
  • no need to maintain two test environments

I also converted the test_sample_files to an parametrized test which makes it more useful for pytest --sw

I also don't know how I should handle the timeouts.
I removed them for test_sample_files because they are not really useful for single file checks.
I kept them for test_module_files because this takes already a bit longer.
What's your opinion there?

I wanted to get your feedback before I continue.

Copy link
Owner

@alexmojaki alexmojaki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks very cool!

tests/test_main.py Show resolved Hide resolved
tests/test_main.py Outdated Show resolved Hide resolved
@alexmojaki
Copy link
Owner

I also don't know how I should handle the timeouts.

I think they're not necessary any more since we don't use travis.

@15r10nk 15r10nk force-pushed the use-pytest branch 2 times, most recently from fed5298 to e1f2838 Compare September 21, 2022 19:23
tests/test_pytest.py Outdated Show resolved Hide resolved
executing/executing.py Outdated Show resolved Hide resolved
@15r10nk
Copy link
Collaborator Author

15r10nk commented Sep 22, 2022

I rebased it and checked for LOAD_CONST (... see unit tests)

should I exclude _pytest stuff from the test_module_files? The tests failed once for some pytest source file.

Comment on lines 402 to 403
from executing.executing import is_rewritten_by_pytest,get_instructions
import dis
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please put these at the top

@alexmojaki
Copy link
Owner

should I exclude _pytest stuff from the test_module_files? The tests failed once for some pytest source file.

That sounds like a bug to investigate.

Besides, I see a new failure after that.

Also, the previous build failed to report the incorrect node: AttributeError: 'Subscript' object has no attribute 'end_lineno'

@15r10nk
Copy link
Collaborator Author

15r10nk commented Sep 25, 2022

I minimized the failure down to the following. (currently working on the issue minimization)

class Block(ABC):
    def __init__(self, container: Optional["Container"] = None):
        self._lines: List[str] = []

The annotated assignment seems to be the problem.
Can you look into it, because it is part of the 3.11 code.

@alexmojaki
Copy link
Owner

I think all you need to do is check is_annotation starting from 3.6 (when the syntax was introduced) rather than 3.8.

@15r10nk
Copy link
Collaborator Author

15r10nk commented Sep 27, 2022

I looked at the failure and the problem is that the pre 3.11 test code does not analyse deadcode.

example:

class A:
    def __cmp__(self, other):
        if type(other) is self.__class__:
            if self is other:
                return 0
            return -1
        return NotImplemented
        raise TypeError("unorderable types: %s() and %s()" % (self.__class__.__name__, other.__class__.__name__))

There is no bytecode for the TypeError ast-node.

I added the test also to the samples for now but I think I will remove it later again.

Should I continue here and fix this problems?
I think it might be better to skip all files which create problems and fix them in another issue.
Maybe even skip the module_tests for python < 3.11, because there could be much more of this kind of errors and the module_tests are not so deterministic.

@alexmojaki
Copy link
Owner

How hard do you think it'd be to get deadcode analysis working for older versions?

@15r10nk
Copy link
Collaborator Author

15r10nk commented Sep 27, 2022

I don't know. The problem is that different python implementations could decide differently what is deadcode.
However, I think it and is necessary for #42, because I would like to test it with a similar file count like #31.

@alexmojaki
Copy link
Owner

I don't know. The problem is that different python implementations could decide differently what is deadcode.

Assuming 3.11 is the most optimised, I think it'd be a good compromise to always ignore nodes marked as deadcode when they don't match bytecode, but only complain about bytecode matching deadcode in 3.11. That might make things easier.

@15r10nk
Copy link
Collaborator Author

15r10nk commented Sep 27, 2022

Therefore I would have to make the deadcode analysis work down to python 2.7 ...

I would like to improve the tests before I start to change there anything again (see #55).

I try to ignore the enum module for now, if this is also ok.

@@ -708,6 +711,9 @@ def test_module_files(self):
if (
# The sentinel actually appearing in code messes things up
'executing' in filename
# ignore pytest files because they break the SentinelNodeFinder
or '_pytest' in filename
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this file stopped being a problem once annotations were considered in 3.6+

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no problem any more.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So these lines should be removed, right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry you are right. I forgot that.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, it just hit a genuine failure because it doesn't know how to distinguish between the comprehension codes in {t[0] for t in lines2} - {t[0] for t in lines1}. This is covered in the normal tests. So pytester.py specifically should be excluded.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sigh, another pypy optimisation. can you skip if PYPY and '__debug__' in source.text'?

if it fails again you can go back to skipping all pytest files

tests/test_main.py Outdated Show resolved Hide resolved
@15r10nk
Copy link
Collaborator Author

15r10nk commented Oct 4, 2022

this expression gets evaluated at compile time ... __debug__ is known and the literals get evaluated ...

PYC_EXT = ".py" + (__debug__ and "c" or "o")

@alexmojaki
Copy link
Owner

Thanks!

@alexmojaki alexmojaki merged commit 38a81d4 into alexmojaki:master Oct 5, 2022
@15r10nk 15r10nk deleted the use-pytest branch October 5, 2022 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants