Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

epylint runner confounding results of test_relative_beyond... test methods #7252

Closed
jacobtylerwalls opened this issue Aug 1, 2022 · 1 comment · Fixed by #7279
Closed
Labels
Bug 🪲 Emacs 💾 Needs PR This issue is accepted, sufficiently specified and now needs an implementation tests

Comments

@jacobtylerwalls
Copy link
Member

The tests in TestImportChecker added in #5059 use the epylint pylint runner, not the regular one. epylint patches cwd, so it's not an optimal runner for these tests that are import- and namespace-sensitive.

Indeed, I'm proposing to xfail one test in #7114, but the tests should probably just be removed.

Diff indicating that they don't pass using the regular runner:
diff --git a/tests/checkers/unittest_imports.py b/tests/checkers/unittest_imports.py
index ac36f784f..11f6f19ab 100644
--- a/tests/checkers/unittest_imports.py
+++ b/tests/checkers/unittest_imports.py
@@ -8,10 +8,10 @@ import os
 
 import astroid
 
-from pylint import epylint as lint
 from pylint.checkers import imports
 from pylint.interfaces import UNDEFINED
-from pylint.testutils import CheckerTestCase, MessageTest
+from pylint.lint.run import Run
+from pylint.testutils import CheckerTestCase, MessageTest, GenericTestReporter
 
 REGR_DATA = os.path.join(os.path.dirname(__file__), "..", "regrtest_data", "")
 
@@ -41,38 +41,36 @@ class TestImportsChecker(CheckerTestCase):
 
     @staticmethod
     def test_relative_beyond_top_level_two() -> None:
-        output, errors = lint.py_run(
-            f"{os.path.join(REGR_DATA, 'beyond_top_two')} -d all -e relative-beyond-top-level",
-            return_std=True,
+        run1 = Run(
+            [os.path.join(REGR_DATA, 'beyond_top_two'), "--disable=all -e relative-beyond-top-level"],
+            exit=False,
         )
         top_level_function = os.path.join(
             REGR_DATA, "beyond_top_two/namespace_package/top_level_function.py"
         )
-        output2, errors2 = lint.py_run(
-            f"{top_level_function} -d all -e relative-beyond-top-level",
-            return_std=True,
+        run2 = Run(
+            [top_level_function, "--disable=all -e relative-beyond-top-level"],
+            exit=False,
         )
 
-        assert len(output.readlines()) == len(output2.readlines())
-        assert errors.readlines() == errors2.readlines()
+        assert len(run1.linter.reporter.out.readlines()) == len(run2.linter.reporter.out.readlines())
 
     @staticmethod
     def test_relative_beyond_top_level_three() -> None:
-        output, errors = lint.py_run(
-            f"{os.path.join(REGR_DATA, 'beyond_top_three/a.py')} -d all -e relative-beyond-top-level",
-            return_std=True,
+        run = Run(
+            [os.path.join(REGR_DATA, 'beyond_top_three/a.py'), "--disable=all -e relative-beyond-top-level"],
+            exit=False,
+            reporter=GenericTestReporter()
         )
-        assert len(output.readlines()) == 5
-        assert errors.readlines() == []
+        assert len(run.linter.reporter.out.readlines()) == 5
 
     @staticmethod
     def test_relative_beyond_top_level_four() -> None:
-        output, errors = lint.py_run(
-            f"{os.path.join(REGR_DATA, 'beyond_top_four/module')} -d missing-docstring,unused-import",
-            return_std=True,
+        run = Run(
+            [os.path.join(REGR_DATA, 'beyond_top_four/module'), "--disable=missing-docstring,unused-import"],
+            exit=False,
         )
-        assert len(output.readlines()) == 5
-        assert errors.readlines() == []
+        assert len(run.linter.reporter.out.readlines()) == 5
 
     def test_wildcard_import_init(self) -> None:
         module = astroid.MANAGER.ast_from_module_name("init_wildcard", REGR_DATA)
==================================================== short test summary info =====================================================
FAILED tests/checkers/unittest_imports.py::TestImportsChecker::test_relative_beyond_top_level_three - AssertionError: assert 0 ...
FAILED tests/checkers/unittest_imports.py::TestImportsChecker::test_relative_beyond_top_level_four - assert 0 == 5

The first test also fails in reality but unexpectedly passes because the comparison is to another incorrect result:

$ pylint tests/regrtest_data/beyond_top_two
************* Module beyond_top_two
tests/regrtest_data/beyond_top_two/__init__.py:1:0: F0010: error while code parsing: Unable to load file tests/regrtest_data/beyond_top_two/__init__.py:
[Errno 2] No such file or directory: 'tests/regrtest_data/beyond_top_two/__init__.py' (parse-error)

cc/ @DanielNoord

@DanielNoord
Copy link
Collaborator

I'll remove these.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug 🪲 Emacs 💾 Needs PR This issue is accepted, sufficiently specified and now needs an implementation tests
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants