8.0.0: pytest fails in many modules test suites on .::project unit

[kloczek]

• a detailed description of the bug or problem you are having
• output of pip list from the virtual environment you are using
• pytest and operating system versions
• minimal example if possible

Linux x86/64.
I just started testing all my modules packaged as rpm packages (+1.2k) with new pytates 8.0.0 and found that many test suites fails in the same .::project unit.

Example with installer module

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-installer-0.7.0-7.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-installer-0.7.0-7.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.8.18, pytest-8.0.0, pluggy-1.3.0
rootdir: /home/tkloczko/rpmbuild/BUILD/installer-0.7.0
plugins: checkdocs-2.10.1
collected 121 items

. F                                                                                                                                                                                   [  0%]
tests/test_core.py ........                                                                                                                                                           [  7%]
tests/test_destinations.py .........                                                                                                                                                  [ 14%]
tests/test_main.py .....                                                                                                                                                              [ 19%]
tests/test_records.py ..........................................                                                                                                                      [ 53%]
tests/test_scripts.py ...........                                                                                                                                                     [ 62%]
tests/test_sources.py ..................                                                                                                                                              [ 77%]
tests/test_utils.py ...........................                                                                                                                                       [100%]

========================================================================================= FAILURES ==========================================================================================
_______________________________________________________________________________________ test session ________________________________________________________________________________________

cls = <class '_pytest.runner.CallInfo'>, func = <function call_runtest_hook.<locals>.<lambda> at 0x7fac98c1c700>, when = 'call'
reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)

    @classmethod
    def from_call(
        cls,
        func: Callable[[], TResult],
        when: Literal["collect", "setup", "call", "teardown"],
        reraise: Optional[
            Union[Type[BaseException], Tuple[Type[BaseException], ...]]
        ] = None,
    ) -> "CallInfo[TResult]":
        """Call func, wrapping the result in a CallInfo.

        :param func:
            The function to call. Called without arguments.
        :param when:
            The phase in which the function is called.
        :param reraise:
            Exception or exceptions that shall propagate if raised by the
            function, instead of being wrapped in the CallInfo.
        """
        excinfo = None
        start = timing.time()
        precise_start = timing.perf_counter()
        try:
>           result: Optional[TResult] = func()

/usr/lib/python3.8/site-packages/_pytest/runner.py:345:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>       lambda: ihook(item=item, **kwds), when=when, reraise=reraise
    )

/usr/lib/python3.8/site-packages/_pytest/runner.py:266:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <HookCaller 'pytest_runtest_call'>, kwargs = {'item': <CheckdocsItem project>}, firstresult = False

    def __call__(self, **kwargs: object) -> Any:
        """Call the hook.

        Only accepts keyword arguments, which should match the hook
        specification.

        Returns the result(s) of calling all registered plugins, see
        :ref:`calling`.
        """
        assert (
            not self.is_historic()
        ), "Cannot directly call a historic hook - use call_historic instead."
        self._verify_all_args_are_provided(kwargs)
        firstresult = self.spec.opts.get("firstresult", False) if self.spec else False
>       return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)

/usr/lib/python3.8/site-packages/pluggy/_hooks.py:493:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <_pytest.config.PytestPluginManager object at 0x7fac99f98850>, hook_name = 'pytest_runtest_call'
methods = [<HookImpl plugin_name='runner', plugin=<module '_pytest.runner' from '/usr/lib/python3.8/site-packages/_pytest/runner...eption', plugin=<module '_pytest.threadexception' from '/usr/lib/python3.8/site-packages/_pytest/threadexception.py'>>]
kwargs = {'item': <CheckdocsItem project>}, firstresult = False

    def _hookexec(
        self,
        hook_name: str,
        methods: Sequence[HookImpl],
        kwargs: Mapping[str, object],
        firstresult: bool,
    ) -> object | list[object]:
        # called from all hookcaller instances.
        # enable_tracing will set its own wrapping function at self._inner_hookexec
>       return self._inner_hookexec(hook_name, methods, kwargs, firstresult)

/usr/lib/python3.8/site-packages/pluggy/_manager.py:115:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

hook_name = 'pytest_runtest_call'
hook_impls = [<HookImpl plugin_name='runner', plugin=<module '_pytest.runner' from '/usr/lib/python3.8/site-packages/_pytest/runner...eption', plugin=<module '_pytest.threadexception' from '/usr/lib/python3.8/site-packages/_pytest/threadexception.py'>>]
caller_kwargs = {'item': <CheckdocsItem project>}, firstresult = False

    def _multicall(
        hook_name: str,
        hook_impls: Sequence[HookImpl],
        caller_kwargs: Mapping[str, object],
        firstresult: bool,
    ) -> object | list[object]:
        """Execute a call into multiple python functions/methods and return the
        result(s).

        ``caller_kwargs`` comes from HookCaller.__call__().
        """
        __tracebackhide__ = True
        results: list[object] = []
        exception = None
        only_new_style_wrappers = True
        try:  # run impl and wrapper setup functions in a loop
            teardowns: list[Teardown] = []
            try:
                for hook_impl in reversed(hook_impls):
                    try:
                        args = [caller_kwargs[argname] for argname in hook_impl.argnames]
                    except KeyError:
                        for argname in hook_impl.argnames:
                            if argname not in caller_kwargs:
                                raise HookCallError(
                                    f"hook call must provide argument {argname!r}"
                                )

                    if hook_impl.hookwrapper:
                        only_new_style_wrappers = False
                        try:
                            # If this cast is not valid, a type error is raised below,
                            # which is the desired response.
                            res = hook_impl.function(*args)
                            wrapper_gen = cast(Generator[None, Result[object], None], res)
                            next(wrapper_gen)  # first yield
                            teardowns.append((wrapper_gen,))
                        except StopIteration:
                            _raise_wrapfail(wrapper_gen, "did not yield")
                    elif hook_impl.wrapper:
                        try:
                            # If this cast is not valid, a type error is raised below,
                            # which is the desired response.
                            res = hook_impl.function(*args)
                            function_gen = cast(Generator[None, object, object], res)
                            next(function_gen)  # first yield
                            teardowns.append(function_gen)
                        except StopIteration:
                            _raise_wrapfail(function_gen, "did not yield")
                    else:
                        res = hook_impl.function(*args)
                        if res is not None:
                            results.append(res)
                            if firstresult:  # halt further impl calls
                                break
            except BaseException as exc:
                exception = exc
        finally:
            # Fast path - only new-style wrappers, no Result.
            if only_new_style_wrappers:
                if firstresult:  # first result hooks return a single value
                    result = results[0] if results else None
                else:
                    result = results

                # run all wrapper post-yield blocks
                for teardown in reversed(teardowns):
                    try:
                        if exception is not None:
                            teardown.throw(exception)  # type: ignore[union-attr]
                        else:
                            teardown.send(result)  # type: ignore[union-attr]
                        # Following is unreachable for a well behaved hook wrapper.
                        # Try to force finalizers otherwise postponed till GC action.
                        # Note: close() may raise if generator handles GeneratorExit.
                        teardown.close()  # type: ignore[union-attr]
                    except StopIteration as si:
                        result = si.value
                        exception = None
                        continue
                    except BaseException as e:
                        exception = e
                        continue
                    _raise_wrapfail(teardown, "has second yield")  # type: ignore[arg-type]

                if exception is not None:
>                   raise exception.with_traceback(exception.__traceback__)

/usr/lib/python3.8/site-packages/pluggy/_callers.py:113:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

hook_name = 'pytest_runtest_call'
hook_impls = [<HookImpl plugin_name='runner', plugin=<module '_pytest.runner' from '/usr/lib/python3.8/site-packages/_pytest/runner...eption', plugin=<module '_pytest.threadexception' from '/usr/lib/python3.8/site-packages/_pytest/threadexception.py'>>]
caller_kwargs = {'item': <CheckdocsItem project>}, firstresult = False

    def _multicall(
        hook_name: str,
        hook_impls: Sequence[HookImpl],
        caller_kwargs: Mapping[str, object],
        firstresult: bool,
    ) -> object | list[object]:
        """Execute a call into multiple python functions/methods and return the
        result(s).

        ``caller_kwargs`` comes from HookCaller.__call__().
        """
        __tracebackhide__ = True
        results: list[object] = []
        exception = None
        only_new_style_wrappers = True
        try:  # run impl and wrapper setup functions in a loop
            teardowns: list[Teardown] = []
            try:
                for hook_impl in reversed(hook_impls):
                    try:
                        args = [caller_kwargs[argname] for argname in hook_impl.argnames]
                    except KeyError:
                        for argname in hook_impl.argnames:
                            if argname not in caller_kwargs:
                                raise HookCallError(
                                    f"hook call must provide argument {argname!r}"
                                )

                    if hook_impl.hookwrapper:
                        only_new_style_wrappers = False
                        try:
                            # If this cast is not valid, a type error is raised below,
                            # which is the desired response.
                            res = hook_impl.function(*args)
                            wrapper_gen = cast(Generator[None, Result[object], None], res)
                            next(wrapper_gen)  # first yield
                            teardowns.append((wrapper_gen,))
                        except StopIteration:
                            _raise_wrapfail(wrapper_gen, "did not yield")
                    elif hook_impl.wrapper:
                        try:
                            # If this cast is not valid, a type error is raised below,
                            # which is the desired response.
                            res = hook_impl.function(*args)
                            function_gen = cast(Generator[None, object, object], res)
                            next(function_gen)  # first yield
                            teardowns.append(function_gen)
                        except StopIteration:
                            _raise_wrapfail(function_gen, "did not yield")
                    else:
                        res = hook_impl.function(*args)
                        if res is not None:
                            results.append(res)
                            if firstresult:  # halt further impl calls
                                break
            except BaseException as exc:
                exception = exc
        finally:
            # Fast path - only new-style wrappers, no Result.
            if only_new_style_wrappers:
                if firstresult:  # first result hooks return a single value
                    result = results[0] if results else None
                else:
                    result = results

                # run all wrapper post-yield blocks
                for teardown in reversed(teardowns):
                    try:
                        if exception is not None:
>                           teardown.throw(exception)  # type: ignore[union-attr]

/usr/lib/python3.8/site-packages/pluggy/_callers.py:96:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    @pytest.hookimpl(wrapper=True, tryfirst=True)
    def pytest_runtest_call() -> Generator[None, None, None]:
>       yield from thread_exception_runtest_hook()

/usr/lib/python3.8/site-packages/_pytest/threadexception.py:87:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def thread_exception_runtest_hook() -> Generator[None, None, None]:
        with catch_threading_exception() as cm:
            try:
>               yield

/usr/lib/python3.8/site-packages/_pytest/threadexception.py:63:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

hook_name = 'pytest_runtest_call'
hook_impls = [<HookImpl plugin_name='runner', plugin=<module '_pytest.runner' from '/usr/lib/python3.8/site-packages/_pytest/runner...eption', plugin=<module '_pytest.threadexception' from '/usr/lib/python3.8/site-packages/_pytest/threadexception.py'>>]
caller_kwargs = {'item': <CheckdocsItem project>}, firstresult = False

    def _multicall(
        hook_name: str,
        hook_impls: Sequence[HookImpl],
        caller_kwargs: Mapping[str, object],
        firstresult: bool,
    ) -> object | list[object]:
        """Execute a call into multiple python functions/methods and return the
        result(s).

        ``caller_kwargs`` comes from HookCaller.__call__().
        """
        __tracebackhide__ = True
        results: list[object] = []
        exception = None
        only_new_style_wrappers = True
        try:  # run impl and wrapper setup functions in a loop
            teardowns: list[Teardown] = []
            try:
                for hook_impl in reversed(hook_impls):
                    try:
                        args = [caller_kwargs[argname] for argname in hook_impl.argnames]
                    except KeyError:
                        for argname in hook_impl.argnames:
                            if argname not in caller_kwargs:
                                raise HookCallError(
                                    f"hook call must provide argument {argname!r}"
                                )

                    if hook_impl.hookwrapper:
                        only_new_style_wrappers = False
                        try:
                            # If this cast is not valid, a type error is raised below,
                            # which is the desired response.
                            res = hook_impl.function(*args)
                            wrapper_gen = cast(Generator[None, Result[object], None], res)
                            next(wrapper_gen)  # first yield
                            teardowns.append((wrapper_gen,))
                        except StopIteration:
                            _raise_wrapfail(wrapper_gen, "did not yield")
                    elif hook_impl.wrapper:
                        try:
                            # If this cast is not valid, a type error is raised below,
                            # which is the desired response.
                            res = hook_impl.function(*args)
                            function_gen = cast(Generator[None, object, object], res)
                            next(function_gen)  # first yield
                            teardowns.append(function_gen)
                        except StopIteration:
                            _raise_wrapfail(function_gen, "did not yield")
                    else:
                        res = hook_impl.function(*args)
                        if res is not None:
                            results.append(res)
                            if firstresult:  # halt further impl calls
                                break
            except BaseException as exc:
                exception = exc
        finally:
            # Fast path - only new-style wrappers, no Result.
            if only_new_style_wrappers:
                if firstresult:  # first result hooks return a single value
                    result = results[0] if results else None
                else:
                    result = results

                # run all wrapper post-yield blocks
                for teardown in reversed(teardowns):
                    try:
                        if exception is not None:
>                           teardown.throw(exception)  # type: ignore[union-attr]

/usr/lib/python3.8/site-packages/pluggy/_callers.py:96:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    @pytest.hookimpl(wrapper=True, tryfirst=True)
    def pytest_runtest_call() -> Generator[None, None, None]:
>       yield from unraisable_exception_runtest_hook()

/usr/lib/python3.8/site-packages/_pytest/unraisableexception.py:90:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def unraisable_exception_runtest_hook() -> Generator[None, None, None]:
        with catch_unraisable_exception() as cm:
            try:
>               yield

/usr/lib/python3.8/site-packages/_pytest/unraisableexception.py:65:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

hook_name = 'pytest_runtest_call'
hook_impls = [<HookImpl plugin_name='runner', plugin=<module '_pytest.runner' from '/usr/lib/python3.8/site-packages/_pytest/runner...eption', plugin=<module '_pytest.threadexception' from '/usr/lib/python3.8/site-packages/_pytest/threadexception.py'>>]
caller_kwargs = {'item': <CheckdocsItem project>}, firstresult = False

    def _multicall(
        hook_name: str,
        hook_impls: Sequence[HookImpl],
        caller_kwargs: Mapping[str, object],
        firstresult: bool,
    ) -> object | list[object]:
        """Execute a call into multiple python functions/methods and return the
        result(s).

        ``caller_kwargs`` comes from HookCaller.__call__().
        """
        __tracebackhide__ = True
        results: list[object] = []
        exception = None
        only_new_style_wrappers = True
        try:  # run impl and wrapper setup functions in a loop
            teardowns: list[Teardown] = []
            try:
                for hook_impl in reversed(hook_impls):
                    try:
                        args = [caller_kwargs[argname] for argname in hook_impl.argnames]
                    except KeyError:
                        for argname in hook_impl.argnames:
                            if argname not in caller_kwargs:
                                raise HookCallError(
                                    f"hook call must provide argument {argname!r}"
                                )

                    if hook_impl.hookwrapper:
                        only_new_style_wrappers = False
                        try:
                            # If this cast is not valid, a type error is raised below,
                            # which is the desired response.
                            res = hook_impl.function(*args)
                            wrapper_gen = cast(Generator[None, Result[object], None], res)
                            next(wrapper_gen)  # first yield
                            teardowns.append((wrapper_gen,))
                        except StopIteration:
                            _raise_wrapfail(wrapper_gen, "did not yield")
                    elif hook_impl.wrapper:
                        try:
                            # If this cast is not valid, a type error is raised below,
                            # which is the desired response.
                            res = hook_impl.function(*args)
                            function_gen = cast(Generator[None, object, object], res)
                            next(function_gen)  # first yield
                            teardowns.append(function_gen)
                        except StopIteration:
                            _raise_wrapfail(function_gen, "did not yield")
                    else:
                        res = hook_impl.function(*args)
                        if res is not None:
                            results.append(res)
                            if firstresult:  # halt further impl calls
                                break
            except BaseException as exc:
                exception = exc
        finally:
            # Fast path - only new-style wrappers, no Result.
            if only_new_style_wrappers:
                if firstresult:  # first result hooks return a single value
                    result = results[0] if results else None
                else:
                    result = results

                # run all wrapper post-yield blocks
                for teardown in reversed(teardowns):
                    try:
                        if exception is not None:
>                           teardown.throw(exception)  # type: ignore[union-attr]

/usr/lib/python3.8/site-packages/pluggy/_callers.py:96:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <_pytest.logging.LoggingPlugin object at 0x7fac98deaeb0>, item = <CheckdocsItem project>

    @hookimpl(wrapper=True)
    def pytest_runtest_call(self, item: nodes.Item) -> Generator[None, None, None]:
        self.log_cli_handler.set_when("call")

>       yield from self._runtest_for(item, "call")

/usr/lib/python3.8/site-packages/_pytest/logging.py:839:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <_pytest.logging.LoggingPlugin object at 0x7fac98deaeb0>, item = <CheckdocsItem project>, when = 'call'

    def _runtest_for(self, item: nodes.Item, when: str) -> Generator[None, None, None]:
        """Implement the internals of the pytest_runtest_xxx() hooks."""
        with catching_logs(
            self.caplog_handler,
            level=self.log_level,
        ) as caplog_handler, catching_logs(
            self.report_handler,
            level=self.log_level,
        ) as report_handler:
            caplog_handler.reset()
            report_handler.reset()
            item.stash[caplog_records_key][when] = caplog_handler.records
            item.stash[caplog_handler_key] = caplog_handler

            try:
>               yield

/usr/lib/python3.8/site-packages/_pytest/logging.py:822:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

hook_name = 'pytest_runtest_call'
hook_impls = [<HookImpl plugin_name='runner', plugin=<module '_pytest.runner' from '/usr/lib/python3.8/site-packages/_pytest/runner...eption', plugin=<module '_pytest.threadexception' from '/usr/lib/python3.8/site-packages/_pytest/threadexception.py'>>]
caller_kwargs = {'item': <CheckdocsItem project>}, firstresult = False

    def _multicall(
        hook_name: str,
        hook_impls: Sequence[HookImpl],
        caller_kwargs: Mapping[str, object],
        firstresult: bool,
    ) -> object | list[object]:
        """Execute a call into multiple python functions/methods and return the
        result(s).

        ``caller_kwargs`` comes from HookCaller.__call__().
        """
        __tracebackhide__ = True
        results: list[object] = []
        exception = None
        only_new_style_wrappers = True
        try:  # run impl and wrapper setup functions in a loop
            teardowns: list[Teardown] = []
            try:
                for hook_impl in reversed(hook_impls):
                    try:
                        args = [caller_kwargs[argname] for argname in hook_impl.argnames]
                    except KeyError:
                        for argname in hook_impl.argnames:
                            if argname not in caller_kwargs:
                                raise HookCallError(
                                    f"hook call must provide argument {argname!r}"
                                )

                    if hook_impl.hookwrapper:
                        only_new_style_wrappers = False
                        try:
                            # If this cast is not valid, a type error is raised below,
                            # which is the desired response.
                            res = hook_impl.function(*args)
                            wrapper_gen = cast(Generator[None, Result[object], None], res)
                            next(wrapper_gen)  # first yield
                            teardowns.append((wrapper_gen,))
                        except StopIteration:
                            _raise_wrapfail(wrapper_gen, "did not yield")
                    elif hook_impl.wrapper:
                        try:
                            # If this cast is not valid, a type error is raised below,
                            # which is the desired response.
                            res = hook_impl.function(*args)
                            function_gen = cast(Generator[None, object, object], res)
                            next(function_gen)  # first yield
                            teardowns.append(function_gen)
                        except StopIteration:
                            _raise_wrapfail(function_gen, "did not yield")
                    else:
                        res = hook_impl.function(*args)
                        if res is not None:
                            results.append(res)
                            if firstresult:  # halt further impl calls
                                break
            except BaseException as exc:
                exception = exc
        finally:
            # Fast path - only new-style wrappers, no Result.
            if only_new_style_wrappers:
                if firstresult:  # first result hooks return a single value
                    result = results[0] if results else None
                else:
                    result = results

                # run all wrapper post-yield blocks
                for teardown in reversed(teardowns):
                    try:
                        if exception is not None:
>                           teardown.throw(exception)  # type: ignore[union-attr]

/usr/lib/python3.8/site-packages/pluggy/_callers.py:96:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <CaptureManager _method='fd' _global_capturing=<MultiCapture out=<FDCapture 1 oldfd=5 _state='suspended' tmpfile=<_io....xtIOWrapper name='/dev/null' mode='r' encoding='utf-8'>> _state='suspended' _in_suspended=False> _capture_fixture=None>
item = <CheckdocsItem project>

    @hookimpl(wrapper=True)
    def pytest_runtest_call(self, item: Item) -> Generator[None, None, None]:
        with self.item_capture("call", item):
>           return (yield)

/usr/lib/python3.8/site-packages/_pytest/capture.py:882:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

hook_name = 'pytest_runtest_call'
hook_impls = [<HookImpl plugin_name='runner', plugin=<module '_pytest.runner' from '/usr/lib/python3.8/site-packages/_pytest/runner...eption', plugin=<module '_pytest.threadexception' from '/usr/lib/python3.8/site-packages/_pytest/threadexception.py'>>]
caller_kwargs = {'item': <CheckdocsItem project>}, firstresult = False

    def _multicall(
        hook_name: str,
        hook_impls: Sequence[HookImpl],
        caller_kwargs: Mapping[str, object],
        firstresult: bool,
    ) -> object | list[object]:
        """Execute a call into multiple python functions/methods and return the
        result(s).

        ``caller_kwargs`` comes from HookCaller.__call__().
        """
        __tracebackhide__ = True
        results: list[object] = []
        exception = None
        only_new_style_wrappers = True
        try:  # run impl and wrapper setup functions in a loop
            teardowns: list[Teardown] = []
            try:
                for hook_impl in reversed(hook_impls):
                    try:
                        args = [caller_kwargs[argname] for argname in hook_impl.argnames]
                    except KeyError:
                        for argname in hook_impl.argnames:
                            if argname not in caller_kwargs:
                                raise HookCallError(
                                    f"hook call must provide argument {argname!r}"
                                )

                    if hook_impl.hookwrapper:
                        only_new_style_wrappers = False
                        try:
                            # If this cast is not valid, a type error is raised below,
                            # which is the desired response.
                            res = hook_impl.function(*args)
                            wrapper_gen = cast(Generator[None, Result[object], None], res)
                            next(wrapper_gen)  # first yield
                            teardowns.append((wrapper_gen,))
                        except StopIteration:
                            _raise_wrapfail(wrapper_gen, "did not yield")
                    elif hook_impl.wrapper:
                        try:
                            # If this cast is not valid, a type error is raised below,
                            # which is the desired response.
                            res = hook_impl.function(*args)
                            function_gen = cast(Generator[None, object, object], res)
                            next(function_gen)  # first yield
                            teardowns.append(function_gen)
                        except StopIteration:
                            _raise_wrapfail(function_gen, "did not yield")
                    else:
                        res = hook_impl.function(*args)
                        if res is not None:
                            results.append(res)
                            if firstresult:  # halt further impl calls
                                break
            except BaseException as exc:
                exception = exc
        finally:
            # Fast path - only new-style wrappers, no Result.
            if only_new_style_wrappers:
                if firstresult:  # first result hooks return a single value
                    result = results[0] if results else None
                else:
                    result = results

                # run all wrapper post-yield blocks
                for teardown in reversed(teardowns):
                    try:
                        if exception is not None:
>                           teardown.throw(exception)  # type: ignore[union-attr]

/usr/lib/python3.8/site-packages/pluggy/_callers.py:96:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

item = <CheckdocsItem project>

    @hookimpl(wrapper=True)
    def pytest_runtest_call(item: Item) -> Generator[None, None, None]:
        xfailed = item.stash.get(xfailed_key, None)
        if xfailed is None:
            item.stash[xfailed_key] = xfailed = evaluate_xfail_marks(item)

        if xfailed and not item.config.option.runxfail and not xfailed.run:
            xfail("[NOTRUN] " + xfailed.reason)

        try:
>           return (yield)

/usr/lib/python3.8/site-packages/_pytest/skipping.py:257:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

hook_name = 'pytest_runtest_call'
hook_impls = [<HookImpl plugin_name='runner', plugin=<module '_pytest.runner' from '/usr/lib/python3.8/site-packages/_pytest/runner...eption', plugin=<module '_pytest.threadexception' from '/usr/lib/python3.8/site-packages/_pytest/threadexception.py'>>]
caller_kwargs = {'item': <CheckdocsItem project>}, firstresult = False

    def _multicall(
        hook_name: str,
        hook_impls: Sequence[HookImpl],
        caller_kwargs: Mapping[str, object],
        firstresult: bool,
    ) -> object | list[object]:
        """Execute a call into multiple python functions/methods and return the
        result(s).

        ``caller_kwargs`` comes from HookCaller.__call__().
        """
        __tracebackhide__ = True
        results: list[object] = []
        exception = None
        only_new_style_wrappers = True
        try:  # run impl and wrapper setup functions in a loop
            teardowns: list[Teardown] = []
            try:
                for hook_impl in reversed(hook_impls):
                    try:
                        args = [caller_kwargs[argname] for argname in hook_impl.argnames]
                    except KeyError:
                        for argname in hook_impl.argnames:
                            if argname not in caller_kwargs:
                                raise HookCallError(
                                    f"hook call must provide argument {argname!r}"
                                )

                    if hook_impl.hookwrapper:
                        only_new_style_wrappers = False
                        try:
                            # If this cast is not valid, a type error is raised below,
                            # which is the desired response.
                            res = hook_impl.function(*args)
                            wrapper_gen = cast(Generator[None, Result[object], None], res)
                            next(wrapper_gen)  # first yield
                            teardowns.append((wrapper_gen,))
                        except StopIteration:
                            _raise_wrapfail(wrapper_gen, "did not yield")
                    elif hook_impl.wrapper:
                        try:
                            # If this cast is not valid, a type error is raised below,
                            # which is the desired response.
                            res = hook_impl.function(*args)
                            function_gen = cast(Generator[None, object, object], res)
                            next(function_gen)  # first yield
                            teardowns.append(function_gen)
                        except StopIteration:
                            _raise_wrapfail(function_gen, "did not yield")
                    else:
>                       res = hook_impl.function(*args)

/usr/lib/python3.8/site-packages/pluggy/_callers.py:77:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

item = <CheckdocsItem project>

    def pytest_runtest_call(item: Item) -> None:
        _update_current_test_var(item, "call")
        try:
            del sys.last_type
            del sys.last_value
            del sys.last_traceback
        except AttributeError:
            pass
        try:
            item.runtest()
        except Exception as e:
            # Store trace info to allow postmortem debugging
            sys.last_type = type(e)
            sys.last_value = e
            assert e.__traceback__ is not None
            # Skip *this* frame
            sys.last_traceback = e.__traceback__.tb_next
>           raise e

/usr/lib/python3.8/site-packages/_pytest/runner.py:181:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

item = <CheckdocsItem project>

    def pytest_runtest_call(item: Item) -> None:
        _update_current_test_var(item, "call")
        try:
            del sys.last_type
            del sys.last_value
            del sys.last_traceback
        except AttributeError:
            pass
        try:
>           item.runtest()

/usr/lib/python3.8/site-packages/_pytest/runner.py:173:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <CheckdocsItem project>

    def runtest(self):
>       desc = self.get_long_description()

/usr/lib/python3.8/site-packages/pytest_checkdocs/__init__.py:30:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <CheckdocsItem project>

    def get_long_description(self):
>       return Description.from_md(ensure_clean(load_metadata('.')))

/usr/lib/python3.8/site-packages/pytest_checkdocs/__init__.py:61:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

source_dir = '.', isolated = True

    def project_wheel_metadata(
        source_dir: PathType,
        isolated: bool = True,
        *,
        runner: RunnerType = pyproject_hooks.quiet_subprocess_runner,
    ) -> metadata.PackageMetadata:
        """
        Return the wheel metadata for a project.

        Uses the ``prepare_metadata_for_build_wheel`` hook if available,
        otherwise ``build_wheel``.

        :param source_dir: Project source directory
        :param isolated: Whether or not to run invoke the backend in the current
                         environment or to create an isolated one and invoke it
                         there.
        :param runner: An alternative runner for backend subprocesses
        """

        if isolated:
>           with DefaultIsolatedEnv() as env:

/usr/lib/python3.8/site-packages/build/util.py:41:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <build.env.DefaultIsolatedEnv object at 0x7fac98a0d3a0>

    def __enter__(self) -> DefaultIsolatedEnv:
        try:
            self._path = tempfile.mkdtemp(prefix='build-env-')
            # use virtualenv when available (as it's faster than venv)
            if _should_use_virtualenv():
                self.log('Creating virtualenv isolated environment...')
                self._python_executable, self._scripts_dir = _create_isolated_env_virtualenv(self._path)
            else:
                self.log('Creating venv isolated environment...')

                # Call ``realpath`` to prevent spurious warning from being emitted
                # that the venv location has changed on Windows. The username is
                # DOS-encoded in the output of tempfile - the location is the same
                # but the representation of it is different, which confuses venv.
                # Ref: https://bugs.python.org/issue46171
                self._path = os.path.realpath(tempfile.mkdtemp(prefix='build-env-'))
>               self._python_executable, self._scripts_dir = _create_isolated_env_venv(self._path)

/usr/lib/python3.8/site-packages/build/env.py:89:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

path = '/tmp/build-env-3kcm_ott'

    def _create_isolated_env_venv(path: str) -> tuple[str, str]:
        """
        On Python 3 we use the venv package from the standard library.

        :param path: The path where to create the isolated build environment
        :return: The Python executable and script folder
        """
        import venv

        import packaging.version

        if sys.version_info < (3, 8):
            import importlib_metadata as metadata
        else:
            from importlib import metadata

        symlinks = _fs_supports_symlink()
        try:
            with warnings.catch_warnings():
                if sys.version_info[:3] == (3, 11, 0):
                    warnings.filterwarnings('ignore', 'check_home argument is deprecated and ignored.', DeprecationWarning)
                venv.EnvBuilder(with_pip=True, symlinks=symlinks).create(path)
        except subprocess.CalledProcessError as exc:
>           raise FailedProcessError(exc, 'Failed to create venv. Maybe try installing virtualenv.') from None
E           build._exceptions.FailedProcessError: Failed to create venv. Maybe try installing virtualenv.
E             Command '/tmp/build-env-3kcm_ott/bin/python3 -Im ensurepip --upgrade --default-pip' failed with return code 1
E             stdout:
E               Traceback (most recent call last):
E                 File "/usr/lib64/python3.8/runpy.py", line 185, in _run_module_as_main
E                   mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
E                 File "/usr/lib64/python3.8/runpy.py", line 144, in _get_module_details
E                   return _get_module_details(pkg_main_name, error)
E                 File "/usr/lib64/python3.8/runpy.py", line 111, in _get_module_details
E                   __import__(pkg_name)
E                 File "/usr/lib64/python3.8/ensurepip/__init__.py", line 29, in <module>
E                   _SETUPTOOLS_VERSION = _get_most_recent_wheel_version("setuptools")
E                 File "/usr/lib64/python3.8/ensurepip/__init__.py", line 26, in _get_most_recent_wheel_version
E                   return str(max(_wheels[pkg], key=distutils.version.LooseVersion))
E               ValueError: max() arg is an empty sequence

/usr/lib/python3.8/site-packages/build/env.py:220: FailedProcessError
================================================================================== short test summary info ==================================================================================
FAILED .::project - build._exceptions.FailedProcessError: Failed to create venv. Maybe try installing virtualenv.
=============================================================================== 1 failed, 120 passed in 0.70s ===============================================================================

Why pytest wants to test test this unit in venv? 🤔

List of installed modules in build env for above case

Package                       Version
----------------------------- -------
alabaster                     0.7.16
Babel                         2.14.0
build                         1.0.3
charset-normalizer            3.3.2
cppclean                      0.13
distro                        1.9.0
dnf                           4.18.2
docutils                      0.20.1
exceptiongroup                1.1.3
gpg                           1.23.2
idna                          3.6
imagesize                     1.4.1
importlib_metadata            7.0.1
iniconfig                     2.0.0
invoke                        2.2.0
Jinja2                        3.1.3
libdnf                        0.72.0
markdown-it-py                3.0.0
MarkupSafe                    2.1.3
mdit-py-plugins               0.4.0
mdurl                         0.1.2
mock                          5.1.0
myst-parser                   2.0.0
packaging                     23.2
pluggy                        1.3.0
Pygments                      2.17.2
pyproject_hooks               1.0.0
pytest                        8.0.0
python-dateutil               2.8.2
pytz                          2024.1
PyYAML                        6.0.1
requests                      2.31.0
six                           1.16.0
snowballstemmer               2.2.0
Sphinx                        7.1.2
sphinx-argparse               0.4.0
sphinxcontrib-applehelp       1.0.4
sphinxcontrib-devhelp         1.0.5
sphinxcontrib-htmlhelp        2.0.5
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.7
sphinxcontrib-serializinghtml 1.1.10
tomli                         2.0.1
urllib3                       1.26.18
wheel                         0.42.0
zipp                          3.17.0

[The-Compiler]

Your stack trace shows a pytest-checkdocs creating a virtualenv and failing. Nothing of this seems to be related to pytest core.

[kloczek]

Your stack trace shows a pytest-checkdocs creating a virtualenv and failing. Nothing of this seems to be related to pytest core.

Yes however all prev versions of the pytest did not require access to pip or .whl archives.
I don't see anything about that in release notes about that new behaviour.

[The-Compiler]

Yes, because whatever you're seeing is not something pytest core is doing. Maybe take a look at the stack trace you posted...

[kloczek]

So where is the bug? in pytest-checkdocs? 🤔
And why with pytest 7.4.4 did not been trying to use that modules?
And/or what should be adjusted in all those modules which are now failing the same way? 🤔

[The-Compiler]

Possibly. Hard to say more without a reproducer that works outside your specific environment. The stack trace you posted shows that this isn't coming from pytest itself.

[kloczek]

I've only stared testing my +1.2k modules and on first 50 found that +10 are failing the same way.
Could you please have look at least on installer module which has no to much build dependencies? 🤔

I don't know enough about pytest internals and what exactly has been changed in last 8.0.0 so investigating that is kind of difficult for me 😞

[The-Compiler]

Works fine with a nox -s test after cloning the installer repo. Maybe start with not having pytest-checkdocs installed, which is the culprit according to the stacktrace, and funnily enough not even listed in your environment packages.

[kloczek]

Works fine with a nox -s test after cloning the installer repo. Maybe start with not having pytest-checkdocs installed, which is the culprit according to the stacktrace, and funnily enough not even listed in your environment packages.

Why you are using nox when I'v been reporting issue with pytest? 🤔
Could you pleas try to reproduce testing using pytest?

Please have look one more time on list of modules which I've provided. There is no on that list pytest-checkdocs.

[The-Compiler]

nox invokes pytest, and your output clearly shows that plugin installed.