pytest skipping "getting started" test with async error

[sfinnie]

I'm seeing an error trying to run the sample server + test from the getting started docs. Specifically, pytest skips the test instead of running it.

Re-creating

1. Create new venv & activate
2. Install pytest v7.3.2, pygls v1.0.2 and pytest-lsp v0.3.0
3. create server.py and test_server.py from docs
4. run pytest

Result:

================================ test session starts =================================
platform linux -- Python 3.10.6, pytest-7.3.2, pluggy-1.1.0
rootdir: /home/sfinnie/projects/lsp/pytest-lsp
plugins: lsp-0.3.0, typeguard-3.0.2, asyncio-0.21.0
asyncio: mode=strict
collected 1 item                                                                     

test_server.py s                                                               [100%]

================================== warnings summary ==================================
test_server.py::test_completions
  /home/sfinnie/projects/lsp/pytest-lsp/venv/lib/python3.10/site-packages/_pytest/python.py:183: PytestUnhandledCoroutineWarning: async def functions are not natively supported and have been skipped.
  You need to install a suitable plugin for your async framework, for example:
    - anyio
    - pytest-asyncio
    - pytest-tornasync
    - pytest-trio
    - pytest-twisted
    warnings.warn(PytestUnhandledCoroutineWarning(msg.format(nodeid)))

test_server.py::test_completions
  /home/sfinnie/projects/lsp/pytest-lsp/venv/lib/python3.10/site-packages/pytest_asyncio/plugin.py:444: DeprecationWarning: pytest-asyncio detected an unclosed event loop when tearing down the event_loop
  fixture: <_UnixSelectorEventLoop running=False closed=False debug=False>
  pytest-asyncio will close the event loop for you, but future versions of the
  library will no longer do so. In order to ensure compatibility with future
  versions, please make sure that:
      1. Any custom "event_loop" fixture properly closes the loop after yielding it
      5. Your code does not modify the event loop in async fixtures or tests
  
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== 1 skipped, 2 warnings in 0.64s ===========================

Resolution (partial)

Marking the test as async means pytest runs it successfully:

import pytest

@pytest.mark.asyncio
async def test_completions(client: LanguageClient):
    # test body unchanged

It still generates a warning though:

================================ test session starts =================================
platform linux -- Python 3.10.6, pytest-7.3.2, pluggy-1.1.0
rootdir: /home/sfinnie/projects/lsp/pytest-lsp
plugins: lsp-0.3.0, typeguard-3.0.2, asyncio-0.21.0
asyncio: mode=strict
collected 1 item                                                                     

test_server.py .                                                               [100%]

================================== warnings summary ==================================
test_server.py::test_completions
  /home/sfinnie/projects/lsp/pytest-lsp/venv/lib/python3.10/site-packages/pytest_asyncio/plugin.py:444: DeprecationWarning: pytest-asyncio detected an unclosed event loop when tearing down the event_loop
  fixture: <_UnixSelectorEventLoop running=False closed=False debug=False>
  pytest-asyncio will close the event loop for you, but future versions of the
  library will no longer do so. In order to ensure compatibility with future
  versions, please make sure that:
      1. Any custom "event_loop" fixture properly closes the loop after yielding it
      2. Your code does not modify the event loop in async fixtures or tests
  
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================ 1 passed, 1 warning in 0.65s ============================

I think that's covered in a separate issue though.

Alternative Resolution

Rather than having to decorate every test as async, add a pyproject.toml:

[tool.pytest.ini_options]
asyncio_mode = "auto"

The test will then run as per docs (though still generates the warning).

HTH.

[alcarney]

Thank you for the detailed report!

Both resolutions you posted (adding the mark or setting the config option) are equivalent methods to fix this - it looks like I missed a step or two when writing up the getting started guide!

As for the warning, you are right it was covered in an issue here however, the fix requires a patch to pygls which I don't think has been released yet (perhaps another note for the docs in the meantime! 😅). The warning itself is fairly mild - pytest-lsp/pygls are not cleaning up after themselves. So should be safe to ignore while we wait for the next pygls release

[sfinnie]

Thanks @alcarney & you're welcome :).

Both resolutions you posted (adding the mark or setting the config option) are equivalent methods to fix this

Yes. I mentioned both for completeness given two slightly different use-cases: decorator for the docs (1 less file), pyproject.toml for users who want to write more than 1 test (save repetition).

Thanks for info on the pygls patch, didn't know about that.