VSCode cannot detect pytest unit tests, even pytest has been installed, error comes from inside .vscode-serve folder, behind company's proxy

[franva]

Issue Type: Bug

Behaviour

Expected vs. Actual

Expect a list of unit tests, actually got the error message: Error discovering pytest tests (see Output > Python):

Here is the error

One strange thing I notice in the above screenshot is: it always has a "d" inside the search bar under Testing even I closed and re-open VSCode, you can see it from the screenshot above.

Here is the evidence that I have installed the pytest

Here is the error message from Output > Python:

LSP Notebooks experiment is disabled -- not in treatment group

python -c "import sys;print(sys.executable)" python -c "import sys;print(sys.executable)" Python interpreter path: /usr/local/bin/python python -c "import sys;print(sys.executable)" Starting Pylance language server. python -c "import sys;print(sys.executable)" python -c "import sys;print(sys.executable)" python -c "import sys;print(sys.executable)" python -c "import sys;print(sys.executable)" python ./.vscode-server/extensions/ms-python.python-2022.10.1/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear workflow cwd: . [ERROR 2022-6-26 13:1:2.338]: Error discovering pytest tests: [n [Error]: Traceback (most recent call last): File "/home/cdsw/.vscode-server/extensions/ms-python.python-2022.10.1/pythonFiles/testing_tools/run_adapter.py", line 17, in from testing_tools.adapter.main import parse_args, main File "/home/cdsw/.vscode-server/extensions/ms-python.python-2022.10.1/pythonFiles/testing_tools/adapter/main.py", line 9, in from . import pytest, report File "/home/cdsw/.vscode-server/extensions/ms-python.python-2022.10.1/pythonFiles/testing_tools/adapter/pytest/init.py", line 7, in from ._discovery import discover File "/home/cdsw/.vscode-server/extensions/ms-python.python-2022.10.1/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 8, in import pytest ImportError: No module named pytest

at ChildProcess. (/home/cdsw/.vscode-server/extensions/ms-python.python-2022.10.1/out/client/extension.js:2:232783) at Object.onceWrapper (node:events:510:26) at ChildProcess.emit (node:events:390:28) at maybeClose (node:internal/child_process:1064:16) at Process.ChildProcess._handle.onexit (node:internal/child_process:301:5)]

python -c "import sys;print(sys.executable)"

Steps to reproduce:

1. Click the Testing button in the side bar

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.6.13
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Global
• Value of the python.languageServer setting: Default

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

XXX

User Settings

languageServer: "Pylance"

linting
• flake8Args: "<placeholder>"

sortImports
• args: "<placeholder>"

formatting
• provider: "black"

testing
• pytestArgs: "<placeholder>"
• pytestEnabled: true

Extension version: 2022.10.1
VS Code version: Code 1.69.2 (3b889b090b5ad5793f524b5d1d39fda662b96a2a, 2022-07-18T16:12:52.460Z)
OS version: Windows_NT x64 10.0.19042
Restricted Mode: No
Remote OS version: Linux x64 3.10.0-1160.59.1.el7.x86_64

System Info

Item	Value
CPUs	11th Gen Intel(R) Core(TM) i5-1145G7 @ 2.60GHz (8 x 1498)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_renderer: enabled_on video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled
Load (avg)	undefined
Memory (System)	15.73GB (5.70GB free)
Process Argv	--crash-reporter-id bbd9672c-80fc-4ddb-9d19-83961752b2ab
Screen Reader	no
VM	0%

Item	Value
Remote	SSH: deo
OS	Linux x64 3.10.0-1160.59.1.el7.x86_64
CPUs	Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz (64 x 2597)
Memory (System)	251.62GB (160.16GB free)
VM	0%

A/B Experiments

vsliv368:30146709
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vswsl492:30256859
vslsvsres303:30308271
pythonvspyl392:30443607
vserr242:30382549
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
vscorecescf:30445987
pythondataviewer:30285071
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
cmake_vspar411cf:30525689
vsaa593cf:30376535
vsc1dst:30438360
pythonvs932:30410667
wslgetstarted:30449410
vscscmwlcmt:30465135
cppdebug:30492333
pylanb8912cf:30529770
vsclangdc:30486549
c4g48928:30534093

[karthiknadig]

@franva Can you try will python version >= 3.7? we dropped support for 3.6.

[franva]

@karthiknadig I can't try newer version of Python. It is managed by company, we can only use what is provided.

It was working before I upgraded Viscose to the latest version. I used to be using 1.59 version.

[karthiknadig]

Try pinning the python extension to version 2022.8.*. You should be able to do this from the extensions view:

[franva]

Hi @karthiknadig

Installed and re-started, the issue persists.

2 Things I'd like to mention:

1. Why is there a "d" in the search bar every time when I open the Testing from the Sidebar?
   

2. I see "unable to watch large workspace files"
   

Not sure whether these 2 things are related to the possible cause.

The error message remains the same, still complains cannot find pytest inside .vscode-server folder.

[karthiknadig]

Why is there a "d" in the search bar every time when I open the Testing from the Sidebar?

That UI is a general testing UI, not specific to python. So there might be some caching going on there. Or some other extension is interfering.

"unable to watch large workspace files"

Thant might be related. Have you installed pytest ? if yes then is it in a virtual environment or installed in the global environment? Can you share the logs from Output > Python panel.

[franva]

Hi @karthiknadig

Yes, I have installed pytest, also provided the log in my 1st post.

I do not use virtual env, there is no (virtual env name) in front of my console command line.

I am aware of that there is conda , but it is not used and the conda version is: 4.10.3

Here is the new log from Output > Python after I installed 2022.8.1:

LSP Notebooks experiment is disabled -- not in treatment group
> conda info --json
> /usr/bin/python ./.vscode-server/extensions/ms-python.python-2022.8.1/pythonFiles/get_output_via_markers.py ./.vscode-server/extensions/ms-python.python-2022.8.1/pythonFiles/interpreterInfo.py
> /usr/bin/python3 ./.vscode-server/extensions/ms-python.python-2022.8.1/pythonFiles/get_output_via_markers.py ./.vscode-server/extensions/ms-python.python-2022.8.1/pythonFiles/interpreterInfo.py
> /usr/local/bin/python ./.vscode-server/extensions/ms-python.python-2022.8.1/pythonFiles/get_output_via_markers.py ./.vscode-server/extensions/ms-python.python-2022.8.1/pythonFiles/interpreterInfo.py
> /usr/local/bin/python3 ./.vscode-server/extensions/ms-python.python-2022.8.1/pythonFiles/get_output_via_markers.py ./.vscode-server/extensions/ms-python.python-2022.8.1/pythonFiles/interpreterInfo.py
> /opt/conda/bin/python ./.vscode-server/extensions/ms-python.python-2022.8.1/pythonFiles/get_output_via_markers.py ./.vscode-server/extensions/ms-python.python-2022.8.1/pythonFiles/interpreterInfo.py
> python -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
Python interpreter path: /usr/local/bin/python
> python -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
Starting Pylance language server.
> python -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> python ./.vscode-server/extensions/ms-python.python-2022.8.1/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear .
cwd: .
> python ./.vscode-server/extensions/ms-python.python-2022.8.1/pythonFiles/linter.py -m pylint ./workflow/tests/conftest.py
cwd: .
[ERROR 2022-6-26 21:28:39.973]: Error discovering pytest tests:
 [n [Error]: ============================= test session starts ==============================
platform linux2 -- Python 2.7.18, pytest-4.6.11, py-1.11.0, pluggy-0.13.1
rootdir: /home/cdsw, inifile: pytest.ini
collected 0 items / 1 errors

==================================== ERRORS ====================================
________________________ ERROR collecting test session _________________________
.local/lib/python2.7/site-packages/py/_path/common.py:383: in visit
    for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
.local/lib/python2.7/site-packages/py/_path/common.py:435: in gen
    for p in self.gen(subdir):
.local/lib/python2.7/site-packages/py/_path/common.py:425: in gen
    if p.check(dir=1) and (rec is None or rec(p))])
.local/lib/python2.7/site-packages/_pytest/main.py:667: in _recurse
    ihook = self.gethookproxy(dirpath)
.local/lib/python2.7/site-packages/_pytest/main.py:482: in gethookproxy
    my_conftestmodules = pm._getconftestmodules(fspath)
.local/lib/python2.7/site-packages/_pytest/config/__init__.py:431: in _getconftestmodules
    mod = self._importconftest(conftestpath.realpath())
.local/lib/python2.7/site-packages/_pytest/config/__init__.py:470: in _importconftest
    raise ConftestImportFailure(conftestpath, sys.exc_info())
E   ConftestImportFailure: (local('/home/cdsw/workflow/tests/conftest.py'), (<type 'exceptions.ImportError'>, ImportError('No module named lib.utils',), <traceback object at 0x7f3deb7b1050>))
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=========================== 1 error in 0.17 seconds ============================

Traceback (most recent call last):
  File "/home/cdsw/.vscode-server/extensions/ms-python.python-2022.8.1/pythonFiles/testing_tools/run_adapter.py", line 22, in <module>
    main(tool, cmd, subargs, toolargs)
  File "/home/cdsw/.vscode-server/extensions/ms-python.python-2022.8.1/pythonFiles/testing_tools/adapter/__main__.py", line 100, in main
    parents, result = run(toolargs, **subargs)
  File "/home/cdsw/.vscode-server/extensions/ms-python.python-2022.8.1/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 44, in discover
    raise Exception("pytest discovery failed (exit code {})".format(ec))
Exception: pytest discovery failed (exit code 2)

	at ChildProcess.<anonymous> (/home/cdsw/.vscode-server/extensions/ms-python.python-2022.8.1/out/client/extension.js:2:232795)
	at Object.onceWrapper (node:events:510:26)
	at ChildProcess.emit (node:events:390:28)
	at maybeClose (node:internal/child_process:1064:16)
	at Process.ChildProcess._handle.onexit (node:internal/child_process:301:5)]
> python -c "import pylint"
> python -c "import pylint"

Linter 'pylint' is not installed. Please install it or select another linter".
Error: Module 'pylint' not installed.
> python -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"

Also one more observation:

[franva]

I noticed, the VSCode automatically picks up Python2 , rather than Python3, this could be the cause of the entire issue.
How can we change it to pick up P3 by default?

Also I have installed pylint

[karthiknadig]

@karrtikr It looks like there is some issue with python selection, can you take a look?

[karrtikr]

You can change the interpreter selected using Python: Select Interpreter command. Also. it looks like python in terminal refers to /usr/local/bin/python which is python2, so alternatively modify the PATH environment variable to point to python3 instead.

[franva]

@karrtikr and @karrtikr

Thanks for your help. The issue is solved by selecting python interpreter.

Just one suggestion: could we make Python3 the default interpreter if OS has both P2 and P3?

Also, one more observation, the Testing button in sidebar sometimes disappeared and I had no way to show it when that happened.

[karrtikr]

@franva We should ideally be recommending Python3, can you send a screenshot of the list returned using Python: Select Interpreter command. The one marked by a star at the top is the default interpreter we choose.

[franva]

@karrtikr

Sure, here it is :

As I can see that it picks up the conda interpreter by default and it has an exclamation mark in front, I do not know what it means. Doesn't look good tho.

But the default installed conda is a very old version built in with the system image used by our company on CDSW.

So I guess the suggestion is:
it seems that the VSCode can already detect issues for a particular Python Interpreter, so the priority for picking up the default interpreter should be:

1. pick up conda interpreter if no issues found, else,
2. pick up the latest non-conda Python Interpreter, in our case, it is the Python 3.6.13 rather than Python2

[karrtikr]

@franva Thanks for the info, I have ensured interpreters with issues appear towards the end of the list, so they're not autoselected. And we select the first non-conda global interpreters, if no interpreters exist in the workspace.

[karrtikr]

Fix should be out in the pre-release version of the extension in a few minutes, use the following to try it out, make sure to run Python: Clear workspace interpreter setting command to clear any workspace configs, and the double-check the list to see which one we recommend:

[franva]

Hi @karthiknadig and @karrtikr
Thanks for fixing this issue.

However, I found another issue and created an issue here: #19642

Hopefully you could also take a look.

Many thanks

Winston