TPI: UNITTEST output

[eleanorjboyd]

Refs: 21861

• macOS @aeschli
• linux @connor4312
• windows @karrtikr

Complexity: 4

Author: @eleanorjboyd

Create Issue

---

Thank you for doing this testing!

General Setup

1. Clone the test repo from here or use your own repo of pytests.

2. Launch VSCode and open the inc_dec_example_repo/inc_dec_example_test_suite testing repo folder.

3. Add the following to your settings in VSCode: "python.experiments.optInto": ["pythonTestAdapter"],

4. Use the "create environment" command from the command palette, select venv and a Python version >=3.7.

5. click the test explorer icon (beaker)

6. configure tests
   1. unittest: unittest → root → pattern *test*.py

7. Testing:

   1. in your settings.json you should see a "python.testing.unittestArgs": [] make the following changes and see the desired output. Output will be in the Test Results panel unless stated otherwise

8. switch to using log level error (in UNITTEST)

   1. check to see that only error and critical logs show up (change back after)

9. in the test result panel click on a failed test item

   1. check that you can see a stack trace of sorts about the failed test

10. change test assert to assert True

    1. check that printing and logging does not appear

11. Add ["-v"], and keep assert True

    1. check that logging now occurs

12. From the terminal run python -m unittest test_logging.py

    1. make sure it looks the same as it looks when you run just test_logging.py from the test explorer

Non execution related tests

1. switch to “python test logs” confirm you see a deprecation message

[aeschli]

@eleanorjboyd Can you clarify 7. and 8. ?
7. python.testing.unittestArgs is in my settings, not with an empty array, but reflecting the options configured in step 6. Not sure what changes I'm supposed to do.
8. How do I change the log level?

[eleanorjboyd]

for 7, the formatting got messed up you have it correct that it should have the configure options.

Log level are changed via these lines in the test_logging.py file. Sorry that wasn't clear in the steps!

logger.setLevel(logging.DEBUG)  # Set the desired log level
        # logger.setLevel(logging.ERROR)  # Set the desired log level

[connor4312]

For (7), I observe the same as Martin.

For (10) / (11)'s reference to the asserts, I am a little confused since testing_logging2 where the assert is is not discovered/run as a test, either by the extension or python -m unittest test_logging.py. For (11), -v is already on the unittestArgs seen in step 6.

[eleanorjboyd]

yes sorry for unittest you should be seeing test_logging under TestLogger class as unittest can only discover that test and not test_logging2.

class TestLogger(unittest.TestCase):
    def test_logging(self):

For test_logging if you could instead add an assert True and see if you get less output that would be ideal. Thanks!

[aeschli]

On 11. as mentioned by Connor, -v is already part of the array. Removing it does not make a difference in the output of the Test Result view

On 12. Running from the terminal does not seem to update the Test Result, Python Test log or the decorators. Not sure what is the expected result...

I have mostly guessed what we need to test here. Something with output. A brief summary of what has been implemented would have been nice. It's hard to know what from #21861 has been implemented at last.

[eleanorjboyd]

Thank you for testing @aeschli, I will provide more information with a summary next time! The 12 is correct, running from the terminal does not integrate with the rest of the extension and therefore you shouldn't see any updates to the Test Result, Python test logs etc. Instead I was just asking to see if what is printed as output from unittest (such as stack traces for errored tests) look the same as what a user can find in the Test Result panel (as they look to find the same information in our test results tab they would find when running unittest from the most basic scenario). Thanks!