You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
If one of the tests in tap output is skipped before some failure, avocado will mark the whole output as skip instead of fail and users are not notified about the failure.
Steps to reproduce
TAP output generated by tap test stored in example.tap:
TAP version 13
# MUST be treated as a SKIP test
ok 1 - must be skipped test # SKIP
not ok 2 - actual failure
ok 3 - passed test
bash script for reading this file as test.sh:
#!/bin/bash -
cat example.tap
JSON for running the tap test as test.json:
[
{"kind": "tap", "uri": "./test.sh"}
]
Avocado command:
$ avocado run test.json
Expected behavior
A clear and concise description of what you expected to happen.
JOB ID : f9305c1fa08da1d0a2e0a668e76905e23182e46a
JOB LOG : /home/janrichter/avocado/job-results/job-2024-08-22T11.41-f9305c1/job.log
(1/1) /home/janrichter/Avocado/testing/tap/test.sh: STARTED
(1/1) /home/janrichter/Avocado/testing/tap/test.sh: FAIL
RESULTS : PASS 0 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML : /home/janrichter/avocado/job-results/job-2024-08-22T11.41-f9305c1/results.html
JOB TIME : 3.51 s
Current behavior
What you are getting?
JOB ID : f9305c1fa08da1d0a2e0a668e76905e23182e46a
JOB LOG : /home/janrichter/avocado/job-results/job-2024-08-22T11.41-f9305c1/job.log
(1/1) /home/janrichter/Avocado/testing/tap/test.sh: STARTED
(1/1) /home/janrichter/Avocado/testing/tap/test.sh: SKIP
RESULTS : PASS 0 | ERROR 0 | FAIL 0 | SKIP 1 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML : /home/janrichter/avocado/job-results/job-2024-08-22T11.41-f9305c1/results.html
JOB TIME : 3.51 s
System information (please complete the following information):
This commit adds fixes for TAP runner to not hide failures when some
tests are skipped. Also, it updates Avocado documentation to respect
newest changes in TAP runner.
Reference: avocado-framework#6007
Signed-off-by: Jan Richter <[email protected]>
Describe the bug
If one of the tests in tap output is skipped before some failure, avocado will mark the whole output as
skip
instead offail
and users are not notified about the failure.Steps to reproduce
TAP output generated by tap test stored in
example.tap
:bash script for reading this file as
test.sh
:JSON for running the tap test as test.json:
Avocado command:
Expected behavior
A clear and concise description of what you expected to happen.
Current behavior
What you are getting?
System information (please complete the following information):
master
github
The text was updated successfully, but these errors were encountered: