Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Pass info in v1.7 and later #83

Merged
merged 5 commits into from
Mar 30, 2022
Merged

Use Pass info in v1.7 and later #83

merged 5 commits into from
Mar 30, 2022

Conversation

mmiller-max
Copy link
Member

@mmiller-max mmiller-max commented Feb 26, 2022

I think the original expression is the best thing to use for testname, and thought we may as well keep using the count in the id field as this also closes #62.

Closes #77

EDIT: okay it ended up being a bit more of a change, and it changes test case IDs and names so I have made it a breaking change. Now all passes, failures and errors (except for errors outside of tests) have the id set to a number which increments per testset. For Passes, the name is the original test expression for v1.7.0 and above; and pass (info lost) (Test 1) is the name below v.1.7.0.

@codecov
Copy link

codecov bot commented Feb 26, 2022

Codecov Report

Merging #83 (acbbedc) into master (e170f9d) will decrease coverage by 5.07%.
The diff coverage is 91.66%.

@@            Coverage Diff             @@
##           master      #83      +/-   ##
==========================================
- Coverage   96.92%   91.84%   -5.08%     
==========================================
  Files           7        7              
  Lines         520      552      +32     
==========================================
+ Hits          504      507       +3     
- Misses         16       45      +29     
Impacted Files Coverage Δ
src/to_xml.jl 99.28% <88.88%> (-0.72%) ⬇️
src/testsets.jl 99.06% <100.00%> (+0.02%) ⬆️
src/runner.jl 86.20% <0.00%> (-11.58%) ⬇️
src/compat_check.jl 85.71% <0.00%> (-4.18%) ⬇️

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@mmiller-max mmiller-max marked this pull request as draft February 26, 2022 12:59
@mmiller-max
Copy link
Member Author

Just needs updating docs.

@mmiller-max mmiller-max marked this pull request as ready for review February 26, 2022 13:47
@mmiller-max mmiller-max requested a review from oxinabox March 9, 2022 21:15
@toollu
Copy link

toollu commented Mar 24, 2022

@oxinabox ping. Would love to have it for our repo.

src/to_xml.jl Outdated
@@ -199,7 +199,8 @@ function to_xml(ts::AbstractTestSet)
set_attribute!(x_testcase, "classname", ts.description)
set_attribute!(x_testcase, "time", time_taken(result)::Millisecond)
# Set attributes which require variables in this scope
(result isa Pass || result isa ReportingResult{Pass}) && set_attribute!(x_testcase, "name", x_testcase["name"] * " (Test $total_ntests)")
ntests > 0 && set_attribute!(x_testcase, "id", total_ntests) # Ignore both testsuites and errors outside of tests
(result isa Pass || result isa ReportingResult{Pass}) && VERSION < v"1.7.0" && set_attribute!(x_testcase, "name", x_testcase["name"] * " (Test $total_ntests)")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we introduce a ispass(result) function that covers both these?

@mmiller-max mmiller-max merged commit d120625 into master Mar 30, 2022
@mmiller-max mmiller-max deleted the mm/pass-info branch March 30, 2022 07:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Pass information is now available! 🎆 Put count into testcase id
3 participants