Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit verbosity of fizzy-spectests #450

Closed
chfast opened this issue Aug 1, 2020 · 1 comment · Fixed by #453
Closed

Limit verbosity of fizzy-spectests #450

chfast opened this issue Aug 1, 2020 · 1 comment · Fixed by #453
Assignees

Comments

@chfast
Copy link
Collaborator

chfast commented Aug 1, 2020

The output from fizzy-spectests is quite large as it reports status of every executed test case. I expect users are mostly interested in failures. Some ideas how to improve it.

  1. Skip PASSED reports by default. Easy to implement.
  2. Repeat status of each file in the end. This will allow at least quickly find the file with failures and repeat the run on the subset of files.
  3. Use "dot notation" per file. Each test result is represented by a single character. E.g.
    binary.wast: ......F.....s.....
    Line 98: assert_return FAILED Incorrect returned value. Expected: 9219994337134247936 (0x7ff4000000000000) Actual: 0 (0x0)
    Line 99: assert_return SKIPPED Unsupported feature: Floating point instruction.
    
@axic
Copy link
Member

axic commented Aug 2, 2020

I like a combination of 1. and 3.:

  • display it like you propose in 3.
  • do not display SKIPPED by default, but add an option --show-skipped to show them

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants