Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --iterate option #354

Merged
merged 9 commits into from
Aug 18, 2024
Merged

Add --iterate option #354

merged 9 commits into from
Aug 18, 2024

Conversation

sourcefrog
Copy link
Owner

@sourcefrog sourcefrog commented Jun 9, 2024

Adds a --iterate option that will skip mutants that passed in a previous run. The idea is that if you are adding tests that, presumably, only improve coverage, you can easily find out if you've now fixed every previous gap.

Fixes #53

This will only be stable if you're using the same filter options (--file) etc on consecutive runs.

Fuzzy matching?

There is a question whether it's better to match the line and column numbers, or better to consider mutants as matched even if they've moved a bit. It's easy to imagine that in the process of improving coverage, some code would move around a bit and ideally we'd detect that it moved but did not change. On the other hand, ignoring the line and column is likely to be ambiguous when there are similarly-named mutants within the same file or function, which might cause some things to be skipped incorrectly.

The process of catching mutants is probably often about adding more test coverage. Since that occurs typically in either integration test files or in unit tests at the bottom of files, it typically shouldn't cause code to move? If the code under test needs to be changed to improve coverage, perhaps it's better to re-test it. So, overall, I think I won't do any fuzzy matching at this point.

TODO

  • Add an --iterate command-line option (it would not make sense to have in the config.)
  • Read in caught.txt and previously_caught.txt before creating the new output directory.
  • Write all of them into previously_caught.txt in the new output directory.
  • Skip previously caught mutants.
  • Integration type tests.
  • Document how to use this.
  • Document the new file in mutants.out.
  • Test that --list excludes previously-caught mutants, but does not update the files.
  • Test there's no previously_caught.txt without --iterate.

@sourcefrog sourcefrog linked an issue Jun 9, 2024 that may be closed by this pull request
6 tasks
@sourcefrog sourcefrog mentioned this pull request Jun 11, 2024
3 tasks
@sourcefrog sourcefrog force-pushed the 53-iterate branch 3 times, most recently from 0418b11 to d30c7ec Compare June 12, 2024 15:24
@sourcefrog sourcefrog marked this pull request as ready for review June 20, 2024 14:50
@sourcefrog sourcefrog enabled auto-merge June 20, 2024 14:50
@sourcefrog sourcefrog disabled auto-merge June 20, 2024 14:50
@sourcefrog sourcefrog merged commit b213788 into main Aug 18, 2024
41 checks passed
@sourcefrog sourcefrog deleted the 53-iterate branch August 18, 2024 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"Iterate failures" mode that skips mutations that were caught last time
1 participant