"Iterate failures" mode that skips mutations that were caught last time

[sourcefrog]

If someone's iterating on trying to improve coverage it might be helpful to only try mutants that were not caught on the last run, maybe by automatically looking in mutants.out.

However you can get close to this today by narrowing it with -f.

Closely related to #57.

---

Implementation approach

I'll pick this up, based on ideas for @cl3joly but in a separate branch.

This could be done in either a positive or negative sense:

• Skip mutants that passed on a previous run: will pick up newly added mutants (probably good); will test mutants that were not yet known to have passed; does require remembering which one are skipped or they will be included in a third run
• Only test mutants that failed on a previous run

Basic idea is to:

• Add an --iterate command-line option (it would not make sense to have in the config.)
• Add the concept of an "previous output directory", by default the same path that will be used for output.
• Read the previously caught mutants from caught.txt in that output directory
• (It would help here to also have a list of mutants that were still pending and untested when the previous run finished, Write a file with a list of pending mutants into mutants.out #290.)
• If those lists are all empty, print an info message that there's no iterative work to do.
• After generating all the mutants, filter down to only the names that occur in the list. (There's an opportunity here to only generate mutants from files that occur in the lists, but that could be left for later.)

In this approach if you also change the filters, for example focussing on one file at a time, then after that change it will test everything, which seems OK.

[cl3joly]

I’m going to give this one a try.

[sourcefrog]

Great! If you want to talk about the approach or anything please feel to post here or put up a draft PR.

[cl3joly]

Thanks @sourcefrog , I’ve opened #174 as a draft PR, trying to explain our thought process. Feel free to chime in, keeping in mind that we still need to fix and finish things there.

[ASuciuX]

Hello, I've added my question on the PR, please let me know if it is better to post here instead #174 (comment) .

[sourcefrog]

Draft docs: https://github.com/sourcefrog/cargo-mutants/pull/354/files?short_path=16132c8#diff-16132c81ea2bb6b499b9b60b2a5a7e941ed553e028ae2807d89ba609d980c19a