Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parameter for retrying a property while shrinking #212

Merged
merged 3 commits into from
Dec 16, 2021

Conversation

jmid
Copy link
Collaborator

@jmid jmid commented Dec 16, 2021

This PR adds a parameter ~retries to QCheck and QCheck2 tests.
When passing a ~retries number n greater than 1, the tested property is checked n times for each shrunk input candidate.
The default value is 1, thus causing no change in behaviour.

This feature can be useful when testing non-deterministic code with QCheck (such as https://github.com/jmid/multicoretests)
In the words of Hughes-Bolinder:Erlang11, Sec.6:
"As we explained in section 4, we ensure that tests fail when races are present simply by repeating each test a large number of times, and by running on a dual core machine. We obtained the minimal failing cases in the previous section by repeating each test 100 times during shrinking: thus we stopped shrinking a test case only when all of its candidate shrinkings passed 100 tests in a row."

@c-cube
Copy link
Owner

c-cube commented Dec 16, 2021

That seems most reasonable. Would you consider adding these explanations in comments somewhere, so they live nearer the code?

@c-cube c-cube self-requested a review December 16, 2021 14:14
Copy link
Owner

@c-cube c-cube left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good to merge, but feel free to comment more if you want to :)

@jmid jmid merged commit 1eefb22 into c-cube:master Dec 16, 2021
@jmid jmid deleted the shrinking-retries branch April 2, 2022 11:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants