You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It would be useful to be able to set a custom number of fuzz runs for a particular test. See, for instance, the discussion about fuzzing arrays I had with @mds1here. Array fuzzing is difficult, and the user often needs to adjust the array values to match the business logic of the protocol, e.g. sorting the array in ascending order.
Array operations are expensive and time intensive, so setting the fuzz.runs to a high value in the Foundry config is guaranteed to lead to slow test run times if the user has several tests that fuzz arrays.
A quick and dirty solution is to add a Foundry profile specifically for the fuzzed array tests - in this profile, the fuzz runs would be lower than usual, and potentially the path to test would be different (though until #3841 is implemented, this isn't practical).
A better solution would be for Foundry to provide a way to tweak the number of fuzz runs on a per-test basis, and there might be an elegant way to enable this with custom NatSpec tags. The Solidity SMTChecker currently uses the following custom tag to aid the compiler during formal verification:
/// @custom:smtchecker abstract-function-nondet
We could inspire from this and imagine a similar tag for Foundry to selectively change the fuzz runs, like this:
Adding the NatSpec comment above would instruct Foundry to fuzz the x argument 500 times, ignoring the fuzz.runs value from the Foundry profile only for this particular test.
The text was updated successfully, but these errors were encountered:
PaulRBerg
changed the title
Selective fuzzing with NatSpec comments "@custom:fuzz-runs NUMBER"
Selective fuzz runs with NatSpec comments "@custom:fuzz-runs NUMBER"
Jan 13, 2023
Component
Forge
Describe the feature you would like
It would be useful to be able to set a custom number of fuzz runs for a particular test. See, for instance, the discussion about fuzzing arrays I had with @mds1 here. Array fuzzing is difficult, and the user often needs to adjust the array values to match the business logic of the protocol, e.g. sorting the array in ascending order.
Array operations are expensive and time intensive, so setting the
fuzz.runs
to a high value in the Foundry config is guaranteed to lead to slow test run times if the user has several tests that fuzz arrays.A quick and dirty solution is to add a Foundry profile specifically for the fuzzed array tests - in this profile, the fuzz runs would be lower than usual, and potentially the path to
test
would be different (though until #3841 is implemented, this isn't practical).A better solution would be for Foundry to provide a way to tweak the number of fuzz runs on a per-test basis, and there might be an elegant way to enable this with custom NatSpec tags. The Solidity SMTChecker currently uses the following custom tag to aid the compiler during formal verification:
/// @custom:smtchecker abstract-function-nondet
We could inspire from this and imagine a similar tag for Foundry to selectively change the fuzz runs, like this:
Adding the NatSpec comment above would instruct Foundry to fuzz the
x
argument 500 times, ignoring thefuzz.runs
value from the Foundry profile only for this particular test.The text was updated successfully, but these errors were encountered: