Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See #227 for discussion on this.
With this PR, configuration settings needed for testing now live in
tests/conf
, while settings needed for reproducibility now live inconf
. I've done my best to restore the defaults back to the settings when these files were originally added, but let me know if I've missed anything. Once @calebrob6 has all of these datasets on a HDD somewhere we can try running these to make sure the settings are valid and produce good results.My goal for settings in
tests/conf
is to simplify the problem as much as needed so that the tests run quickly (small num_workers, batch_size, num_classes, etc.). My goal for settings inconf
is to achieve state-of-the-art performance on datasets wherever possible. The defaults in this file should be reasonable (1 GPU, 4 workers, datasets indata
), but can be overridden on the CLI if needed.Unfortunately there isn't an easy way to test
conf
, and the settings in these files have rapidly become out-of-date. We'll have to be careful to update both locations in the future, not justtests/conf
.Closes #227