Demo of typical changes to experiment template when making a specific experiment #8
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Not planning to merge. This draft PR exists to demonstrate the typical changes that we make to the
experiment
template branch when making a new branch for a specific experiment.exp_
prefix. Depending on the experiment, this could branch off theexperiment
template branch or a prior specific experiment branch if that's easier. For example, if we make a bunch of changes for theexp-complex-experiment
branch, and then want to run a second batch for that experiment with a few small tweaks, it makes more sense to branchexp-complex-experiment-batch-2
fromexp-complex-experiment
.run_iteration.sh
andsummarize.py
prepare.py
to create the scenarios you need for this experiment. This is typically the most complex and thought-intensive step in defining a new experiment. All the other steps are just logistics; this is where we have to think through how to define the scenarios for the experiment.Then follow the steps in the README to run the experiment.