Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Demo of typical changes to experiment template when making a specific experiment #8

Draft
wants to merge 4 commits into
base: experiment
Choose a base branch
from

Conversation

sandypreiss
Copy link
Collaborator

Not planning to merge. This draft PR exists to demonstrate the typical changes that we make to the experiment template branch when making a new branch for a specific experiment.

  • Make a new branch with the exp_ prefix. Depending on the experiment, this could branch off the experiment template branch or a prior specific experiment branch if that's easier. For example, if we make a bunch of changes for the exp-complex-experiment branch, and then want to run a second batch for that experiment with a few small tweaks, it makes more sense to branch exp-complex-experiment-batch-2 from exp-complex-experiment.
  • Change the S3 bucket name in run_iteration.sh and summarize.py
  • Change prepare.py to create the scenarios you need for this experiment. This is typically the most complex and thought-intensive step in defining a new experiment. All the other steps are just logistics; this is where we have to think through how to define the scenarios for the experiment.
  • Update the README to reflect these changes.

Then follow the steps in the README to run the experiment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants