CTSM Testing and PR complex workflows

Workflows for a branch-off-a-branch

In many cases, it's best to start each feature branch off of the latest version of the ctsm master branch. However, there are some cases where you have two separate developments (or two stages of one development), where the second depends on changes in the first. This often happens when you are separating your answer changes from your bit-for-bit refactoring.

This section lays out some workflows you can use for doing separate PRs and test suite runs for these two branches, when branch2 depends on changes in branch1.

(The images here were generated via the 'git gui' command, with the menu command, 'Repository: Visualize All Branch History'.)

General workflow

Here, the initial development happened on branch1. Testing was run on this branch and a PR was submitted. Then branch2 had some separate but related developments, so started from the head of branch1. Separate testing was run on branch2 (with comparisons against the baselines generated from branch1), and a separate PR was submitted.

In the PR reviews, changes were requested for both branch1 and branch2. The best way to handle this is to:

1. Change branch1 as per reviewer comments
2. Rerun testing on branch1
3. Merge branch1 to master
4. Change branch2 as per reviewer comments
5. Merge new master into branch2
6. Rerun testing on branch2 (with comparisons against the baselines generated from the last test run done on branch1)
7. Merge branch2 to master

Simplified workflow when no additional testing is needed

As is pointed out here, you can skip the final round of testing if you already ran testing when submitting your PR, and no substantive changes were requested as part of the PR review.

If you were lucky enough that no changes were requested on either of your PRs, then the workflow simplifies to this:

Alternative workflow to reduce testing

Now we're back to the general scenario where changes were requested on both branches. If you want to only do one final run of the test suite rather than two, then you can use the following workflow.

This workflow is only allowed if both branches are bit-for-bit. If either or both branches change answers, then they should be tested and merged separately.

Here, the workflow following the PR reviews is:

1. Change branch1 as per reviewer comments
2. Change branch2 as per reviewer comments
3. Merge branch2 into branch1
4. Run testing on branch1
5. Merge branch1 to master