Improve Release Automation

[willarmiros]

Background

Currently, the release process is outlined in RELEASING.md. It is an entirely manual process, and is partially responsible for contrib releases to have fallen behind those in the core repo. To summarize the process, you must first prepare the repo for release by updating the dependency and package versions and updating the changelog, create a PR with these changes (example: #316), make additional fixes to example code/documentation in that PR, get the PR approved & merged, actually release each package to NPM, then finally publish a GitHub release.

Note: There is also a final step to update the changelog again, but this seemed like it was handled in #316 without the need for a separate PR.

Proposal

Many of the commands in the release guide can be automated using GitHub Actions, but some steps like fixing breaking outdated example code are necessarily manual. GH Actions can be used in a 3-step approach to take advantage of automation and manual review:

1. An automated GH Workflow that takes a single input, the version (e.g. 0.14.0), and produces a release PR like chore: version 0.13.0 #316. The PR would include version bumps and the changelog update.
2. Maintainers would manually update examples/ and getting-started/ code as they do today, then approve the PR.
3. A second workflow would be triggered by merging of the release PR and would actually release the packages to NPM and cut the GitHub Release.

As a side note, one benefit of having this 3-step, partially manual approach is that even when new contributors are given write access so that they can be added to CODEOWNERS, they will not be able to complete a release with their access to GH workflows.

Open Questions

• Where should the release pull request be made from? An agreed upon fork like dynatrace-oss-contrib, or a branch within the main repo?
• Similar vein, who's NPM token to use for publishing? I've noticed there's already one in the core repo for canary tests.
• How can we enforce the second workflow only running after the release PR is merged? My current idea is to give the release PR a label like "release", and trigger the workflow when any PR is merged, but make it a no-op unless the PR has the "release" label. Open to other ideas, but GH Workflows are definitely limited here.
• Can we create a draft release in workflow 1, then publish that draft in workflow 2? Need to research GH Actions ability to do this more.

[dyladan]

from SIG:

• Where should the release pull request be made from? An agreed upon fork like dynatrace-oss-contrib, or a branch within the main repo?

Branch

• Similar vein, who's NPM token to use for publishing? I've noticed there's already one in the core repo for canary tests.

I will create a token in this repo with the same name as the one in the core repo

• How can we enforce the second workflow only running after the release PR is merged? My current idea is to give the release PR a label like "release", and trigger the workflow when any PR is merged, but make it a no-op unless the PR has the "release" label. Open to other ideas, but GH Workflows are definitely limited here.

I think the common way to do this is to use a branch pattern like release/0.12.1

• Can we create a draft release in workflow 1, then publish that draft in workflow 2? Need to research GH Actions ability to do this more.

Should be possible I would think but it's not a blocker if it's difficult or impossible.

[willarmiros]

I think the common way to do this is to use a branch pattern like release/0.12.1

Thanks for this idea, should definitely be an easier way to go about it :) Should be able to detect when PRs from such a branch are merged and trigger workflow 3 from there.

Also a follow up from the SIG, it turns out GitHub only support text input for manually dispatched actions. So, to ensure versions are calculated by the workflow and not manually, we could only accept minor and patch as input (and fail otherwise). The default text could even be patch | minor.

That being said, only accepting the patch/minor/major keywords could cause confusion by itself, and be less flexible if we wanted to release, say, a 1.0.0-rc.1 which couldn't be computed by the workflow. As for potential typos in the manual input, the release PRs are being reviewed for a reason :)

[willarmiros]

Can we create a draft release in workflow 1, then publish that draft in workflow 2?

This should be easy enough, we can create the release with the Github-provided create-release action. This does not support updating releases, but this 3rd party action does, so we can use it to move the release out of draft.

[dyladan]

i'm fine entering a version manually I suppose

[obecny]

Do you have somewhere some example how this workflow looks like (with accepting some text input from a user) ?, can be screenshot etc. ?

[willarmiros]

@obecny I'll wire it up to a little sample repo to test it and demonstrate how it works. This is how the input part looks on another repo though (ignore the last run failing because Gradle is annoying):

[willarmiros]

Can we create a draft release in workflow 1, then publish that draft in workflow 2?

This should be easy enough, we can create the release with the Github-provided create-release action. This does not support updating releases, but this 3rd party action does, so we can use it to move the release out of draft.

It actually turns out that in order to update an existing draft release via GH Actions, the release has to be tagged. I don't think we should be pushing tags prior to publishing the release, in case more changes are added to the release after the release PR is created as was the case in #316. This would also cause some disagreement between the CHANGELOG and the changes listed in the release body. Since it's not a blocker, I'll probably only create the release in workflow 3.

Alternatively, you could enforce not having additional changes in a release PR once the workflow is kicked off, but that feels over-constrictive.

In the future, e.g. after everything reaches 1.0 stability and we are comfortable not having a manual PR, this problem would go away.

[dyladan]

You could also have a proposal tag and change the tag a release points to before publishing it.

[willarmiros]

@dyladan thanks for the pointer! So after looking into it, I can create a draft release that's linked from the release PR, then publish that release after the PR is merged automatically. However, that process will be pretty complicated because it involves several GitHub API calls in both workflows. I would also be re-generating the body of the release (the changelog) in case additional features were merged while the release PR was open, so manually checking the draft release wouldn't do much good anyway since it'll be overwritten when the PR is merged.

tldr: I think since the release is autogenerated, there is little value in reviewing it, so we should simply create the final release after the release PR is merged, along with publishing the packages to NPM.