[META] Thoughts on release notes generation

[ryangribble]

As mentioned in #1137 we can work towards automating the release notes (using octokit of course!). We are doing a similar concept at work so here are a few ideas/thoughts based on what we've done so far

Obviously the key is "grooming things as we go" so that there is not much (ideally not any) i dotting or t crossing or rewording to be done, when we are "ready" for a release. In an ideal world, we would just execute a script which cuts the branch, generates the release notes and only needs to be reviewed before hitting the button to publish

• The reworked labels let us categorize the different types of issues (bug fix, feature, etc) the same way @shiftkey has been organizing the release notes manually

• We could add the concept that a comment made on an issue/PR with specific wording, can "set" what the release notes should say for that issue. If not present it could either fail the release notes process, or fall back to using the actual issue title
  For example, if the issue is "Error xxxxx when loading my public repositories" we might make a comment saying

  release_notes: An issue affecting some Public Repositories from being loaded has now been rectified

• this could be even further extended for example if there was a comment for "breaking changes" then that whole comment could also be called out in the release notes under that section.
  eg:

  release_notes_breaking_changes:
  ** intro/title of breaking change **
  stuff goes here
  including code samples
  etc

• Should every PR be associated with an issue or do we need to handle release notes sometimes being for a PR item and other times being for one (or more) fixed issues via a PR?

• For each PR/issue we can get the commits/authors so we can continue the tradition of @mention each author against that line item, as @shiftkey has been doing

• If @shiftkey wants to mention anybody who "assisted" even if that was reporting an issue or taking part in review/discussions, we could also build up a list of commenters on all the PR's in a release then have a general "thanks to the following who were part of helping with this release" or something similar at the end of the release notes. Or alternatively could @mention each user that discussed the PR, on that PR line item rather than just the commit authors (in this case we should probably scrub any repo maintainers from those mentions, since we would then be listed on nearly every line item!)

• Do standalone commits (ie commit on master that wasn't associated with a pull request) need to be handled? Perhaps in octokit we can safely say we only have merged PR's to contend with but in my case we did have to handle "stand alone" commits as well. In this case it was then important to filter out the merge commits of the PR's into master, which was easy since github generates that text so it's easy to parse/identify these. Probably not much a concern in Octokit's case but in our case we were looking for work item ID's in commit messages and if people named their branches after their workitems that did cause the PR merge commits to be seen to "mention" the work items...

[shiftkey]

Me right now:

We could add the concept that a comment made on an issue/PR with specific wording, can "set" what the release notes should say for that issue.

I like this - I'm fine with reading all the issues for this syntax - we can also look at the author to ensure it's someone trusted (whatever that means)

If not present it could either fail the release notes process, or fall back to using the actual issue title
For example, if the issue is "Error xxxxx when loading my public repositories" we might make a comment saying

Yeah, the PR titles often need a bit of love. I wouldn't fail the process here. Just grab the title and use it as a placeholder, e.g. "CLARIFY: Error xxxxx when loading my public repositories"

this could be even further extended for example if there was a comment for "breaking changes" then

I'm still not settled on a syntax for breaking changes in the release notes (I'd rather explain things in a human friendly way), so perhaps we should just use a label to flag PRs that have breaking changes and see what happens next?

Should every PR be associated with an issue or do we need to handle release notes sometimes being for a PR item and other times being for one (or more) fixed issues via a PR?

Let's not assume every PR has an issue associated with it. The IDs I use in the release notes are for the PR, as the contribution is the thing I care about here. So let's continue down that path.

For each PR/issue we can get the commits/authors so we can continue the tradition of @mention each author against that line item

If we're going to script this, we should look at the author of the PR commits as well as the person who opened the PR.

If @shiftkey wants to mention anybody who "assisted" even if that was reporting an issue or taking part in review/discussions, we could also build up a list of commenters on all the PR's in a release then have a general "thanks to the following who were part of helping with this release" or something similar at the end of the release notes

YES. I'd love to shout out to people who helped with reviewing. Let's do this!

Do standalone commits (ie commit on master that wasn't associated with a pull request) need to be handled?

Nope. I want everything to go (including my stuff) through a PR, even if it's trivial.

One little thing that I don't have a good answer for is how multiple PRs can span a feature. But let's cross that bridge later.

[ryangribble]

One little thing that I don't have a good answer for is how multiple PRs can span a feature. But let's cross that bridge later.

I guess you are referring to here to a large body of work (eg implementing your API Pagination changes) that is implemented via multiple PRs? In some ways I wonder if just having each PR be listable is ok - eg "Added pagination support to xxx client" "added pagination support to yyy client". Is there a need to group these together?

if so some possibilities would be:

• use more feature labels - like feature-pagination and then that allows you to further group the PRs under the features area, into higher level features (eg pagination in that case)
• use a single issue for the parent feature and link to it in the multiple PRs
• use an optional milestone to relate multiple PRs - if milestone is set you can group or consolidate PRs for that milestone into their own subsection (or single line item with comma separated PR numbers)

[shiftkey]

@ryangribble

Is there a need to group these together?

Perhaps, perhaps not. Here's a recent example from v0.19:

Add support for Visibility and Affiliation to repository search - #1096, #1132 via @Sarmad93, @AlexP11223

The first PR was to implement the feature, and the second was to address a regression that was introduced not caught during review. I didn't feel that the second PR warranted it's own line as it was within the context of the same release.

That's kind of how I feel about the upcoming pagination PRs - having a separate entry for each PR feels noisy, and I'd rather roll it all up into one item to summarize.

Out of those suggestions, the milestone one actually feels the best fit - I'll have a bunch of issues which are all linked to that initial feature, and I don't want to publish this release until it's all merged. We could look for a milestone and group items in that way separate to all this - or just "squash" together the PRs and contributors into a single item...

Thoughts?

[haacked]

That's kind of how I feel about the upcoming pagination PRs - having a separate entry for each PR feels noisy, and I'd rather roll it all up into one item to summarize.

Agreed.

Also, I think we should only look at PRs for release notes comments. It'll reduce the number of times we might have double notes (issue and associated PR) and it keeps us in the discipline of always submitting a PR for changes.

I don't think we need to worry about "trusted" authors. Instead, when we review a PR that should have a release note, we can simply edit any comment that has the release note to fit what we want. In fact, the release note should be part of the review and we should encourage PR submitters to try and articulate what the note should be and then we can edit it if it needs editing.

BTW, I ❤️ where this is going. When we come up with something good here, I'll want to use it on every OSS project I'm on. 😄

[M-Zuber]

This all looks lovely 💖

Maybe once a syntax is decided on, you can make an pullrequest_template?

edit: Oh and the actual script/code should be a separate repo IMO

[ryangribble]

for anyone who hadnt seen: https://github.com/shiftkey/hermes

I dont know if I want to tackle fsharp at this point though LOL so I may just contribute cerebrally 😀

Having a look at what @shiftkey has produced so far I think we could benefit from adding a new label (in addition to feature and bugfix) to group up all of the recent items around fixing inconsistencies and tidying things up - something like code quality as the label name

I also think we can start "decorating" the PR's with the special comment to set what the release notes text for that PR should be, as the raw list is all over the shop 😭. Ideally we will do this as we go, but for this time around I'll go back to all the merged PR's and add it. The tool will need to find the latest appropriate comment and use that text if found, rather than the PR title if on.

At this stage I think we should keep it simple and just have the comment match a regex like ^release_notes:\W*(?<notes>.*)$ (using SingleLine option and taking the match group "notes")

[github-actions]

👋 Hey Friends, this issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Please add the pinned label if you feel that this issue needs to remain open/active. Thank you for your contributions and help in keeping things tidy!