Automate NEWS generation

[matthijskooijman]

The NEWS has a high-level changelog, which is currently manually written at release time (or sometimes in chunks between releases).

I would like to investigate a scheme were changelog entries are written along with the code changes (i.e. as part of the pull request). An obvious way would be to just modify the NEWS file in a commit inside the PR, but that is going to cause a lot of conflicts, and makes it harder to include the PR number in the changelog).

An obvious way would be to just use the git log to generate the changelog, but IMHO that would produce a too finegrained changelog. e.g. for NEWS it useful to know a bug was fixed, but there is no need to know that three refactoring commits were needed before that. Also, if you want the git log, you can just use git or github to see it.

So ideally, you would explicitly add a message to each PR description (or selected commit messages), which are then automatically generated into a NEWS file at releasetime with some script (probably with some chance to do manual post-processing if needed).

It seems there are quite some tools for this. Reading through this article, Changelox looked promising, but their website is giving an SSL failure right now. The github-changelog-generator project also has a list of competitors that might prove useful.

For the upcoming 3.0.2 release, the changelog is simple enough to just do it manually as before, but I'd like to see if we can set up something like this for the next release cycle.

[matthijskooijman]

I just remembered I recently saw some other repository where they collected changelog entries in separate files in the repository, to be removed and merged into a NEWS file on release. Putting things in separate files, rather than in a single file helps to prevent git conflicts, so that's also a thing to consider (but it does suffer from the same problems with not being to include PR numbers easily).

This also made me realize one more requirement: It should be possible to update changelog entries between the moment they are created and the release. You might find a typo, or want to add some additional information later. You can of course do this at release time, but you are likely to forget that unless we have a mechanism to remember these changes. It would be better if such a change could be made immediately, e.g. by editing the PR description or editing a file in the repo. This is probably not possible when putting changelog entries in commit messages.

[matthijskooijman]

Then, it might be useful if there was some automatic categorization of changelog entries (e.g. to split off user-visible changes, internal improvements, build system changes, etc.). Also, this could allow marking (breaking) API changes (e.g. in dbus, python imports or commandline) separately and having those in a separate list in the changelog (this might mean that a single change needs to appear in two lists as well, so that should also be possible).

[mwilck]

I just remembered I recently saw some other repository where they collected changelog entries in separate files in the repository
Then, it might be useful if there was some automatic categorization of changelog entries

All fine, but IMHO over-engineered at this time for hamster. We don't expect to move forward at high pace and loosing oversight over the changelog entries, do we?

Assuming that PRs are self-contained and focused on certain issues or enhancements, I would simply require PRs to update the NEWS file when they are merged.

[matthijskooijman]

All fine, but IMHO over-engineered at this time for hamster. We don't expect to move forward at high pace and loosing oversight over the changelog entries, do we?

Yeah, that's why I stopped here and focused on actually releasing instead. However, I do think making releases easier by more automation might be helpful. And I might want to apply the same approach to some other repositories I maintain as well and all this can be ported over to the next-gen-hamster repos as well if that speeds up again at some point.

Assuming that PRs are self-contained and focused on certain issues or enhancements, I would simply require PRs to update the NEWS file when they are merged.

Yeah, that's certainly a simple and good policy we can just adopt right away, inded.

[matthijskooijman]

Just noticed another project that uses changelog entries in separate files, with a tool (custom-written go, it seems) to generate a changelog from them, which is restic. Might be useful as inspiration if we ever want to go this route. See:

• https://github.com/restic/restic/tree/master/changelog
• https://github.com/restic/restic/blob/master/helpers/prepare-release/main.go