Support changelogs

[wbamberg]

This is the fourth of a series of PRs to ship the new spec platform, as part of #2906.

It adds support for rendering changelogs.

This is all tied up with how we want to handle spec versioning, and what's implemented here is based on what was discussed in #2877, but that thread is a bit hard to follow, so I'll summarise it again here.

---

Changelogs and versions

The main branch of the matrix-doc repository is always the current unstable version of the spec. Each major.minor release of the spec is a separate tagged branch. Each patch version of the spec is an incremental change in top of its existing major.minor branch. That is, we use separate branches for major.minor versions but not for patch versions.

There is metadata in the repo telling it which version of the spec it represents, and the site uses this for things like "unstable" warnings as well as to build the changelog. Specifically, we have the following properties in config.toml:

• status: this can have one of three values:
  • "unstable": this is the unstable version of the spec
  • "current": this is the most recent released version of the spec
  • "historical": this is an older release of the spec
• current_version_url: the place we put the current version of the spec (e.g. "https://spec.matrix.org/latest")
• major_version: the major version number for this version of the spec (omitted if status is "unstable")
• minor_version: the minor version number for this version of the spec (omitted if status is "unstable")
• patch_version: the patch version number for this version of the spec (omitted if status is "unstable")

The repo also contains a directory /changelogs.

/changelogs in the unstable branch

In the main ("unstable") branch, /changelogs is exactly the way it was before: a directory for each API, and under each of these directories a pyproject.toml describing the towncrier format and a newsfragments directory containing changes in towncrier format:

/changelogs
    /application_service   (other API directories also have this internal structure)
        /newsfragments
        pyproject.toml
    /client_server
    /identity_service
    /push_gateway
    /server_server

/changelogs in release branches

In a release branch, /changelogs is expected to contain a "releases" directory which is expected to contain one directory for the initial major.minor release for which we created this branch, and one for each patch release which has been added. Inside each of these directories is:

• a "release.yaml" file which includes the date of this release and its Git tag
• a directory for each API, containing changes included for that API in this release.

For example, if we've made two patches, it would look like:

/changelogs
    /releases
        /0                         <- initial major.minor release
            /application_service   (other API directories also have this internal structure)
                /newsfragments
                pyproject.toml
            /client_server
            /identity_service
            /push_gateway
            /server_server
            release.yaml
        /1                         <- first patch
            /application_service   (other API directories also have this internal structure)
                /newsfragments
                pyproject.toml
            /client_server
            /identity_service
            /push_gateway
            /server_server
            release.yaml
        /2                          <- second patch
            /application_service   (other API directories also have this internal structure)
                /newsfragments
                pyproject.toml
            /client_server
            /identity_service
            /push_gateway
            /server_server
            release.yaml

This PR

This PR assumes we will have a setup like the above. So it includes:

• a new content file, /content/changelog.md
• two new templates to help render changelogs, under /layouts/shortcodes/changelog.
  • changelog-changes.html: this renders the changelog entries differently depending on whether we are in the unstable branch or a released branch.
  • changelog-description.html: this renders different boilerplate text depending on whether we are in the unstable branch or a released branch.

Because this PR is targeting the main branch, of course you'll see the "unstable" version of the changelog. I made another branch: https://github.com/wbamberg/matrix-doc/tree/demonstrate-release-branch to show what the thing would look like in a release branch.

Finally, this PR changes the changelog entry descriptions from RST to Markdown, which is a fix for #2919.

It's probably worth mentioning that we don't have to buy into the whole versioning process outlined above until we actually want to make a release. So we can merge this change and still redo the versioning process. But I've gone into it in detail here just to explain what this PR is trying to support.

[anoadragon453]

Code changes looks good to me! Was agreement met in #2877 (and is that a blocker here at all?). I saw that the releases dropdown was punted, though not sure about @turt2live's suggestion for being able to support -beta etc?

[wbamberg]

Code changes looks good to me! Was agreement met in #2877 (and is that a blocker here at all?). I saw that the releases dropdown was punted, though not sure about @turt2live's suggestion for being able to support -beta etc?

I started writing a response to #2992 (review), mostly along the lines of "but how will we represent patch versions then?" then went and looked at https://github.com/matrix-org/matrix-doc/blob/master/proposals/2844-global-versioning.md#full-matrix-version-grammar and realised that we are now not representing patch versions at all :).

So I've pushed an update that represents versions using an opaque version.number, and that renders changelogs for a released version without any idea of patches. It's quite a bit simpler like this.

[turt2live]

This looks great - thank you! It's good to see the Matrix versioning scheme doing its thing :D

[KitsuneRal]

Just to double-check: that means we do not expose patch-level changes in the changelog at all, and the only way to understand those changes will be by looking into git log contents, correct?

[wbamberg]

Just to double-check: that means we do not expose patch-level changes in the changelog at all, and the only way to understand those changes will be by looking into git log contents, correct?

I'm not sure what "patch-level changes" means here. All changes listed in the newsfragments files in any given branch would be rendered in the log, but they wouldn't be grouped by patch version, because according to my reading of https://github.com/matrix-org/matrix-doc/blob/master/proposals/2844-global-versioning.md#full-matrix-version-grammar, patch versions wouldn't exist any longer.

I think in general this PR rests on a model of how we make releases and what we decide to put inside a release, and I might have misunderstood this!

[KitsuneRal]

Well, as the mentioned section of #2844 describes, patch releases only contain insignificant changes (typos and such) but don't entirely disappear. My double-checking was just to make it explicit that we don't care about attributing those insignificant changes to particular patch releases in the rendered spec (which, personally, is totally fine by me).

P.S. Thanks for all the spec goodness you brought in!

[turt2live]

We probably don't need to represent patch levels in the spec docs at all - they might show up in the git version history, but at the spec level fixing typos isn't really worth showing the .2 at the end. The release process will probably be modified to have a long-running branch (release-1.1) with git tags on top (v1.1.2) to represent historical references, but generally the patch version won't be exposed any further.

[anoadragon453]

Latest change looks good to me. Is this PR waiting on anything else before being merged?