CI: check for changes with semver violations

[yanns]

In our CI, we could add https://github.com/obi1kenobi/cargo-semver-checks to check for backwards incompatible changes.

Is it something that was already discussed?

[jplatte]

I don't think this was previously discussed. How do you imagine this working? Would it run for every PR and just post a summary of breaking changes? Would it run whenever a PR touches the version field of one of the crate, to ensure non-breaking bumps are really non-breaking? Something rlsr?

[yanns]

From my experience with other open source projects where we use a similar tooling in a different language, we use this check on each PR. This gives us early feedback when we break the compatibility.

Having said that, I have no experience with https://github.com/obi1kenobi/cargo-semver-checks.
Maybe doing that on each PR would be too costly.
-> We could try this out, check if it's working, how long it takes.

As an alternative, if it's too costly, we could add this check to do before each release.
AFAIU, the semver checks will be integrated in cargo publish at the time. So doing that before a publishing is just reflecting what will become the norm later.

[yanns]

I see that https://github.com/obi1kenobi/cargo-semver-checks is using that before publishing: https://github.com/obi1kenobi/cargo-semver-checks/blob/2615b5db141012cef8219013cc2c4aaa5151e15d/.github/workflows/ci.yml#L1884-L1903

[jplatte]

I don't think it should be very costly. Really we just need to ensure to then have every author of a PR with breaking changes update the changelog in response of the check, so we only need to check the changelog for breaking changes when making a release.

[jplatte]

What I would like for the PR thing:

• use github artifacts to upload rustdoc bundles from main that live for a while (if possible to have them expire?) so multiple pushes to a PR branch don't end up recomputing the same baseline to check the latest API against
• make sure that when a PR branch is pushed multiple times, this does not result in more and more bot comments on the PR (assuming we have the check post the breaking changes as a comment); updating one comment should be possible