WIP: Infra: Try running CI for merge queues

[xFrednet]

Testing merge queues?

[xFrednet]

I believe the merge queue should be working now, but I want to figure out how to have a bors try like comment or option, before I enable them by default and merge this PR. I'll leave this as a todo for weekend me :D

[Veetaha]

I believe there isn't the direct replacement for bors try. I am also considering to abandon bors in favour of merge queues at my work.

Here are the results of my research (although in my case I didn't use bors try very often):

There have been several problems with bors, and it has its own disadvantages.

• (This applies if you run a self-hosted bors instance). Bors requires some low-effort maintenance when it has some internal glitches. Usually restarting the heroku server helped. One time we also needed to maintain its Postgres database when bors went over the free Postgres tier. We also never updated bors (it still runs the version of code as of several years ago and works).
• Bors requires additional configuration in bors.toml file within the repo that partially duplicates the list of required jobs that we have on CI with the CI workflows themselves
• Bors requires us to issue commands like bors r+ on PRs
• Bors "closes" the PRs and it renames PRs with the label [Merged by Bors] so PRs appear in red "closed" status not in purple "merged" status, which makes for an unordinary github experience (we use squash-and-merge strategy)

Github Merge queues are the new native Github feature that is intended to replace bors. And it doesn't have problems described higher, however it may have own problems. I know of at least one thing that could pose problems

Our CI that we run with bors assumes it runs exclusively (one at a time), while with Github's merge queues PRs are not batched into groups like with bors (for the docs, see here, also in the FAQ from mergify bot), all PRs get their CIs to run in parallel.

For CI jobs that require exclusive access to some test environment you'll need to use a workaround:

• Use native github locking with concurrency: 1 setting on them at the job level. If these checks are not on the critical path of merging a PR, it should be affordable.
• Move such checks out of merge-CI-path, making them async and not blocking merges (e.g. run them on master), but async checks get unnoticed by developers, and don't make it clear what and who broke them providing late feedback

bors try replacement (one option)

You may create a manual github workflow where you could use a branch name as an input. In this case if you want to try something you'll trigger this workflow via the github UI specifying the name of your branch.

Basically consider Github Actions CI as your own service, and the manual workflows as the API to invoke something useful in your personal CI service.

[xFrednet]

Hey @Veetaha, thank you for the detailed comment. My main motivation for migrating is the deprecation of bors. The user forum said that the public instance will most likely continue working until merge queues have been stabilized.

The problems of bors you listed are definitely valid. It would have been cool if GH added options to fix these somehow. I believe that Rust will stay with their @bors bot as it also contains permissions and some PR state information. That might be interesting if you want to keep bors even if bors-ng is deprecated. Though it has the same problems.

Our CI that we run with bors assumes it runs exclusively (one at a time), while with Github's merge queues PRs are not batched into groups like with bors (for the docs, see here, also in the FAQ from mergify bot), all PRs get their CIs to run in parallel.

That's really good to know. This should be fine for Marker, but it's somewhat unexpected.

You may create a manual github workflow where you could use a branch name as an input. In this case if you want to try something you'll trigger this workflow via the github UI specifying the name of your branch.

This sounds like a good solution! Do you have an example for such a workflow? My CI wizard skills are sadly still developing, this already gives me some things to search for. Thank you!

[Veetaha]

As an example, we have a crates release tool, which does publishing on demand. For example: you changed something in a repo and want to publish the new version of the crate to the crates registry, then you can use a manual github workflow to do that:

By default there is a branch selection parameter that will be used by github to decide what version of the CI .yml file to use to run the workflow, and what branch the checkout action will checkout on CI.

An example template for that may be like this:

name: try-my-pr
on:
  workflow_dispatch:
    inputs:
      input_name:
        description: 'There could be inputs to your workflow if you need any (required or optional)'
        required: true

# Run the CI jobs that you want.
# In my example we bump the versions in git, push an automated commit
# then run an analogue of `cargo publish`
jobs:

[xFrednet]

Nice, this looks exactly like the thing I was looking for ❤️

[Veetaha]

This may be closed in favor of #222