How to ensure Github merge queue does not allow failed checks into master

[ncjones]

Select Topic Area

Question

Body

We are experimenting with the merge queue and having great success. The problem is that sometimes the merge is allowed when there are still failing checks and I'm not sure how to prevent this.

In the following example the sequence of events was like:

1. Checks were failing
2. Amended commit was force pushed and "merge when ready" was enabled.
3. PR was approved
4. Branch was merged while checks for amended commit was still inprogress and led to failing tests on master branch.

[b8zeek]

Hey Nathan. 👋🏼

I think this is what you need, pull requests to the protected branch able to be merged if all the required checks pass and the branch is up to date with the base branch:

Your repository ➡️ Settings ➡️ Branches ➡️ Branch protection rules ➡️ Add rule ➡️ In the Branch name pattern, enter the name of the branch you want to protect (master, main, etc.) ➡️
In Protect matching branches, check Require status checks to pass before merging ➡️ Check Require branches to be up to date before merging ➡️ Under Status checks found in the last week for this repository check the boxes for the checks you want to require ➡️ Create.

Hopes this solves your issue. 🍺

🍀

[pete-watters]

I agree with @kyranjamie . My definition of Merge when ready would be under the assumption that to be ready all checks have passed

[SphericalPotatoInVacuum]

What happens to the PRs that don't trigger the checks? For example, if I have a check set up to build code, which is not triggered by pull requests modifying READMEs. Will those PRs forever be unmergeable?

[air3ijai]

@ncjones

Some of my pipelines use the job matrix so might be painful keeping this list up to date.

We may create a martix-status job which will report its status when whole matrix passed its checks and then use just that job as a requirement, but we lose some visibility.

@SphericalPotatoInVacuum

What happens to the PRs that don't trigger the checks?
Will those PRs forever be unmergeable?

Yes - A pull request that requires those checks to be successful will be blocked from merging and some options described in Handling skipped but required checks.

[yeongrokgim]

This feels VERY confusing as this information is not available anywhere docs.github.com , effectively preventing working configuration to be made.

PLEASE update this page ( https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue#configuring-continuous-integration-ci-workflows-for-merge-queues ) to have something simillar to "Merge Queue will use status names declared in Require branches to be up to date before merging", at least.

[air3ijai]

It is a known issue - Pull Request Merge Queue Public Beta Feedback #46757.

Top feature requests
- API to enqueue/dequeue and view queue
- Require different status checks for a PR's entry into the queue vs. for merging by the queue

PR created - Add additional clarification about the merge queue #31844.

[ncjones]

create a matrix-status job which will report its status when whole matrix passed its checks and then use just that job as a requirement

This is the key. In fact, I am now using a general "status-checks" job which depends on everything that is required. This also enables having different required checks for merge queue and PR pipelines. It also allows non-admins to modify the required checks which is super helpful.

[ncjones]

My general "status-checks" job looks like:

  required-checks:
    needs: [ lint, test, cucumber ]
    if: always()
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: echo '${{ toJSON(needs) }}' | jq -e 'to_entries | all(.value.result == "success")'

If we don't explicitly check for successful dependent jobs then the checks job will be skipped. Skipped jobs are considered passed when github decides to merge from the merge queue.