team-reviewers option doesn't work anymore

[aswen]

Subject of the issue

Since 5 days ago the option to request a review from any of the owners, or later, the members of a team doesn't work anymore with the following error:

Reviews may only be requested from collaborators. One or more of the teams you specified is not a collaborator of the company-name/repository-name repository.

Steps to reproduce

Create a PR with:

      - name: Create PR
        id: create-pr
        if: ${{ steps.prepare-pr.outputs.create_pr == 'True' }}
        uses: peter-evans/create-pull-request@v4
        with:
          path: reponame-repo
          add-paths: node/${{ inputs.hostname }}
          token: ${{ env.GITHUB_REPO_TOKEN }}
          branch: EOL_${{ inputs.hostname }}
          commit-message: Remove ${{ inputs.hostname }}
          title: Remove ${{ inputs.hostname }}
          body: ${{ env.pull_request_body }}
          assignees: ${{ github.actor }}
          team-reviewers: owners
          labels: Prepare for removal
          draft: ${{ steps.prepare-pr.outputs.remnants_warning }}

Later, I tried with the same as above, but:

          team-reviewers: company-name/team-name

But that failed with same error. I also tried other things like team-reviewers: team-name and:

          team-reviewers: |
            company-name/team-name

All of them failed.

[peter-evans]

Hi @aswen

No changes have been made to the action recently that could cause this. If it was working until five days ago then I think some changes have been made to teams for your org. Perhaps check with your GitHub org admin to find out what changed.

The other possibility is that the permissions of your PAT contained in GITHUB_REPO_TOKEN have changed.

[harpener]

Hi guys, we have the same issue. I figured it is some change in Github, that it does not recognize private teams anymore. :-/

[peter-evans]

Ok, so it sounds like GitHub may have made some permissions related changes.

Could you try giving the PAT you are using with the action the org:read scope.

[harpener]

We are using the default GITHUB_TOKEN automatically generated for the workflow and its default permissions.

[peter-evans]

We are using the default GITHUB_TOKEN automatically generated for the workflow and its default permissions.

It's possible that GitHub closed a security hole where you could "find" private teams. I would suggest trying to use a PAT, issued on a user that belongs to the org.

[harpener]

I also tried

jobs:
    <job>:
        permissions:
            org: read

and I got The workflow is not valid. ... Unexpected value 'org'.
We'd rather avoid PAT, it's not that important to automatically request review from team when we would have to use and manage PAT of someone, especially with expiration time.
Thanks for the advice though. :-)

[alfetopito]

I have the same issue, except that my repo nor team are private.
I wonder if I mis-cofigured something or this is just not working?

The team was added to the repo and has maintain role.

Like harpener above, I haven't tried and rather not go the PAT route.

[matthiasthomas]

We're experiencing the same issue with some of our actions, but they will sometimes go through. Looking at our most recent 25 runs, it worked 10 times.

When it doesn't go through the message is the same as mentioned here: Reviews may only be requested from collaborators. One or more of the teams you specified is not a collaborator of the company-name/repository-name repository.

Given that it sometimes works I doubt it's a permission change from GH 🤔

[peter-evans]

Given that it sometimes works I doubt it's a permission change from GH

I've not had time to look into this issue in depth, but I'm fairly confident that the cause is either a GitHub bug, or they have deliberately changed the permissions model. The latter of those possibilities is far more likely. I've experienced them making permissions model changes a few times over the last couple of years.

[peter-evans]

@harpener What you tried will not work because the default GITHUB_TOKEN cannot be given the org scope. See here. You can only give that scope to a PAT.

If anyone tries the possible solution I commented here, please let me know if it worked.

Could you try giving the PAT you are using with the action the org:read scope.

I understand that some don't want to use a PAT, but you might have no other choice if GitHub has changed the permissions model to require org:read in order to "see" and request reviews from private teams.

[davejrt]

Chiming in to say we've tried with a PAT, and we're still seeing the same issue. Using the same token with the gh pr create command works fine....but is failing on the action.

[peter-evans]

Chiming in to say we've tried with a PAT, and we're still seeing the same issue. Using the same token with the gh pr create command works fine....but is failing on the action.

gh is using the GraphQL API, while this action is using the REST API. So if it works with gh then that makes me think this is a bug with the REST API. If we can reproduce the issue in a clear way and show it works with gh then we can make a support case with GitHub. @davejrt Would you be willing to share code or a link to your workflows that demonstrate this?

[davejrt]

@peter-evans you can find the code here and the logs here

I've tried multiple PAT's, including my own which is an admin for our org and works with the gh cli. I created a new PAT with no luck

[peter-evans]

@davejrt I've tried to reproduce the issue and your working solution with gh.

I created a secret team in an org called secret-team and confirmed that the action cannot assign it as a team reviewer, even when the PAT has admin:org scope. But I also cannot get it to work with gh either.

You can see my tests here:
https://github.com/create-pull-request/team-reviewers-test/actions/runs/4551578136/jobs/8025833797#step:5:8

gh returns GraphQL: Could not resolve to a User with the login of 'secret-team'.

Additionally, when you read the docs about secret teams it doesn't make any sense that you could add a secret team as reviewers to a pull request. I don't even seem to be able to do that manually through the UI.

If I've misunderstood please let me know, but it seems to me that if this was working before it was probably a mistake on GitHub's part to allow it, and now it's been fixed.

[harpener]

@peter-evans I should mention that in my case the team is not secret, but visible to everyone in our organization. Only the repository is private to the outside world.

[peter-evans]

Thanks @harpener, I misunderstood the conditions under which this was occurring. I've got it now so I'll test some more.

[peter-evans]

Perhaps this is the cause. Did GitHub stop allowing requesting team reviewers for private repositories under the GitHub Free billing plans recently?

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/requesting-a-pull-request-review

I can't find any public announcement about this if they did change it recently.

[harpener]

Good to know. But I am told we are using Github Team subscription: https://docs.github.com/en/get-started/learning-about-github/githubs-products#github-team

[davejrt]

Here is the workflow I used with gh to create a PR with a team reviewer. Our repo is public, we have a github teams plan. I created a new token only with read:org and repo to make sure my admin privileges weren't masking the issue.

? What account do you want to log into? GitHub.com
? You're already logged into github.com. Do you want to re-authenticate? Yes
? What is your preferred protocol for Git operations? SSH
? Upload your SSH public key to your GitHub account? Skip
? How would you like to authenticate GitHub CLI? Paste an authentication token
Tip: you can generate a Personal Access Token here https://github.com/settings/tokens
The minimum required scopes are 'repo', 'read:org'.
? Paste your authentication token: ****************************************
- gh config set -h github.com git_protocol ssh
✓ Configured git protocol
✓ Logged in as davejrt
➜  sourcegraph git:(dt/action_test) gh pr create --title "dave testing actions" --body "testing team reviewers"  -r "sourcegraph/dev-experience"

Creating pull request for dt/action_test into main in sourcegraph/sourcegraph

https://github.com/sourcegraph/sourcegraph/pull/50119
➜  sourcegraph git:(dt/action_test)

[peter-evans]

I've opened a GitHub support case because something is clearly wrong with the API and it seems difficult to reproduce and debug.

In the meantime, I'll look into using the GraphQL API to request team reviewers, since that appears to work.

[harpener]

For the record, I tried it with default GITHUB_TOKEN and received following error:

Create or update the pull request
  Attempting creation of pull request
  Created pull request ...
  Error: Request failed due to following response errors:
   - Resource not accessible by integration

The PR was created, but review was not requested from given team.

[peter-evans]

I received a response to the support ticket. This is the explanation given.

I took a look and it does seem like we made changes to the API endpoint about the time the report was first made in the issue linked(Feb 15).

Before then, we return a 201 response also when the requested reviewer or team is non-existent. This response is incorrect, as we should be returning a 422 -- the change on the endpoint was targeted at fixing that behavior.

So what that suggests is, the users who are experiencing this issue are trying to request a review from a team that is "non-existent". It also suggests that users who suddenly had this issue in their workflow never had team-reviewers working in the first place.

My guess as for why a team would be "non-existent" for the majority of cases affected by this is that the team is being specified as <org>/<team>, which does not work. It should be specified as <team> only. The REST API for requesting team reviews only supports the team slug, not including the org.

Please specify teams like this:

      - name: Create Pull Request
        uses: peter-evans/create-pull-request@v4
        with:
          token: ${{ secrets.PAT }}
          team-reviewers: |
            dev-experience
            developers
            qa-team

I might make an update to the action to remove the leading <org>/ from any team names supplied, because it's a very easy mistake to make and I've also not made it clear in the documentation.

[harpener]

My guess as for why a team would be "non-existent" for the majority of cases affected by this is that the team is being specified as /, which does not work. It should be specified as  only. The REST API for requesting team reviews only supports the team slug, not including the org.

I remember testing this as well, it returned a different kind of error about actually non-existing object. The organization prefix was necessary for us before.

[harpener]

The error looked like the one in this issue: #155 (comment)

[peter-evans]

@harpener You are getting that error because you are using the default GITHUB_TOKEN. I'm not sure if the default token worked for requesting reviews from teams before, but it definitely does not work anymore since GitHub's update. And I'm fairly sure that is the correct behaviour. The token must have at least repo scope to be able to request a review. If it was working before without that then I believe it was a mistake which has now been corrected.

Summary:

• Specify the teams as team slugs without the leading <org>/, as I described here.
• Use a PAT with repo scope.

If you don't specify the teams correctly without the leading <org>/, then you will see the following error:

Reviews may only be requested from collaborators. One or more of the teams you specified is not a collaborator of the org/repo repository.

If you don't use a PAT with repo scope then you will see the following error:

Validation Failed: "Could not resolve to a node with the global id of 'xxxxxxxxxxxxx'."

[peter-evans]

I've made a couple of changes that I've released in v5.0.0 / v5.

• Improved the handling of error Validation Failed: "Could not resolve to a node with the global id of 'xxxxxxxxxxxxx'." to provide a clearer reason for the failure.
• Strip the leading <org>/ from teams if supplied.