GitHub Enterprise provider compatibility

[mengesb]

With the recent provider update to 2.5.0 we experienced an issue with our organization management using this provider.

hashicorp/terraform-provider-github@v2.4.1...v2.5.0

Included in this, which seems more likely, was an update in google/go-github :
google/go-github@v29.0.2...v29.0.3

With the upgrade in go-github it looks like the APIs surrounding teams got chagned. In at least one case, I see a deprecation message in the documentation. The result is that all github_team resources are thought to not exist, so it wants to re-create them (unsure if creation would even succeed at this point). Additionally github_team_repository resources, which rely on the github_team relationship are also affected and these APIs have changed as well.

While I imagine that this isn't so much as a bug as it is a compatibility issue, I think that GitHub Enterprise users need messaging regarding compatibility with this provider. As it stands, it looks like GitHub Enterprise v2.20.3 is compatible only up to v2.4.1 of this provider.

Terraform Version

0.12.24

Affected Resource(s)

• github_team
• github_team_repository

Note: When using GitHub Enterprise (tested with v2.20.3)

Terraform Configuration Files

https://www.terraform.io/docs/providers/github/r/team_repository.html

# Add a repository to the team
resource "github_team" "some_team" {
  name        = "SomeTeam"
  description = "Some cool team"
}

resource "github_repository" "some_repo" {
  name = "some-repo"
}

resource "github_team_repository" "some_team_repo" {
  team_id    = "${github_team.some_team.id}"
  repository = "${github_repository.some_repo.name}"
  permission = "pull"
}

Debug Output

https://gist.github.com/mengesb/811cadacde485dba7d36807ff9ae42b5

Panic Output

N/A

Expected Behavior

Teams and team repositories shouldn't change after provider update, or there should be some documentation about provider compatibility in the case of GitHub Enterprise vs GitHub

Actual Behavior

When a plan is ran with provider 2.4.x, and generates resources, an update to provider version 2.5.x results in it re-generating the same team and team_repository resources. If there are members of the team - those trigger regeneration due to vertex changes

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply with v2.4.1
2. Update provider to 2.5.0
3. terraform apply

Important Factoids

Github Enterprise v2.20.3

https://developer.github.com/enterprise/2.20/v3/teams/#get-team
vs
https://developer.github.com/v3/teams/#get-team-by-name
https://developer.github.com/v3/teams/#get-team-legacy (deprecated - compat w/2.20.3)

https://developer.github.com/enterprise/2.20/v3/teams/#check-if-a-team-manages-a-repository
vs
https://developer.github.com/v3/teams/#list-team-repos

References

• hashicorp/terraform-provider-github@v2.4.1...v2.5.0
• google/go-github@v29.0.2...v29.0.3
• Updated to go-github v29.0.3 #369

[anGie44]

I'm happy to look into this - wondering if it makes sense to bring back the github.NewEnterpriseClient to support backwards compatibility. I saw the forthcoming graphql implementation(https://github.com/terraform-providers/terraform-provider-github/pull/331) makes use of this client -- @jcudit will this issue be prioritized after that is in place?

[jcudit]

Yes, I would assist in merging if an implementation appeared after the incremental changes towards GraphQL support are merged.

[patrickmarabeas]

Can confirm issues in GHE. Issues also present when building current master.

[kumar0608]

getting the same problem any fix available ?
Error: Provider produced inconsistent result after apply

When applying changes to github_team.org1_team, provider
"registry.terraform.io/-/github" produced an unexpected new value for was
present, but now absent.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

[analytically]

+1 seeing this as well

[mengesb]

Seems like GHE is falling behind in versions to use this provider... v2.4.1 was the last compatible release.

Any update on getting compatibility aligned between OSS Github and GHE?

[jcudit]

@mengesb my next step on this front is to expand our acceptance tests to run against a current version of GHES. This will be a workflow improvement to unblock alignment and avoid drift between the two use cases of this project.

Hoping to perform a cursory pass on this in the next two weeks and report back with better time estimates. Thank you for keeping this issue active 😄

[jcudit]

Quick update here: we have a version 2.21 of GitHub Enterprise running and publicly accessible.

Next steps are to create a second acceptance testing job that points at our GHES instance. We will then be able to run tests for various configurations of this project (anonymous, organization, individual modes) once this pair (https://github.com/terraform-providers/terraform-provider-github/pull/512, https://github.com/terraform-providers/terraform-provider-github/pull/506) of deprecations ship.

Again, aiming to post an update here in a couple of weeks after we get v3.0.0 released.

[jcudit]

Another update here. A couple of PRs are up for review that will improve our testing around GHES use cases going forward.

• https://github.com/terraform-providers/terraform-provider-github/pull/521 is some general cleanup around provider configuration
• https://github.com/terraform-providers/terraform-provider-github/pull/523 breaks out our acceptance test runs into GHES and github.com use cases

[r-bennett]

@jcudit any more updates on this one now that 3.0.0 has landed? We're managing a GHE instance with 2.4.1 and it would be great to be able to get some of the features that have landed since.

Is there a clear plan of what's still to be done? We may have some capacity on our end to help out a bit.

It sounds like go-github don't have an interest in supporting GHE on their end google/go-github#1524

[r-bennett]

I asked GitHub support about this as well - the API I've seen causing problems is /organizations/:org_id/team/:team_id/members, and it looks like that is added in v2.21 of GHE https://docs.github.com/en/enterprise/2.21/user/rest/reference/teams (well, the /organizations/:org_id/team/:team_id/... endpoints are added in 2.21 although /organizations/:org_id/team/:team_id/members isn't actually mentioned, but nor is it mentioned in the github.com docs apart from the old-format ones at https://developer.github.com/v3/teams/members/#list-team-members). So hopefully we may just need to upgrade GHE versions to get this all working again.

I see you did mention above that you had a 2.21 version of GHE set up for your acceptance tests, is that all up and passing?

[jcudit]

👋 @r-bennett. I've got this one queued up to start working next week and to be released as part of v3.1.0. The blocker at the moment is the transfer of this repository to the github organization. Once the transfer completes, we will be able to configure secrets and attach the GHES testing suite.

[mengesb]

Is there an angle that some of us GHES users should be taking with Microsoft/GitHub now? I really don't want to see two terraform providers that maintain the API inconsistencies between an on-prem and cloud solution. I realize that a vast majority of users may likely be github.com users instead of GHES users... but it seems that paying for an on-prem solution makes you a second class citizen/consumer.

[jcudit]

My goal is to get GitHub.com and enterprise on equal footing with this provider. The upcoming test suite will allow us to write tests spanning both deployment types, which should help us avoid releasing changes introduce incompatibilities.

two terraform providers that maintain the API inconsistencies between an on-prem and cloud

I will have to do more reading into how other projects handle this situation. Thanks for highlighting this and any suggestions are welcome.

[mengesb]

So I can confirm that with regard to #536 which is related to this, we updated to GHES 2.21.x and now my module tests pass for setting delete_branch_on_merge ; I'll move on to checking out the team API endpoints, unless that's known to be still broken

[r-bennett]

The teams API should be ok on 2.21 according to the github docs, would be great to verify that though

[analytically]

I've just migrated from 2.9.2 (GH 2.21.5) to 3.0.0 and it seems like the user API's are failing again (getting 406 and "This resource can only be used in the context of an organization, "redacted" is a user.").

[r-bennett]

@analytically is that something to do with the changes around provider configuration with owner vs organization (https://www.terraform.io/docs/providers/github/index.html)? Not sure when that change came in.

From your comment, am I right in reading that 2.9.2 works with GHES 2.21.5?

[analytically]

@r-bennett could be, and yes, 2.9.2 works fine for us with GHES 2.21.5

[mengesb]

I'm also still using 2.9.2 with GHES 2.21.7. Checking teams today/tomorrow

[docwhat]

I can vouch that both /organizations/:org_id/team/:team_id/members and /orgs/:org/teams/:team_slug/members works with 2.21.7

In addition, /organizations/:org_id/team/:team_id has the attribute members_url which points at /organizations/:org_id/team/:team_id/members{/member} in 2.21.7

[colltoaction]

To use the old version you should specify:

terraform {
  required_providers {
    github = {
      # see https://github.com/terraform-providers/terraform-provider-github/issues/404
      version = "2.4.1"
    }
  }
}

[yermulnik]

Is pinning to v2.4.1 is the best solution for GHE at the moment?

[r-bennett]

Is pinning to v2.4.1 is the best solution for GHE at the moment?

You can use 3.0.0 now if your GHES version is reasonably up to date (2.21+ I think), but there's a separate issue affecting versions after that #716