Add Terraform Cloud/Enterprise credential support #959

Valarissa · 2021-01-29T20:01:28Z

This PR seeks to introduce support for the upcoming Terraform Cloud Secrets Engine. It will enable users to create and manage the Terraform Secrets engine config, any associated roles, and generate credentials from those roles.

Community Note

Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
Please do not leave "+1" comments, they generate extra noise for pull request followers and do not help prioritize the request

Relates OR Closes #0000

Release note for CHANGELOG:

Adding support for Terraform Cloud/Enterprise token generation for users, teams, and orgs.

Output from acceptance testing:

/usr/local/Cellar/go/1.15.5/libexec/bin/go test -json ./... -run .*TerraformCloud.*
?   	github.com/hashicorp/terraform-provider-vault	[no test files]
?   	github.com/hashicorp/terraform-provider-vault/cmd/coverage	[no test files]
?   	github.com/hashicorp/terraform-provider-vault/cmd/generate	[no test files]
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-vault/codegen	0.203s [no tests to run]
?   	github.com/hashicorp/terraform-provider-vault/generated	[no test files]
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-vault/util	0.488s [no tests to run]
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-vault/generated/resources/transform/transformation	0.445s [no tests to run]
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-vault/generated/resources/transform/role	0.691s [no tests to run]
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-vault/generated/datasources/transform/decode	0.951s [no tests to run]
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-vault/generated/resources/transform/alphabet	1.198s [no tests to run]
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-vault/generated/datasources/transform/encode	1.427s [no tests to run]
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-vault/generated/resources/transform/template	1.628s [no tests to run]
?   	github.com/hashicorp/terraform-provider-vault/schema	[no test files]
=== RUN   TestAccDataSourceTerraformCloudAccessCredentialsOrganizationClientBasic
--- PASS: TestAccDataSourceTerraformCloudAccessCredentialsOrganizationClientBasic (1.48s)
=== RUN   TestAccDataSourceTerraformCloudAccessCredentialsTeamClientBasic
--- PASS: TestAccDataSourceTerraformCloudAccessCredentialsTeamClientBasic (1.42s)
=== RUN   TestAccDataSourceTerraformCloudAccessCredentialsUserBasic
--- PASS: TestAccDataSourceTerraformCloudAccessCredentialsUserBasic (2.79s)
=== RUN   TestTerraformCloudSecretBackend
--- PASS: TestTerraformCloudSecretBackend (0.46s)
=== RUN   TestTerraformCloudSecretRole
--- PASS: TestTerraformCloudSecretRole (2.02s)
=== RUN   TestTerraformCloudSecretBackendRoleNameFromPath
--- PASS: TestTerraformCloudSecretBackendRoleNameFromPath (0.00s)
=== RUN   TestTerraformCloudSecretBackendRoleBackendFromPath
--- PASS: TestTerraformCloudSecretBackendRoleBackendFromPath (0.00s)
PASS
ok  	github.com/hashicorp/terraform-provider-vault/vault	10.017s

Process finished with exit code 0

Also updated tests to be more comprehensive

Valarissa · 2021-01-29T21:18:31Z

Tests will continue to fail until tf secrets is bundled with vault, or I create a setup script for adding the plugin to vault.

catsby

Looks great so far! Some minor nits that may or may not be accurate, depending on how good my Terraform knowledge has kept up.

The comment on data source and resource will be had internally, I'll bring it up.

The other thing is documentation missing, will that be opened/add separately?

vault/data_source_terraform_cloud_credentials.go

vault/resource_terraform_cloud_secret_backend.go

vault/resource_terraform_cloud_secret_role.go

catsby

LGTM 👍 (data source -> resource discussion goes beyond this PR so we'll leave it as is)

vault/provider.go

vault/resource_terraform_cloud_secret_role.go

husunal · 2021-02-24T21:01:12Z

Great work thanks! We look forward to using this new TFE secret backend.

What happens to the Terraform resource when the generated token's TTL expires? Also, I didn't see any TTL argument in resource_terraform_cloud_secret_creds.go are you going to add this option as well so we can specify its TTL during secret creation.

Valarissa · 2021-02-25T17:50:01Z

@husunal the creds' TTL should be based on the role used to create the creds.

catsby

The secret resource cred needs to look up the current secret by lease ID to verify the token still exists, otherwise we generate a token with every read. I believe that for Team/Org tokens we wont have a lease ID so we can go ahead and do the full read again.

vault/resource_terraform_cloud_secret_creds.go

catsby

We're getting close, but the handling of the lease look up across orgs/teams and users is not quite right. Let me know if you have any questions here or if I need to clarify anything

vault/resource_terraform_cloud_secret_creds.go

catsby

A few more minor edits and I think we'll be good to go here. Please let me know when you have a chance to look!

vault/resource_terraform_cloud_secret_creds.go

calvn · 2021-03-16T20:35:27Z

vault/resource_terraform_cloud_secret_creds.go

+	organization := d.Get("organization")
+	teamId := d.Get("team_id")
+
+	if organization == "" && teamId == "" {


You can also check for leaseId != "" now that it's an attribute.

Good idea, fixed in latest commit.

calvn · 2021-03-16T20:38:16Z

vault/resource_terraform_cloud_secret_creds.go

+	organization := d.Get("organization")
+	teamId := d.Get("team_id")
+
+	if organization == "" && teamId == "" {


Same here, you can now lease perform the lookup if leaseId != "" (maybe even get rid of organization == "" && teamId == "" since it doesn't really depend on that).

Fixed in latest commit.

catsby

👍

catsby · 2021-03-17T15:43:55Z

vault/resource_terraform_cloud_secret_creds.go

+	return &schema.Resource{
+		Create: createTerraformCloudSecretCredsResource,
+		Read:   readTerraformCloudSecretCredsResource,
+		Update: updateTerraformCloudSecretCredsResource,


nit: in hindsight and looking at the updateTerraformCloudSecretCredsResource (which just destroys then recreates) I think we can just omit this and remove the updateTerraformCloudSecretCredsResource method completely. To my general knowledge and specifically here, I don't think we can update credentials, e.g. we can't do renewals through this resource (or in any other comparable credential resources, for that matter)

…FVP support. (#974) * Add docs to accompany the release of the Terraform Cloud/Enterprise TFVP support. * Add reference to lease_id for user tokens

Lauren Voswinkel added 2 commits January 27, 2021 16:55

Rough outline of the finished TFC/E work

80c93d4

Added terraform cloud token data source

72ea167

Also updated tests to be more comprehensive

Valarissa added enhancement resource data-source ecosystem labels Jan 29, 2021

Valarissa requested review from catsby, pcman312, calvn and austingebauer January 29, 2021 20:01

ghost added the size/XXL label Jan 29, 2021

catsby reviewed Feb 1, 2021

View reviewed changes

Lauren Voswinkel added 2 commits February 1, 2021 13:56

Clean up of deprecated and missed issues

d4990ac

Add sensitive tag to token generated by the access_token read

d843823

catsby approved these changes Feb 4, 2021

View reviewed changes

Update Terraform Secret creds to be a resource rather than a data source

25906b1

Valarissa mentioned this pull request Feb 8, 2021

Add docs to accompany the release of the Terraform Cloud/Enterprise TFVP support. #974

Merged

calvn approved these changes Feb 9, 2021

View reviewed changes

vault/provider.go Outdated Show resolved Hide resolved

vault/resource_terraform_cloud_secret_role.go Show resolved Hide resolved

vault/resource_terraform_cloud_secret_role.go Show resolved Hide resolved

vault/resource_terraform_cloud_secret_role.go Show resolved Hide resolved

Final touches and a route fix

2d9de11

catsby suggested changes Feb 25, 2021

View reviewed changes

vault/resource_terraform_cloud_secret_creds.go Show resolved Hide resolved

Lauren Voswinkel added 4 commits March 10, 2021 11:31

Add a way to read creds that does not create by default

d6cf96f

Change behavior so read doesn't auto-create a token if none exists

c15de00

WIP

8f497d3

Differentiate between user and org/team tokens for reading and IDs

a7aeb4b

catsby reviewed Mar 12, 2021

View reviewed changes

Add (hopefully) final touches

53f9d4b

calvn reviewed Mar 15, 2021

View reviewed changes

vault/resource_terraform_cloud_secret_creds.go Outdated Show resolved Hide resolved

catsby reviewed Mar 16, 2021

View reviewed changes

vault/resource_terraform_cloud_secret_creds.go Outdated Show resolved Hide resolved

vault/resource_terraform_cloud_secret_creds.go Outdated Show resolved Hide resolved

vault/resource_terraform_cloud_secret_creds.go Outdated Show resolved Hide resolved

Cleaning up some outstanding things.

6b705f4

Valarissa requested review from catsby and calvn March 16, 2021 20:32

calvn reviewed Mar 16, 2021

View reviewed changes

More cleanup

beff494

catsby approved these changes Mar 17, 2021

View reviewed changes

Add docs to accompany the release of the Terraform Cloud/Enterprise T…

3b7e05d

…FVP support. (#974) * Add docs to accompany the release of the Terraform Cloud/Enterprise TFVP support. * Add reference to lease_id for user tokens

ghost added the documentation label Mar 17, 2021

Valarissa merged commit 1f3a8d9 into master Mar 17, 2021

rhughes1 mentioned this pull request May 2, 2023

feat: Data source for Terraform Cloud credentials #1834

Open

fairclothjm deleted the add_tfc/e_support branch February 7, 2024 17:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Terraform Cloud/Enterprise credential support #959

Add Terraform Cloud/Enterprise credential support #959

Valarissa commented Jan 29, 2021

Valarissa commented Jan 29, 2021

catsby left a comment

catsby left a comment

husunal commented Feb 24, 2021

Valarissa commented Feb 25, 2021

catsby left a comment

catsby left a comment

catsby left a comment

calvn Mar 16, 2021

Valarissa Mar 16, 2021

calvn Mar 16, 2021

Valarissa Mar 16, 2021

catsby left a comment

catsby Mar 17, 2021

Add Terraform Cloud/Enterprise credential support #959

Add Terraform Cloud/Enterprise credential support #959

Conversation

Valarissa commented Jan 29, 2021

Community Note

Valarissa commented Jan 29, 2021

catsby left a comment

Choose a reason for hiding this comment

catsby left a comment

Choose a reason for hiding this comment

husunal commented Feb 24, 2021

Valarissa commented Feb 25, 2021

catsby left a comment

Choose a reason for hiding this comment

catsby left a comment

Choose a reason for hiding this comment

catsby left a comment

Choose a reason for hiding this comment

calvn Mar 16, 2021

Choose a reason for hiding this comment

Valarissa Mar 16, 2021

Choose a reason for hiding this comment

calvn Mar 16, 2021

Choose a reason for hiding this comment

Valarissa Mar 16, 2021

Choose a reason for hiding this comment

catsby left a comment

Choose a reason for hiding this comment

catsby Mar 17, 2021

Choose a reason for hiding this comment