Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update vendor of Azure/azure-sdk-for-go to v21.1.0 #18968

Closed
wants to merge 2 commits into from
Closed

Update vendor of Azure/azure-sdk-for-go to v21.1.0 #18968

wants to merge 2 commits into from

Conversation

donaldguy
Copy link

github.com/Azure/azure-sdk-for-go packages @ v21.1.0
github.com/Azure/go-autorest @ v10.15.4, per azure-sdk-for-go Gopkg.toml
Newly introduced github.com/marstr/guid @ v1.1.0, per azure-sdk-for-go Gopkg.toml

Switch old ARM APIs to what appears to have been the newest API Version at the time
Fix up the interfaces in tests.

github.com/Azure/azure-sdk-for-go packages @ v21.1.0
github.com/Azure/go-autorest @ v10.15.4, per azure-sdk-for-go Gopkg.toml
Newly introduced github.com/marstr/guid @ v1.1.0, per azure-sdk-for-go Gopkg.toml

Switch old ARM APIs to what appears to have been the newest API Version at the time
Fix up the interfaces in tests.
@donaldguy
Copy link
Author

donaldguy commented Sep 28, 2018

Non-acceptance tests pass, not that you should take my word for it:

/Users/donald/go/src/github.com/hashicorp/terraform/backend/remote-state/azure
mediumblue:azure donald$ git rev-parse HEAD
3172eb586e554b74ffa0ea1f9debdbb4c281f59b
mediumblue:azure donald$ go test -test.v
=== RUN   TestBackend_impl
--- PASS: TestBackend_impl (0.00s)
=== RUN   TestBackendConfig
--- PASS: TestBackendConfig (0.00s)
=== RUN   TestBackend
--- SKIP: TestBackend (0.00s)
	backend_test.go:23: azure backend tests require setting TF_ACC or TF_AZURE_TEST
	backend_test.go:24:
=== RUN   TestBackendLocked
--- SKIP: TestBackendLocked (0.00s)
	backend_test.go:23: azure backend tests require setting TF_ACC or TF_AZURE_TEST
	backend_test.go:24:
=== RUN   TestRemoteClient_impl
--- PASS: TestRemoteClient_impl (0.00s)
=== RUN   TestRemoteClient
--- SKIP: TestRemoteClient (0.00s)
	backend_test.go:23: azure backend tests require setting TF_ACC or TF_AZURE_TEST
	backend_test.go:24:
=== RUN   TestRemoteClientLocks
--- SKIP: TestRemoteClientLocks (0.00s)
	backend_test.go:23: azure backend tests require setting TF_ACC or TF_AZURE_TEST
	backend_test.go:24:
=== RUN   TestPutMaintainsMetaData
--- SKIP: TestPutMaintainsMetaData (0.00s)
	backend_test.go:23: azure backend tests require setting TF_ACC or TF_AZURE_TEST
	backend_test.go:24:
PASS
ok  	github.com/hashicorp/terraform/backend/remote-state/azure	0.025s

@donaldguy
Copy link
Author

donaldguy commented Sep 28, 2018

this obsoletes #18476, #18877

The goal here is to give me a place to maybe implement #16763 (or maybe less SAS then the RBAC stuff in preview, https://aka.ms/oauthforstorage )

@donaldguy
Copy link
Author

First attempt at running acceptance tests went failed to list storage account keys storage.AccountsClient#ListKeys: Failure sending request: StatusCode=409 -- Original Error: autorest/azure: Service returned an error. Status=<nil> Code="StorageAccountIsNotProvisioned" Message="The storage account provisioning state must be 'Succeeded' before executing the operation.": which rather makes sense ... I guess I need a real context

@donaldguy
Copy link
Author

Ran the acceptance tests;

  1. Fails TestBackendLocked on backend.TestBackendStateForceUnlock;

this is an accurate reflection of the product when I got here, see also: #17046

  1. Fails TestPutMaintainsMetaData SIGQUIT on me. *** Test killed with quit: ran too long (10m0s)

The rest passed

$ TF_AZURE_TEST=1 go test -test.v
=== RUN   TestBackend_impl
--- PASS: TestBackend_impl (0.00s)
=== RUN   TestBackendConfig
--- PASS: TestBackendConfig (0.00s)
=== RUN   TestBackend
2018/09/28 17:27:59 [DEBUG] Could not lock as state blob did not exist, creating with empty state
2018/09/28 17:27:59 [DEBUG] New state was assigned lineage "b0f05850-7988-e51f-2c3e-3827e61b17e8"
2018/09/28 17:27:59 [TRACE] Preserving existing state lineage "b0f05850-7988-e51f-2c3e-3827e61b17e8"
2018/09/28 17:27:59 [TRACE] Preserving existing state lineage "b0f05850-7988-e51f-2c3e-3827e61b17e8"
2018/09/28 17:27:59 [TRACE] Preserving existing state lineage "b0f05850-7988-e51f-2c3e-3827e61b17e8"
2018/09/28 17:28:00 [TRACE] Preserving existing state lineage "b0f05850-7988-e51f-2c3e-3827e61b17e8"
2018/09/28 17:28:00 [TRACE] Preserving existing state lineage "b0f05850-7988-e51f-2c3e-3827e61b17e8"
2018/09/28 17:28:00 [DEBUG] Could not lock as state blob did not exist, creating with empty state
2018/09/28 17:28:00 [DEBUG] New state was assigned lineage "8af05d5b-cd2d-eb84-495b-1807bad692f6"
2018/09/28 17:28:00 [TRACE] Preserving existing state lineage "8af05d5b-cd2d-eb84-495b-1807bad692f6"
2018/09/28 17:28:00 [TRACE] Preserving existing state lineage "8af05d5b-cd2d-eb84-495b-1807bad692f6"
2018/09/28 17:28:00 [TRACE] Preserving existing state lineage "8af05d5b-cd2d-eb84-495b-1807bad692f6"
2018/09/28 17:28:00 [TRACE] Preserving existing state lineage "8af05d5b-cd2d-eb84-495b-1807bad692f6"
2018/09/28 17:28:00 [TRACE] Preserving existing state lineage "8af05d5b-cd2d-eb84-495b-1807bad692f6"
2018/09/28 17:28:00 [DEBUG] New state was assigned lineage "e8616a38-7cd5-77cb-e3b7-427f5027e221"
2018/09/28 17:28:00 [DEBUG] New state was assigned lineage "22c14952-fb44-f50c-a233-22c4e48d4a99"
2018/09/28 17:28:00 [TRACE] Preserving existing state lineage "b0f05850-7988-e51f-2c3e-3827e61b17e8"
2018/09/28 17:28:00 [TRACE] Preserving existing state lineage "b0f05850-7988-e51f-2c3e-3827e61b17e8"
2018/09/28 17:28:00 [TRACE] Preserving existing state lineage "b0f05850-7988-e51f-2c3e-3827e61b17e8"
2018/09/28 17:28:00 [TRACE] Preserving existing state lineage "8af05d5b-cd2d-eb84-495b-1807bad692f6"
2018/09/28 17:28:00 [TRACE] Preserving existing state lineage "8af05d5b-cd2d-eb84-495b-1807bad692f6"
2018/09/28 17:28:00 [TRACE] Preserving existing state lineage "8af05d5b-cd2d-eb84-495b-1807bad692f6"
2018/09/28 17:28:01 [TRACE] Preserving existing state lineage "b0f05850-7988-e51f-2c3e-3827e61b17e8"
2018/09/28 17:28:01 [TRACE] Preserving existing state lineage "b0f05850-7988-e51f-2c3e-3827e61b17e8"
2018/09/28 17:28:01 [TRACE] Preserving existing state lineage "b0f05850-7988-e51f-2c3e-3827e61b17e8"
2018/09/28 17:28:01 [TRACE] Preserving existing state lineage "b0f05850-7988-e51f-2c3e-3827e61b17e8"
2018/09/28 17:28:01 [TRACE] Preserving existing state lineage "b0f05850-7988-e51f-2c3e-3827e61b17e8"
2018/09/28 17:28:01 [TRACE] Preserving existing state lineage "b0f05850-7988-e51f-2c3e-3827e61b17e8"
2018/09/28 17:28:01 [TRACE] Preserving existing state lineage "8af05d5b-cd2d-eb84-495b-1807bad692f6"
2018/09/28 17:28:01 [TRACE] Preserving existing state lineage "8af05d5b-cd2d-eb84-495b-1807bad692f6"
2018/09/28 17:28:02 [TRACE] Preserving existing state lineage "8af05d5b-cd2d-eb84-495b-1807bad692f6"
2018/09/28 17:28:02 [TRACE] Preserving existing state lineage "8af05d5b-cd2d-eb84-495b-1807bad692f6"
2018/09/28 17:28:02 [DEBUG] Could not lock as state blob did not exist, creating with empty state
2018/09/28 17:28:02 [DEBUG] New state was assigned lineage "0c19687f-3cbd-1f75-0670-0521157adce6"
2018/09/28 17:28:02 [TRACE] Preserving existing state lineage "0c19687f-3cbd-1f75-0670-0521157adce6"
2018/09/28 17:28:02 [TRACE] Preserving existing state lineage "0c19687f-3cbd-1f75-0670-0521157adce6"
2018/09/28 17:28:02 [TRACE] Preserving existing state lineage "0c19687f-3cbd-1f75-0670-0521157adce6"
2018/09/28 17:28:03 [TRACE] Preserving existing state lineage "0c19687f-3cbd-1f75-0670-0521157adce6"
2018/09/28 17:28:03 [TRACE] Preserving existing state lineage "0c19687f-3cbd-1f75-0670-0521157adce6"
--- PASS: TestBackend (131.40s)
	backend_test.go:122: creating resource group terraform-backend-testing-6704764671931056493
	backend_test.go:127: Created resource group: terraform-backend-testing-6704764671931056493
	backend_test.go:130: creating storage account tfbackendtesting8vfz
	backend_test.go:151: fetching access key for storage account
	backend_test.go:168: creating container terraform
	backend_test.go:185: destroying created resources
	backend_test.go:199: Azure resources destroyed
=== RUN   TestBackendLocked
2018/09/28 17:30:21 [DEBUG] Could not lock as state blob did not exist, creating with empty state
2018/09/28 17:30:21 [DEBUG] New state was assigned lineage "de859bb5-c179-fb0a-078f-fe82c33f316d"
2018/09/28 17:30:21 [TRACE] Preserving existing state lineage "de859bb5-c179-fb0a-078f-fe82c33f316d"
2018/09/28 17:30:22 [TRACE] Preserving existing state lineage "de859bb5-c179-fb0a-078f-fe82c33f316d"
2018/09/28 17:30:22 [TRACE] Preserving existing state lineage "de859bb5-c179-fb0a-078f-fe82c33f316d"
2018/09/28 17:30:22 [TRACE] Preserving existing state lineage "de859bb5-c179-fb0a-078f-fe82c33f316d"
2018/09/28 17:30:22 [TRACE] Preserving existing state lineage "de859bb5-c179-fb0a-078f-fe82c33f316d"
--- FAIL: TestBackendLocked (141.13s)
	backend_test.go:122: creating resource group terraform-backend-testing-425029791438780871
	backend_test.go:127: Created resource group: terraform-backend-testing-425029791438780871
	backend_test.go:130: creating storage account tfbackendtestingdlof
	backend_test.go:151: fetching access key for storage account
	backend_test.go:168: creating container terraform
	backend_test.go:93: TestBackend: testing state locking for *azure.Backend
	backend_test.go:94: TestBackend: testing state locking for *azure.Backend
	backend_test.go:94: could not unlock with the reported ID "2043a213-9f57-b2fc-a27b-e672693d7e54": failed to delete lock info from metadata: storage: service returned error: StatusCode=412, ErrorCode=LeaseIdMissing, ErrorMessage=There is currently a lease on the blob and no lease ID was specified in the request.
		RequestId:9c80aab0-401e-0053-6883-571f3d000000
		Time:2018-09-28T23:30:24.0384546Z, RequestInitiated=Fri, 28 Sep 2018 23:30:23 GMT, RequestId=9c80aab0-401e-0053-6883-571f3d000000, API Version=2016-05-31, QueryParameterName=, QueryParameterValue=
		Lock Info:
		  ID:        2043a213-9f57-b2fc-a27b-e672693d7e54
		  Path:      terraform/testState
		  Operation: test
		  Who:       clientA
		  Version:   0.11.9
		  Created:   2018-09-28 23:30:22.736050164 +0000 UTC
		  Info:
	backend_test.go:185: destroying created resources
	backend_test.go:199: Azure resources destroyed
=== RUN   TestRemoteClient_impl
--- PASS: TestRemoteClient_impl (0.00s)
=== RUN   TestRemoteClient
2018/09/28 17:32:30 [DEBUG] New state was assigned lineage "990ddb36-343f-6026-f00b-bc5f00a5083b"
2018/09/28 17:32:30 [TRACE] Preserving existing state lineage "990ddb36-343f-6026-f00b-bc5f00a5083b"
--- PASS: TestRemoteClient (126.46s)
	backend_test.go:122: creating resource group terraform-backend-testing-4764909971965126269
	backend_test.go:127: Created resource group: terraform-backend-testing-4764909971965126269
	backend_test.go:130: creating storage account tfbackendtestinge7s1
	backend_test.go:151: fetching access key for storage account
	backend_test.go:168: creating container terraform
	backend_test.go:185: destroying created resources
	backend_test.go:199: Azure resources destroyed
=== RUN   TestRemoteClientLocks
2018/09/28 17:34:37 [DEBUG] Could not lock as state blob did not exist, creating with empty state
2018/09/28 17:34:37 [DEBUG] New state was assigned lineage "9f4f1947-641e-5655-516d-ce7ba6862e03"
2018/09/28 17:34:37 [TRACE] Preserving existing state lineage "9f4f1947-641e-5655-516d-ce7ba6862e03"
--- PASS: TestRemoteClientLocks (112.77s)
	backend_test.go:122: creating resource group terraform-backend-testing-6349763858108096700
	backend_test.go:127: Created resource group: terraform-backend-testing-6349763858108096700
	backend_test.go:130: creating storage account tfbackendtestingqtm1
	backend_test.go:151: fetching access key for storage account
	backend_test.go:168: creating container terraform
	backend_test.go:185: destroying created resources
	backend_test.go:199: Azure resources destroyed
=== RUN   TestPutMaintainsMetaData
SIGQUIT: quit
PC=0x105ce63 m=0 sigcode=0

will look into TestPutMaintainsMetaData right now and maybe try to get that lease Id in there

@donaldguy
Copy link
Author

donaldguy commented Sep 28, 2018

Got a pass on TestPutMaintainsMetaData the second time; might have been the leaking of the cancel function (context is a pretty new abstraction to me )

oh ... I see thats just a timeout for the whole run... https://stackoverflow.com/questions/27778280/test-killed-with-quit-ran-too-long/31336818

(a complete acceptance run takes 11m 3s 😢 against Go's 10m default timeout )

@tombuildsstuff
Copy link
Contributor

hey @donaldguy

Thanks for this PR - apologies for the delayed review of this PR!

We've recently been taking a look at how we can iterate on the AzureRM Backend to ensure it's consistent with the AzureRM Provider (and in the near future, the Azure Stack Provider). As a result of this we've split the authentication logic used by the AzureRM Provider out into it's own library which has since been incorporated in the AzureRM Repository and the Azure Stack repository - and PR #19414 integrates this library into the AzureRM Backend.

As a result of this we've needed to ensure the version of the Azure SDK for Go used (and thus go-autorest) is consistent across all 3 codebases - as such I've opened #19414 which both includes/switches over to using this library and updates the SDK to v21.3.0 to match the other Providers. As such - whilst I'd like to thank you for this contribution I'm going to close this PR in favour of #19414 which supersedes this.

Thanks!

@ghost
Copy link

ghost commented Mar 31, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked and limited conversation to collaborators Mar 31, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants