provider/azurerm: Upgrading to v10.0.2 of the Azure SDK #14004

tombuildsstuff · 2017-04-26T19:11:17Z

~~Note: this is blocked on Azure/azure-sdk-for-go#593~~

The SDK upgrade is needed so that we can proceed with #13634 / #12001

dominik-lekse · 2017-05-11T12:54:26Z

Is there currently a priority on upgrading to SDK v10 over merging in additional resources? If you need support, just let me know.

tombuildsstuff · 2017-05-19T10:39:00Z

Remote State tests pass:

$ envchain azurerm make testacc TEST=./state/remote TESTARGS='-run=TestAzure'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/05/19 11:29:58 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./state/remote -v -run=TestAzure -timeout 120m
=== RUN   TestAzureClient_impl
--- PASS: TestAzureClient_impl (0.00s)
=== RUN   TestAzureClient
2017/05/19 11:31:01 [DEBUG] New state was assigned lineage "270dc674-f10d-460c-88c9-3ed814354047"
2017/05/19 11:31:01 [TRACE] Preserving existing state lineage "270dc674-f10d-460c-88c9-3ed814354047"
--- PASS: TestAzureClient (155.51s)
=== RUN   TestAzureClientEmptyLease
2017/05/19 11:33:26 [DEBUG] New state was assigned lineage "e1ce5a88-5528-46f3-88c4-16fcbece2fab"
2017/05/19 11:33:26 [TRACE] Preserving existing state lineage "e1ce5a88-5528-46f3-88c4-16fcbece2fab"
--- PASS: TestAzureClientEmptyLease (144.84s)
=== RUN   TestAzureClientLease
2017/05/19 11:35:42 [DEBUG] New state was assigned lineage "cbcb6704-1703-47f2-9908-2e40b1f9fce6"
2017/05/19 11:35:42 [TRACE] Preserving existing state lineage "cbcb6704-1703-47f2-9908-2e40b1f9fce6"
--- PASS: TestAzureClientLease (136.27s)
PASS
ok  	github.com/hashicorp/terraform/state/remote	436.643s

tombuildsstuff · 2017-05-19T15:57:06Z

This is the result of the Azure Service Management Provider tests in this branch - where the outstanding failing tests are currently failing in master:

$ envchain azure make testacc TEST=./builtin/providers/azure TESTARGS='-run=TestAcc'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/05/19 15:32:47 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/azure -v -run=TestAcc -timeout 120m
=== RUN   TestAccAzureAffinityGroupBasic
--- PASS: TestAccAzureAffinityGroupBasic (19.29s)
=== RUN   TestAccAzureAffinityGroupUpdate
--- PASS: TestAccAzureAffinityGroupUpdate (29.74s)
=== RUN   TestAccAzureDataDisk_basic
--- PASS: TestAccAzureDataDisk_basic (294.31s)
=== RUN   TestAccAzureDataDisk_update
--- PASS: TestAccAzureDataDisk_update (632.54s)
=== RUN   TestAccAzureDnsServerBasic
--- PASS: TestAccAzureDnsServerBasic (25.58s)
=== RUN   TestAccAzureDnsServerUpdate
--- PASS: TestAccAzureDnsServerUpdate (54.80s)
=== RUN   TestAccAzureHostedServiceBasic
--- PASS: TestAccAzureHostedServiceBasic (17.61s)
=== RUN   TestAccAzureHostedServiceUpdate
--- PASS: TestAccAzureHostedServiceUpdate (52.85s)
=== RUN   TestAccAzureInstance_basic
--- PASS: TestAccAzureInstance_basic (177.55s)
=== RUN   TestAccAzureInstance_separateHostedService
--- PASS: TestAccAzureInstance_separateHostedService (207.06s)
=== RUN   TestAccAzureInstance_advanced
--- PASS: TestAccAzureInstance_advanced (366.63s)
=== RUN   TestAccAzureInstance_update
--- PASS: TestAccAzureInstance_update (660.67s)
=== RUN   TestAccAzureLocalNetworkConnectionBasic
--- PASS: TestAccAzureLocalNetworkConnectionBasic (21.40s)
=== RUN   TestAccAzureLocalNetworkConnectionUpdate
--- PASS: TestAccAzureLocalNetworkConnectionUpdate (63.90s)
=== RUN   TestAccAzureSecurityGroupRuleBasic
--- PASS: TestAccAzureSecurityGroupRuleBasic (152.18s)
=== RUN   TestAccAzureSecurityGroupRuleAdvanced
--- PASS: TestAccAzureSecurityGroupRuleAdvanced (132.19s)
=== RUN   TestAccAzureSecurityGroupRuleUpdate
--- PASS: TestAccAzureSecurityGroupRuleUpdate (303.76s)
=== RUN   TestAccAzureSecurityGroup_basic
--- PASS: TestAccAzureSecurityGroup_basic (77.90s)
=== RUN   TestAccAzureSqlDatabaseServerFirewallRuleBasic
--- PASS: TestAccAzureSqlDatabaseServerFirewallRuleBasic (41.37s)
=== RUN   TestAccAzureSqlDatabaseServerFirewallRuleAdvanced
--- FAIL: TestAccAzureSqlDatabaseServerFirewallRuleAdvanced (68.71s)
	testing.go:344: Error destroying resource! WARNING: Dangling resources
		may exist. The full state and error is shown below.

		Error: Check failed: Error listing Azure Database Server Firewall Rules for Server "vc2qs7xcss": Error response from Azure. Code: , Message: Subscription '00000000-0000-0000-0000-000000000000' does not have the server 'vc2qs7xcss'.

		State: <no state>
=== RUN   TestAccAzureSqlDatabaseServerFirewallRuleUpdate
--- FAIL: TestAccAzureSqlDatabaseServerFirewallRuleUpdate (87.52s)
	testing.go:344: Error destroying resource! WARNING: Dangling resources
		may exist. The full state and error is shown below.

		Error: Check failed: Error listing Azure Database Server Firewall Rules for Server "ftot322227": Error response from Azure. Code: ResourceNotFound, Message: The Uri doesn't match any specific service operation.

		State: <no state>
=== RUN   TestAccAzureSqlDatabaseServer
--- PASS: TestAccAzureSqlDatabaseServer (52.32s)
=== RUN   TestAccAzureSqlDatabaseServiceBasic
--- FAIL: TestAccAzureSqlDatabaseServiceBasic (160.30s)
	testing.go:344: Error destroying resource! WARNING: Dangling resources
		may exist. The full state and error is shown below.

		Error: Check failed: Error issuing Azure SQL Service list request: Error response from Azure. Code: , Message: Subscription '00000000-0000-0000-0000-000000000000' does not have the server 'diyrupjlfb'.

		State: <no state>
=== RUN   TestAccAzureSqlDatabaseServiceAdvanced
--- FAIL: TestAccAzureSqlDatabaseServiceAdvanced (133.64s)
	testing.go:344: Error destroying resource! WARNING: Dangling resources
		may exist. The full state and error is shown below.

		Error: Check failed: Error issuing Azure SQL Service list request: Error response from Azure. Code: , Message: Subscription '00000000-0000-0000-0000-000000000000' does not have the server 'r3g0ku7ssd'.

		State: <no state>
=== RUN   TestAccAzureSqlDatabaseServiceUpdate
--- FAIL: TestAccAzureSqlDatabaseServiceUpdate (203.06s)
	testing.go:344: Error destroying resource! WARNING: Dangling resources
		may exist. The full state and error is shown below.

		Error: Check failed: Error issuing Azure SQL Service list request: Error response from Azure. Code: , Message: Subscription '00000000-0000-0000-0000-000000000000' does not have the server 'sziaazm58x'.

		State: <no state>
=== RUN   TestAccAzureStorageBlockBlob
--- PASS: TestAccAzureStorageBlockBlob (40.65s)
=== RUN   TestAccAzureStoragePageBlob
--- PASS: TestAccAzureStoragePageBlob (34.18s)
=== RUN   TestAccAzureStorageContainer
--- PASS: TestAccAzureStorageContainer (198.87s)
=== RUN   TestAccAzureStorageQueue
--- PASS: TestAccAzureStorageQueue (13.23s)
=== RUN   TestAccAzureStorageService
--- PASS: TestAccAzureStorageService (60.21s)
=== RUN   TestAccAzureVirtualNetwork_basic
--- PASS: TestAccAzureVirtualNetwork_basic (69.74s)
=== RUN   TestAccAzureVirtualNetwork_advanced
--- PASS: TestAccAzureVirtualNetwork_advanced (203.44s)
=== RUN   TestAccAzureVirtualNetwork_update
--- PASS: TestAccAzureVirtualNetwork_update (323.63s)
FAIL
exit status 1
FAIL	github.com/hashicorp/terraform/builtin/providers/azure	4980.862s
make: *** [testacc] Error 1

This is the result of running (one of) the failing tests against the master branch (once updating the SQL Server version to 12.0 as shown here) - but they all produce the same output:

$ envchain azure make testacc TEST=./builtin/providers/azure TESTARGS='-run=TestAccAzureSqlDatabaseServiceBasic'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/05/19 15:23:10 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/azure -v -run=TestAccAzureSqlDatabaseServiceBasic -timeout 120m
=== RUN   TestAccAzureSqlDatabaseServiceBasic
--- FAIL: TestAccAzureSqlDatabaseServiceBasic (117.96s)
	testing.go:344: Error destroying resource! WARNING: Dangling resources
		may exist. The full state and error is shown below.

		Error: Check failed: Error issuing Azure SQL Service list request: Error response from Azure. Code: , Message: Subscription '00000000-0000-0000-0000-000000000000' does not have the server 'mbctsv9asn'.

		State: <no state>
FAIL
exit status 1
FAIL	github.com/hashicorp/terraform/builtin/providers/azure	117.978s

tombuildsstuff · 2017-05-19T20:16:37Z

@dominik-lekse sorry for the delay in responding to this comment - missed this!

Is there currently a priority on upgrading to SDK v10 over merging in additional resources?

To provide some context here - unfortunately we were unable to upgrade to 9.0.X-beta of the Azure SDK for Go last month due to some upstream issues (these issues have since been resolved in version 10.0.2-beta).

Generally speaking these SDK upgrades can be a little painful, however we've never had one that's been delayed for this long - and unfortunately the changes required in upgrading to the new version of the SDK have also been larger than we anticipated, requiring changes in both of the Azure Providers (Service Management & Resource Manager) and the Azure Remote State. Unfortunately this has meant that we've been both blocking several open PR's (e.g. App Services) - as well as made things more complicated for new PR's coming in (given we're tied to an old version of the SDK).

There's no priority on upgrading the SDK to v10 over merging new resources - however being unable to upgrade to the latest SDK does put us into an unfortunate position, where every new resource which gets added has to be upgraded to use the new version of the SDK - and the longer we delay this PR, the more painful it gets.

In general we tend to roll the SDK upgrade out pretty quickly, however this taken longer than we anticipated, so apologies for any delays here. We're now in a position where the SDK upgrade is complete - so I'd expect this to be merged shortly :)

Hope that helps?

mbfrahry · 2017-05-30T20:17:50Z

builtin/providers/azurerm/resource_arm_eventhub_namespace.go


-	if resp.StatusCode != http.StatusNotFound {
+	if resp.StatusCode == http.StatusNotFound {


Do we want to return nil if the status is StatusNotFound?

Looking into this, internally the Azure SDK is polling for one of the status codes:

azure.WithErrorUnlessStatusCode(http.StatusNoContent, http.StatusOK, http.StatusAccepted)

As such, I've removed this if statement - as we can lean on the SDK to check for the Status Code errors here (which are returned as an error and will be handled below)

mbfrahry

LGTM with a minor nit. I'm running tests and will let you know what failures come up from there

tombuildsstuff · 2017-06-01T09:03:36Z

Remote State tests still pass:

$ envchain azurerm make testacc TEST=./state/remote TESTARGS='-run=TestAzure'
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/06/01 09:55:37 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./state/remote -v -run=TestAzure -timeout 120m
=== RUN   TestAzureClient_impl
--- PASS: TestAzureClient_impl (0.00s)
=== RUN   TestAzureClient
2017/06/01 09:56:25 [DEBUG] New state was assigned lineage "981f527d-d0be-4d3c-9013-27ba71af7e99"
2017/06/01 09:56:25 [TRACE] Preserving existing state lineage "981f527d-d0be-4d3c-9013-27ba71af7e99"
--- PASS: TestAzureClient (137.43s)
=== RUN   TestAzureClientEmptyLease
2017/06/01 09:58:40 [DEBUG] New state was assigned lineage "0524c612-c435-4618-983b-bc67efb27a74"
2017/06/01 09:58:40 [TRACE] Preserving existing state lineage "0524c612-c435-4618-983b-bc67efb27a74"
--- PASS: TestAzureClientEmptyLease (140.61s)
=== RUN   TestAzureClientLease
2017/06/01 10:01:01 [DEBUG] New state was assigned lineage "bba3c596-1304-407f-b56e-ca8187aa5176"
2017/06/01 10:01:01 [TRACE] Preserving existing state lineage "bba3c596-1304-407f-b56e-ca8187aa5176"
--- PASS: TestAzureClientLease (135.99s)
PASS
ok  	github.com/hashicorp/terraform/state/remote	414.059s

ghost · 2020-04-11T02:35:02Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

tombuildsstuff added enhancement provider/azurerm labels Apr 26, 2017

tombuildsstuff changed the title ~~[WIP] provider/azurerm: Upgrading to v9 of the Azure SDK~~ [WIP] provider/azurerm: Upgrading to v9.1 of the Azure SDK Apr 26, 2017

tombuildsstuff mentioned this pull request Apr 26, 2017

provider/azurerm: Create azure App services #12001

Closed

tombuildsstuff mentioned this pull request May 4, 2017

Regenerated the SDK to include the latest Service Bus changes Azure/azure-sdk-for-go#601

Closed

tombuildsstuff changed the title ~~[WIP] provider/azurerm: Upgrading to v9.1 of the Azure SDK~~ [WIP] provider/azurerm: Upgrading to v10 of the Azure SDK May 8, 2017

tombuildsstuff force-pushed the azurerm-sdkv9 branch from 0d1e460 to 40d9df4 Compare May 11, 2017 09:52

tombuildsstuff added 22 commits May 18, 2017 14:11

Upgrading to v9 of the Azure SDK for GO

4dc5b53

Refactoring to add a deprecated location field

eb98098

Deprecating the location field in eventhubs

94e6d1d

Updating to use the new Storage SDK within the main repo

0475c4a

Linting

4ec2123

Removing dead code

852ab52

Fixing the merge conflict from master

1828671

Upgrading the Azure SDK to vq0

b922643

Updating autorest

3d2db5d

Working through the v10 changes

2c5e80f

WIP Storage SDK upgrades

348cb4b

Continuing to upgrade the SDK to v10

42cce6a

Upgrading to make use of the latest Storage SDK

d9999b9

Setting the first admin password we retrieve back for the admin registry

46822a4

Formatting

c362586

Upgrading the tests to the new Storage / Azure SDK

2c53b6e

fixing all the things

23b2a4e

Fixing the build for the classic azure provider

e815e91

Upgrading to Azure SDK 10.0.2-beta

9814b89

Fixing the build

686631c

Adding Location back to the Event Hub Resources

b11b536

Fixing CDN Endpoints

5ed864d

tombuildsstuff added 7 commits May 18, 2017 14:11

Chan -> Error for the tests

6a815cc

Fixing Storage Table

ac14a20

Fixing a bug in Eventhub Namespace's

790382c

Adding Location to the Consumer Group tests

4bff4c9

Fixing a CDN Endpoint Update

68dba07

Container Registry: SKU is now Required

367834b

Setting the SKU in the response

30f5945

tombuildsstuff force-pushed the azurerm-sdkv9 branch from e621801 to 30f5945 Compare May 18, 2017 13:11

tombuildsstuff added 2 commits May 18, 2017 14:20

Fixing the AzureRM build

13c8535

Upgrading the Azure State to work with SDKv10

beccfa7

Fixing the Azure Provider tests

ab0754e

tombuildsstuff added 2 commits May 19, 2017 17:56

Fixing the Blob Source tests

ee47bc3

Implemented support for updating container registries

29b553f

tombuildsstuff changed the title ~~[WIP] provider/azurerm: Upgrading to v10 of the Azure SDK~~ provider/azurerm: Upgrading to v10 of the Azure SDK May 19, 2017

tombuildsstuff requested a review from stack72 May 19, 2017 18:58

mbfrahry reviewed May 30, 2017

View reviewed changes

mbfrahry approved these changes May 30, 2017

View reviewed changes

Leaning on the Azure SDK to do the polling

8bd4bc1

tombuildsstuff changed the title ~~provider/azurerm: Upgrading to v10 of the Azure SDK~~ provider/azurerm: Upgrading to v10.0.2 of the Azure SDK Jun 1, 2017

tombuildsstuff merged commit df28750 into master Jun 1, 2017

tombuildsstuff deleted the azurerm-sdkv9 branch June 1, 2017 09:18

tombuildsstuff added a commit that referenced this pull request Jun 1, 2017

Updating the changelog to include #14004

ba625b7

tombuildsstuff mentioned this pull request Jun 2, 2017

provider/azurerm: Upgrading to AutoRest v8 #15006

Merged

dominik-lekse mentioned this pull request Jun 8, 2017

provider/azurerm: Add resources virtual network gateway and connection #13886

Closed

9 tasks

ghost locked and limited conversation to collaborators Apr 11, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

provider/azurerm: Upgrading to v10.0.2 of the Azure SDK #14004

provider/azurerm: Upgrading to v10.0.2 of the Azure SDK #14004

tombuildsstuff commented Apr 26, 2017 •

edited

Loading

dominik-lekse commented May 11, 2017

tombuildsstuff commented May 19, 2017

tombuildsstuff commented May 19, 2017

tombuildsstuff commented May 19, 2017

mbfrahry May 30, 2017

tombuildsstuff Jun 1, 2017

mbfrahry left a comment •

edited

Loading

tombuildsstuff commented Jun 1, 2017

ghost commented Apr 11, 2020


		if resp.StatusCode != http.StatusNotFound {
		if resp.StatusCode == http.StatusNotFound {

provider/azurerm: Upgrading to v10.0.2 of the Azure SDK #14004

provider/azurerm: Upgrading to v10.0.2 of the Azure SDK #14004

Conversation

tombuildsstuff commented Apr 26, 2017 • edited Loading

dominik-lekse commented May 11, 2017

tombuildsstuff commented May 19, 2017

tombuildsstuff commented May 19, 2017

tombuildsstuff commented May 19, 2017

mbfrahry May 30, 2017

Choose a reason for hiding this comment

tombuildsstuff Jun 1, 2017

Choose a reason for hiding this comment

mbfrahry left a comment • edited Loading

Choose a reason for hiding this comment

tombuildsstuff commented Jun 1, 2017

ghost commented Apr 11, 2020

tombuildsstuff commented Apr 26, 2017 •

edited

Loading

mbfrahry left a comment •

edited

Loading