tests/resource/aws_dx_gateway: Skip when associations (likely from other regions) remain as the gateway will fail to delete #10214
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…her regions) remain as the gateway will fail to delete This sweeper is dependent on the Direct Connect Gateway Association sweeper to run prior, which may skip associations that are associated in a different region than the service client. This will leave lingering associations. Direct Connect Gateways with remaining associations will accept the DeleteDirectConnectGateway API call, enter the deleting state, then after a few minutes revert back to the available state. This causes the sweeper to fail, e.g. ``` [01:36:55][Step 2/4] 2019/09/24 01:36:55 [WARN] WaitForState timeout after 20m0s [01:36:55][Step 2/4] 2019/09/24 01:36:55 [WARN] WaitForState starting 30s refresh grace period [01:36:55][Step 2/4] 2019/09/24 01:36:55 [ERR] error running (aws_dx_gateway): error waiting for Direct Connect Gateway (ce06a898-143e-4740-a219-aa32bf564cca) to be deleted: timeout while waiting for state to become 'deleted' (last state: 'available', timeout: 20m0s) ``` Here we skip deleting the Direct Connect Gateway if associations remain as its likely the sweeper will run in the other region and later perform the deletion successfully. Output from sweeper: ```console $ go test ./aws -v -sweep=us-east-1,us-west-2 -sweep-run=aws_dx_gateway -timeout 10h 2019/09/23 21:59:16 [DEBUG] Running Sweepers for region (us-east-1): ... 2019/09/23 21:59:18 [DEBUG] Sweeper (aws_dx_gateway_association) has dependency (aws_dx_gateway_association_proposal), running.. 2019/09/23 21:59:18 [DEBUG] Sweeper (aws_dx_gateway_association_proposal) already ran in region (us-east-1) ... 2019/09/23 21:59:19 [INFO] Skipping Direct Connect Gateway (ce06a898-143e-4740-a219-aa32bf564cca) Association (vgw-0c30b4b34cfbbeda0) in different home region: us-west-2 2019/09/23 21:59:19 [INFO] Skipping Direct Connect Gateway (ce06a898-143e-4740-a219-aa32bf564cca) Association (vgw-0d6cb1878d8b05231) in different home region: us-west-2 2019/09/23 21:59:19 [INFO] Skipping Direct Connect Gateway (f279f38b-b6a6-4cdf-a376-9f1c5d99a4a4) Association (vgw-0202cd084f386200b) in different home region: us-west-2 2019/09/23 21:59:19 [INFO] Skipping Direct Connect Gateway (f279f38b-b6a6-4cdf-a376-9f1c5d99a4a4) Association (vgw-042795456737cc447) in different home region: us-west-2 2019/09/23 21:59:19 [INFO] Skipping Direct Connect Gateway (09a5ef37-651c-45ac-bab0-1e946dadf958) Association (vgw-087f7eb1fe70dcadb) in different home region: us-west-2 2019/09/23 21:59:19 [INFO] Skipping Direct Connect Gateway (cf2a0cfe-663e-4bb4-bedc-fb5535f46bd9) Association (vgw-06cdd41cfff5fae99) in different home region: us-west-2 2019/09/23 21:59:19 [DEBUG] Sweeper (aws_dx_gateway) has dependency (aws_dx_gateway_association), running.. 2019/09/23 21:59:19 [DEBUG] Sweeper (aws_dx_gateway_association) has dependency (aws_dx_gateway_association_proposal), running.. 2019/09/23 21:59:19 [DEBUG] Sweeper (aws_dx_gateway_association_proposal) already ran in region (us-east-1) 2019/09/23 21:59:19 [DEBUG] Sweeper (aws_dx_gateway_association) already ran in region (us-east-1) ... 2019/09/23 21:59:20 [INFO] Skipping Direct Connect Gateway with remaining associations: ce06a898-143e-4740-a219-aa32bf564cca 2019/09/23 21:59:20 [INFO] Skipping Direct Connect Gateway with remaining associations: f279f38b-b6a6-4cdf-a376-9f1c5d99a4a4 2019/09/23 21:59:20 [INFO] Skipping Direct Connect Gateway with remaining associations: 09a5ef37-651c-45ac-bab0-1e946dadf958 2019/09/23 21:59:20 [INFO] Skipping Direct Connect Gateway with remaining associations: cf2a0cfe-663e-4bb4-bedc-fb5535f46bd9 2019/09/23 21:59:20 Sweeper Tests ran: - aws_dx_gateway_association_proposal - aws_dx_gateway_association - aws_dx_gateway ``` As for deleting the associations in the correct region, the Direct Connect API is currently throwing server side errors: ``` 2019/09/23 22:07:56 [ERR] error running (aws_dx_gateway_association): error deleting Direct Connect Gateway (ce06a898-143e-4740-a219-aa32bf564cca) Association (vgw-0d6cb1878d8b05231): DirectConnectServerException: Unable to process request ``` If they persist, we will need to open an AWS Support case. For now, I have manually deleted the associations via the console, which does not have this issue.
ghost
added
size/S
bflad
deleted the
t-aws_dx_gateway-sweeper-cross-region-associations
branch
|
This has been released in version 2.30.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks! |
|
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks! |
ghost
locked and limited conversation to collaborators
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Community Note
Release note for CHANGELOG:
This sweeper is dependent on the Direct Connect Gateway Association sweeper to run prior, which may skip associations that are associated in a different region than the service client. This will leave lingering associations. Direct Connect Gateways with remaining associations will accept the DeleteDirectConnectGateway API call, enter the deleting state, then after a few minutes revert back to the available state. This causes the sweeper to fail, e.g.
Here we skip deleting the Direct Connect Gateway if associations remain as its likely the sweeper will run in the other region and later perform the deletion successfully.
Output from sweeper:
As for deleting the associations in the correct region, the Direct Connect API is currently throwing server side errors:
If they persist, we will need to open an AWS Support case. For now, I have manually deleted the associations via the console, which does not have this issue.