chore: Support secondary eips #1109
Conversation
feat: Nat gateway secondary ips
Nat gateway secondary ips
| @@ -1074,6 +1075,22 @@ resource "aws_eip" "nat" { | |||
| depends_on = [aws_internet_gateway.this] | |||
| } | |||
|
|
|||
| resource "aws_eip" "secondary" { | |||
| for_each = toset(flatten([for nat in aws_eip.nat : [for suffix in local.seips_suffixs : "${nat.tags.Name}-${suffix}"]])) | |||
There was a problem hiding this comment.
we cannot have computed values as keys in maps
There was a problem hiding this comment.
Hey, could you explain why please and if you could recommend an alternative approach?
I did consider doing this with count like the other arrays were doing, however if you changed the number of AZs or number of ips per az it would start trying to move the ips from 1 natgateway to the other which then wants to destroy and re-create the natgateway which isn't ideal. Using keys seem to make the solution much more solid.
Or is it the use of the nat.tags.Name that is the issue. This does seem like it could be flaky 🤔 I guess we could use a range based on
local.create_vpc && var.enable_nat_gateway && !var.reuse_nat_ips ? local.nat_gateway_count : 0
As the first part of the composite key, this would be much more solid thinking about it 🚀
There was a problem hiding this comment.
I see, but a fresh plan with the current code does not result in unknown values being passed to the for_each argument as the Name tag can be determined.
I added 2 ips per gateway to the complete example and a fresh plan is good.
# module.vpc.aws_eip.nat[0] will be created
+ resource "aws_eip" "nat" {
+ tags = {
+ "Example" = "ex-complete"
+ "GithubOrg" = "terraform-aws-modules"
+ "GithubRepo" = "terraform-aws-vpc"
+ "Name" = "ex-complete-eu-west-1a"
}
}
# module.vpc.aws_eip.secondary["ex-complete-eu-west-1a-s1"] will be created
+ resource "aws_eip" "secondary" {
......
}
# module.vpc.aws_eip.secondary["ex-complete-eu-west-1a-s2"] will be created
+ resource "aws_eip" "secondary" {
......
}
There was a problem hiding this comment.
@AlexisColes I tested your solution and got this errors:
module.vpc.aws_eip.secondary["nebula-paastocaas-eks-vpc-nprd-eu-central-1c-s2"]: Creation complete after 0s [id=eipalloc-0b8add66c71b6ab16]
module.vpc.aws_eip.secondary["nebula-paastocaas-eks-vpc-nprd-eu-central-1b-s2"]: Creation complete after 0s [id=eipalloc-08c1ad57da35acdf2]
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[1] to include
╷
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" changed the planned action from
│ Update to DeleteThenCreate.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[1] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .id: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[1] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .network_interface_id: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[1] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .public_ip: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[1] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .association_id: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[1] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .private_ip: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[1] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .secondary_private_ip_address_count: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[1] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .secondary_private_ip_addresses: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[0] to include
│ Warning: Argument is deprecated
│
│ with module.eks.aws_eks_addon.this["kube-proxy"],
│ on .terraform/modules/eks/main.tf line 400, in resource "aws_eks_addon" "this":
│ 400: resolve_conflicts = try(each.value.resolve_conflicts, "OVERWRITE")
│
│ The "resolve_conflicts" attribute can't be set to "PRESERVE" on initial
│ resource creation. Use "resolve_conflicts_on_create" and/or
│ "resolve_conflicts_on_update" instead
│
│ (and 3 more similar warnings elsewhere)
╵
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" changed the planned action from
│ Update to DeleteThenCreate.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[0] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .id: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[0] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .network_interface_id: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[0] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .public_ip: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[0] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .association_id: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[0] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .private_ip: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[0] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .secondary_private_ip_address_count: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[0] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .secondary_private_ip_addresses: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[2] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" changed the planned action from
│ Update to DeleteThenCreate.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[2] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .id: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[2] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .network_interface_id: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[2] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .public_ip: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[2] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .secondary_private_ip_addresses: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[2] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .association_id: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[2] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .private_ip: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.vpc.aws_nat_gateway.this[2] to include
│ new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .secondary_private_ip_address_count: was known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
Error: Process completed with exit code 1.
There was a problem hiding this comment.
The above errors happened running the code in a previously created VPC (with the 5.13.0 version of the module) with the default of 3 NATGW (1 per az), and now running against your version changing the secondary IP's from 0 to 2.
There was a problem hiding this comment.
I'm testing the following:
Adding to locals:
seips_names = flatten([for nat_index in range(0, local.nat_gateway_count) : [for suffix in local.seips_suffixs : "${aws_eip.nat[nat_index].tags.Name}-${suffix}"]])
and replacing the for_each with:
for_each = toset(local.seips_names)
Would that help?
There was a problem hiding this comment.
guess not... same errors "Error: Provider produced inconsistent final plan" when going from 0 secondary IP's to 2 :-(
There was a problem hiding this comment.
@flaviomoringa I have done some testing and it seems a provider issue indeed. The only way to add secondary IPs with terraform is on NAT Gateway creation. Any further change to that results in a re-creation of the NAT Gateway. However, if you have created it without secondary IPs you get the inconsistent plan error. I believe that has to do with how it associates the private IPs to the NAT Gateway interfaces but it doesn't allow manual association.
I tested with this very basic example:
locals {
nat_gateway_count = 2
public_subnets = ["subnet-xxxxxx", "subnet-xxxxxx"]
number_of_secondary_eips_per_gateway = 0
}
resource "aws_eip" "nat" {
count = local.nat_gateway_count
domain = "vpc"
}
resource "aws_eip" "secondary" {
count = local.nat_gateway_count * local.number_of_secondary_eips_per_gateway
domain = "vpc"
}
resource "aws_nat_gateway" "this" {
count = local.nat_gateway_count
allocation_id = aws_eip.nat[count.index].id
subnet_id = local.public_subnets[count.index]
secondary_allocation_ids = slice(aws_eip.secondary[*].id, count.index * local.number_of_secondary_eips_per_gateway, (count.index + 1) * local.number_of_secondary_eips_per_gateway)
}I think this should be an issue on the aws provider repo if there is not one already.
There was a problem hiding this comment.
|
This PR has been automatically marked as stale because it has been open 30 days |
|
This PR was automatically closed because of stale in 10 days |
|
Please re-open this issue. This is still not fixed and is really needed. |
|
I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further. |
resolves #1108