[Bug]: Creation of aws_security_group_rule succeeds but is not recorded in state file

[karl-sprig]

Terraform Core Version

1.3.0

AWS Provider Version

4.32.0

Affected Resource(s)

aws_security_group_rule

Expected Behavior

I expect Terraform to properly record that the security groups rules are attached to the SG.

Actual Behavior

Terraform does create the rules as expected but does not appear to record their creation.

Relevant Error/Panic Output Snippet

The _exact_ error depends on if doing a `tf apply` or `tf import` but presents as one of:

- Error: InvalidPermission.Duplicate: the specified rule "peer: 0.0.0.0/0, TCP, from port: 80, to port: 80, ALLOW" already exists


 Error: Cannot import non-existent remote object. While attempting to import an existing object to "aws_security_group_rule.atlantis_alb_permit_https_ingress", the provider detected that no object exists with the given id. Only pre-existing objects can be imported; check that the id is correct and that it is associated with the provider's configured region or endpoint, or use "terraform apply" to create a new remote object for this resource.

Terraform Configuration Files

# I have a module that creates an ALB. It creates a few SGs for the ALB but does not create rules as the user of the module
#   is expected to have their own unique rules.
#
# The issue comes from these two rules: this ALB sits in front of a web server and it is meant to be accessed by any/all.
##
# tf import aws_security_group_rule.webApp_alb_permit_https_ingress sg-02..deadbeef..c6_ingress_tcp_80_80_0.0.0.0/0_::0/0
resource "aws_security_group_rule" "webApp_alb_permit_http_ingress" {
  description = "Permits HTTP (TCP/80) inbound from WAN."

  # Incoming TCP/80
  type      = "ingress"
  protocol  = "tcp"
  from_port = 80
  to_port   = 80

  # From WAN
  cidr_blocks      = ["0.0.0.0/0"]
  ipv6_cidr_blocks = ["::0/0"]

  security_group_id = module.webApp_alb.lb_default_sg_id

  # To accelerate debugging/testing
  timeouts {
    create = "1m"
  }
}

# tf import aws_security_group_rule.webApp_alb_permit_https_ingress sg-02..deadbeef..c6_ingress_tcp_443_443_0.0.0.0/0_::0/0
resource "aws_security_group_rule" "webApp_alb_permit_https_ingress" {
  description = "Permits HTTP (TCP/443) inbound from WAN."

  # Incoming TCP/443
  type      = "ingress"
  protocol  = "tcp"
  from_port = 443
  to_port   = 443

  # From WAN
  cidr_blocks      = ["0.0.0.0/0"]
  ipv6_cidr_blocks = ["::0/0"]

  security_group_id = module.webApp_alb.lb_default_sg_id

  # To accelerate debugging/testing
  timeouts {
    create = "1m"
  }
}

Here is how the SG is created inside the ALB module:

resource "aws_security_group" "default" {
  count = module.this.enabled && local._create_sg ? 1 : 0

  # Note: this is not the "human friendly" name. Set the Name tag for that!
  name        = module.sg_label.id
  description = format("Default SG for %s; managed by terraform.", module.main_label.id)

  vpc_id = var.vpc_id

  tags = module.sg_label.tags

  # AWS does not permit changing the name and a SG that's been attached to network things (like an ENI) can't be deleted
  # This is how we work around that
  ##
  lifecycle {
    create_before_destroy = true
  }
}

output "lb_default_sg_id" {
  value       = aws_security_group.default[0].id
  sensitive   = false
  description = "ID of the default SG attached to the LB."
  # Make sure the SG is created before the value can be known. Anything that relies on this output will then implicitly depend on the SG creation
  depends_on = [aws_security_group.default]
}

Steps to Reproduce

• Set up a basic VPC with a public facing ALB.
• Create an aws_security_group, attached to the ALB
• Create 1 or more aws_security_group_rule resources; attach them to the aws_security_group resource created above.
• run tf apply

You should observe:

• Terraform proposes creating the aws_security_group and aws_security_group_rule resources (and, the ALB ... etc, of course!).
• Terraform successfully creates the aws_security_group resource.
• Terraform creates the aws_security_group_rule resources but does not actually record their creation. An error along the lines of InvalidPermission.Duplicate: the specified rule "peer: 0.0.0.0/0, TCP, from port: 80, to port: 80, ALLOW" already exists is emitted.

Debug Output

When setting TF_LOG=trace, I get north of 500K lines. I have tried to tf {apply, import} and have the better part of 2 million lines from the various attempts.

I have run the logs through sed to replace some sensitive/specific details with placeholders.

Here is a gist that contains what seem to be the most relevant lines. If the lines in the gist are not enough, I can provide the full and unobfuscated logs.

There are 3 logs from three operations here:

https://gist.github.com/karl-sprig/2d7cced14d193233b005a22b1bb3a9d0

Panic Output

There is no panic output, thankfully.

Important Factoids

No response

References

Might possibly be related: #12450

Probably is related / the same issue: #26125

A commenter on #26125 thinks that it might be related to #26021 but it seems like the pivitol log line in #26021 is

produced an unexpected new value for ... during refresh.

Which is present in some - but not all - of my TRACE level logs (see linked GIST)

Would you like to implement a fix?

No response

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[mmerickel]

This really needs more attention. It seems like the aws_security_group_rule resource is failing to acknowledge that multiple rules with separate aws ids are created when multiple cidr blocks are defined. Especially between ipv4 and ipv6 cidr blocks. So creating a rule with only ipv4 blocks, and then later adding ipv6 seems to fail every time.

[github-actions]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.