Terraform Test wrongly removing sensitive marks and warning now Terraform Core can handle sensitive inputs

[seblatre]

Terraform Version

Terraform v1.7.5
on windows_amd64
+ provider registry.terraform.io/hashicorp/azuread v2.47.0

Terraform Configuration Files

serviceprincipal/main.tf

variable "owners" {
  type = list(string)
}

variable "secret_keys" {
  type = list(string)
}

resource "azuread_application" "app_registration" {
  display_name     = "myapp"
  identifier_uris  = ["http://${local.name}.example.com"]
  owners           = [ var.owners ]
  sign_in_audience = "AzureADMyOrg"
}

resource "azuread_application_password" "app_secret" {
  for_each = toset(var.secret_keys)

  application_id = azuread_application.app_registration.id
  display_name   = each.key
}

output "app_secret_by_key" {
  value       = azuread_application_password.app_secret
  sensitive   = true
}

testedmodule/main.tf

variable "service_principal" {
  type = object({
    app_secret = object({
      key_id = string
      value  = string # This is the sensitive data
    })
  })
  sensitive = true
}
# ...

testedmodule/tests/unit.tftest.hcl

run "setup_serviceprincipal" {
  command = apply

  variables {
    owners = [
      # Useless stuff...
    ]
    secret_purposes = [ "mysecret" ]
  }

  module {
    source = "./serviceprincipal"
  }
}

run "testedmodule" {
  command = apply

  variables {
    service_principal = {
      app_secret = run.setup_serviceprincipal.app_secret_by_key["mysecret"]
    }
  }

  # assert {...  

Debug Output

$ terraform test
tests\unit.tftest.hcl... in progress
  run "setup_serviceprincipal"... pass
  run "testedmodule"... fail
╷
│ Error: Failed to prepare variable value for plan
│
│ The value for variable "service_principal" could not be serialized to store in the plan: value has marks, so it cannot be
│ serialized.

Expected Behavior

I'm expecting to be able to pass a secret from one Terraform test run block to another on 2 different modules

Actual Behavior

There is an error happening when I'm trying to pass a secret (sensitive variable) between one run block to the other. When the secret is replaced by an hardcoded value, it's working.

Steps to Reproduce

1. terraform init
2. terraform test

Additional Context

No special context, running locally on a Windows PC

References

No response

[liamcervante]

Hi @seblatre, thanks for filing this! This has been fixed in the 1.8.0 release series, are you able to upgrade to the newly released v1.8.0? We're not currently planning any more releases to the v1.7 series.

Thanks!

[seblatre]

Hello @liamcervante, I updated to the latest version and I can confirm that it's working now.
Any link to the original bug fix around this?
Thanls anyway for your very quick answer!

[liamcervante]

Hi @seblatre, we don't have a specific issue for this as it seems to have been fixed by a change made to add additional support for an unrelated feature.

The PR that fixed it is #34414, with this being the more specific commit: fc75657

In the current code the line that removes the marks you can see are being complained about in the error is here: https://github.com/hashicorp/terraform/blob/main/internal/terraform/context_plan.go#L287.

[liamcervante]

Actually, I will reopen this. Within the test framework specifically we are checking for sensitive marks as a way to warn if they are being lost: https://github.com/hashicorp/terraform/blob/main/internal/backend/local/test.go#L1249-L1269

However, we are not checking the embedded values within complex types so that is probably the actual root cause of this issue originally. The fact the plan functionality now handles marked values does not address the bug in the test framework.

As it is I think we don't need to print that warning anymore, as the plan can now handle sensitive marks on inputs properly. I'll leave this open as a bug, and we can stop stripping the sensitive marks from inputs within the test framework.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.