Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switching default_outbound_access_enabled from true to false for azurerm_subnet forces replacement #27848

Closed
1 task done
jcetina opened this issue Oct 31, 2024 · 6 comments · Fixed by #27858
Closed
1 task done

Comments

@jcetina
Copy link

jcetina commented Oct 31, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave comments along the lines of "+1", "me too" or "any updates", they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.8.1

AzureRM Provider Version

3.116.0

Affected Resource(s)/Data Source(s)

azurerm_subnet

Terraform Configuration Files

resource "azurerm_resource_group" "test-default-outbound-switch" {
  name     = "rg-default-outbound-testing"
  location = "eastus2"
}


resource "azurerm_virtual_network" "test-default-outbound-switch" {
  name                = "default-outbound-test"
  location            = azurerm_resource_group.test-default-outbound-switch.location
  resource_group_name = azurerm_resource_group.test-default-outbound-switch.name
  address_space       = ["10.0.0.0/16"]
}

resource "azurerm_subnet" "test-default-outbound-switch" {
  name                 = "example-subnet"
  resource_group_name  = azurerm_resource_group.test-default-outbound-switch.name
  virtual_network_name = azurerm_virtual_network.test-default-outbound-switch.name
  address_prefixes     = ["10.0.0.0/24"]

  default_outbound_access_enabled = false
}

resource "azurerm_subnet" "bastion" {
  name                 = "AzureBastionSubnet"
  resource_group_name  = azurerm_resource_group.test-default-outbound-switch.name
  virtual_network_name = azurerm_virtual_network.test-default-outbound-switch.name
  address_prefixes     = ["10.0.1.0/24"]

  default_outbound_access_enabled = false
}

resource "azurerm_public_ip" "bastion" {
  name                = "bastion"
  location            = azurerm_resource_group.test-default-outbound-switch.location
  resource_group_name = azurerm_resource_group.test-default-outbound-switch.name
  allocation_method   = "Static"
  sku                 = "Standard"
}

resource "azurerm_bastion_host" "bastion" {
  name                = "bastion"
  location            = azurerm_resource_group.test-default-outbound-switch.location
  resource_group_name = azurerm_resource_group.test-default-outbound-switch.name
  sku                 = "Standard"
  tunneling_enabled   = true


  ip_configuration {
    name                 = "configuration"
    subnet_id            = azurerm_subnet.bastion.id
    public_ip_address_id = azurerm_public_ip.bastion.id
  }
}

resource "azurerm_public_ip" "nat-gateway" {
  name                = "nat-gateway-PIP"
  location            = azurerm_resource_group.test-default-outbound-switch.location
  resource_group_name = azurerm_resource_group.test-default-outbound-switch.name
  allocation_method   = "Static"
  sku                 = "Standard"
}

resource "azurerm_nat_gateway" "nat-gateway" {
  name                = "nat-gateway"
  location            = azurerm_resource_group.test-default-outbound-switch.location
  resource_group_name = azurerm_resource_group.test-default-outbound-switch.name
  sku_name            = "Standard"
}

resource "azurerm_nat_gateway_public_ip_association" "nat-gateway" {
  nat_gateway_id       = azurerm_nat_gateway.nat-gateway.id
  public_ip_address_id = azurerm_public_ip.nat-gateway.id
}

Debug Output/Panic Output

[command]/home/runner/work/_temp/ea354170-f195-4f93-ba8f-28d26b2d5dec/terraform-bin plan -no-color -compact-warnings
Running plan in Terraform Cloud. Output will stream here. Pressing Ctrl-C
will stop streaming the logs, but will not stop the plan running remotely.

Preparing the remote plan...

The remote workspace is configured to work with configuration at
/ relative to the target repository.

Terraform will upload the contents of the following directory,
excluding files or directories as defined by a .terraformignore file
at /home/runner/work/azure-security-research/azure-security-research/.terraformignore (if it is present),
in order to capture the filesystem context the remote workspace expects:
/home/runner/work/azure-security-research/azure-security-research

To view this run in a browser, visit:
https://terraform.githubapp.com/app/security-telemetry/azure-security-research/runs/run-UVsSDqSWiqicXCJ3

Waiting for the plan to start...

Terraform v1.8.1
on linux_amd64
Initializing plugins and modules...

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
~ update in-place
-/+ destroy and then create replacement

Terraform will perform the following actions:

# azurerm_bastion_host.bastion will be updated in-place
~ resource "azurerm_bastion_host" "bastion" {
    id                     = "/subscriptions/redacted/resourceGroups/rg-default-outbound-testing/providers/Microsoft.Network/bastionHosts/bastion"
    name                   = "bastion"
    tags                   = {}
  ~ tunneling_enabled      = true -> false
    # (11 unchanged attributes hidden)

    # (1 unchanged block hidden)
}

# azurerm_subnet.test-default-outbound-switch must be replaced
-/+ resource "azurerm_subnet" "test-default-outbound-switch" {
  ~ default_outbound_access_enabled                = true -> false # forces replacement
  ~ enforce_private_link_endpoint_network_policies = false -> (known after apply)
  ~ enforce_private_link_service_network_policies  = false -> (known after apply)
  ~ id                                             = "/subscriptions/<redacted>/resourceGroups/rg-default-outbound-testing/providers/Microsoft.Network/virtualNetworks/default-outbound-test/subnets/example-subnet" -> (known after apply)
    name                                           = "example-subnet"
  ~ private_endpoint_network_policies              = "Enabled" -> (known after apply)
  ~ private_endpoint_network_policies_enabled      = true -> (known after apply)
  ~ private_link_service_network_policies_enabled  = true -> (known after apply)
  - service_endpoint_policy_ids                    = [] -> null
  - service_endpoints                              = [] -> null
    # (3 unchanged attributes hidden)
}

Plan: 1 to add, 2 to change, 1 to destroy.

Expected Behaviour

The expected behavior is for Terraform to simply update the existing resource without replacement. The Azure rest API allows for modification of this setting without replacement. I will also add that the current behavior of this module makes adopting this setting for existing subnets with large amounts of resources on them extremely challenging.

Actual Behaviour

As seen in the plan, terraform wants to replace the subnet specifically because of the change of default_outbound_access_enabled from true to false.

Steps to Reproduce

Create a subnet with default_outbound_access_enabled set to true, deploy it, and then switch it to false and run a plan.

Important Factoids

No response

References

No response

@Chambras
Copy link
Contributor

Chambras commented Nov 1, 2024

Hi @jcetina Have you tried to reproduce it using the latest azurerm provider? as of today is 4.7.0

@jcetina
Copy link
Author

jcetina commented Nov 1, 2024

@Chambras I have not. I'm happy to try it, but also I'm not sure even if it works that it solves my or other Azure customer's problems. Rolling forward a full version from 3->4 is non-trivial at scale. I guess what I'm saying is that if it works in 4, can you backport it to 3?

@Chambras
Copy link
Contributor

Chambras commented Nov 1, 2024

@jcetina sadly I do not think hashicorp doesn't backport fixes :(
So, if an issue is fixed, it goes to newer versions.

@CorrenSoft
Copy link
Contributor

default_outbound_access_enabled is still marked as ForceNew in the code.

According to the FAQs, backporting fixes/changes only happens on exceptional cases.

Sorry for the bad news.

@jcetina
Copy link
Author

jcetina commented Nov 2, 2024

default_outbound_access_enabled is still marked as ForceNew in the code.

According to the FAQs, backporting fixes/changes only happens on exceptional cases.

Sorry for the bad news.

I want to clarify that I truly believe this is an exception case, not just for me, but for any Terraform customer who wants to use this feature. Let me explain.

By default, a machine with out a public IP address on an Azure subnet can still access the Internet. You can read about how it all works here. When you set this flag to false, that behavior is disabled for new VMs being attached to the subnet. That's a good thing for security. Azure is actually removing default outbound access for VMs in 2025. Azure customers who consume Terraform want to get ahead of this by disabling outbound access would normally set the flag to false and add any explicit outbound methods to their subnets. Forcing replacement makes accomplishing this change non-viable. You'd have to remove all resources from your subnet, delete and recreate it, and then reprovision those resources. Nobody is going to do that. I can't predict how people will deal with it, but I don't think it will be easy or good for Azure Terraform customers. It's also not good to introduce this much friction for a good security change. So, while this is obviously in my own interest, I think it's reasonable to consider this as an exceptional case.

Anyway, thanks for reading to the bottom. I understand reasonable people may disagree. I just wanted to make the argument. Thanks.

@Chambras
Copy link
Contributor

Chambras commented Nov 3, 2024

@jcetina you have a very good point and if @CorrenSoft PR gets approved, it might solve your issue. I said it might because you might need to upgrade to the latest version in order to use it.

@github-actions github-actions bot added this to the v4.9.0 milestone Nov 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants