Issue with depending on resource from different module

[pixelicous]

I have a module that calls another module named infrastructure.
The parent module has a azurerm_virtual_machine_extension resource and the child module has a virtual machine resource i am trying to "depend on".
For some reason i cannot make this work, it throws out a message it cant find the module but it seems to me it cant find the resource itself.

Terraform Version

0.11.1

Terraform Configuration Files

Calling the module:

module "Infrastructure" {
  source = "./Infrastructure"
  admUser = "${var.admUser}"
  admPass = "${var.admPass}"
}

The resource which calls the dependancy:

resource "azurerm_virtual_machine_extension" "vm_cse" {
  name                 = "JoinDomain"
  location             = "North Europe"
  resource_group_name  = "${azurerm_resource_group.resource_group.name}"
  virtual_machine_name = "${module.Infrastructure.vm}"
  publisher            = "Microsoft.Compute"
  type                 = "JsonADDomainExtension"
  type_handler_version = "1.3"
  #auto_upgrade_minor_version = false
  depends_on = ["module.Infrastructure.vm"]

  settings = <<SETTINGS
{
		"Name": "${var.DomainName}.com",
		"User": "${var.admUser}@${var.DomainName}.com",
		"OUPath": "",
		"Restart": "true",
		"Options": "3"
}
SETTINGS

protected_settings = <<PROTECTED_SETTINGS
{
    "Password": "${var.admPass}"
}
PROTECTED_SETTINGS
}

The resource that is created under child "infrastructure" module

resource "azurerm_virtual_machine" "vm" {
  name                  = "${local.ResourceNamePrefix}-dc"
  location              = "North Europe"
  resource_group_name   = "${azurerm_resource_group.resource_group.name}"
  network_interface_ids = ["${azurerm_network_interface.vm_nic.id}"]
  vm_size               = "${lookup(var.virtual_machine_sizes, "low")}"
  
  delete_os_disk_on_termination = true
  delete_data_disks_on_termination = true
  

  storage_image_reference {
    publisher = "MicrosoftWindowsServer"
    offer     = "WindowsServer"
    sku       = "2016-Datacenter"
    version   = "latest"
  }

  storage_os_disk {
    name              = "${local.ResourceNamePrefix}-dc-osdisk"
    caching           = "None"
    create_option     = "FromImage"
  #  managed_disk_type = "Standard_LRS"
  }

  os_profile {
    computer_name  = "sig${var.CustomerNumber}dc"
    admin_username = "${var.admUser}"
    admin_password = "${var.admPass}"
    custom_data = "${base64encode("Param($RemoteHostName = \"sig${var.CustomerNumber}dc\", $ComputerName = \"sig${var.CustomerNumber}\", $WinRmPort = \"5986\") ${file("./RemoteScripts/DC/configwinrm.ps1")}")}"
  }

  storage_os_disk {
    name              = "winrmtesting-osdisk"
    caching           = "ReadWrite"
    create_option     = "FromImage"
    managed_disk_type = "Standard_LRS"
  }

  storage_image_reference {
    publisher = "MicrosoftWindowsServer"
    offer     = "WindowsServer"
    sku       = "2016-Datacenter"
    version   = "latest"
  }

  os_profile_windows_config {
      provision_vm_agent = true
      enable_automatic_upgrades = true

        additional_unattend_config {
            pass = "oobeSystem"
            component = "Microsoft-Windows-Shell-Setup"
            setting_name = "AutoLogon"
            content = "<AutoLogon><Password><Value>${var.admPass}</Value></Password><Enabled>true</Enabled><LogonCount>1</LogonCount><Username>${var.admUser}</Username></AutoLogon>"
        }
        #Unattend config is to enable basic auth in WinRM, required for the provisioner stage.
        additional_unattend_config {
            pass = "oobeSystem"
            component = "Microsoft-Windows-Shell-Setup"
            setting_name = "FirstLogonCommands"
            content = "${file("./RemoteScripts/DC/LogonCommands.xml")}"
        }
  }
 
 
  provisioner "file" {
    source      = "./RemoteScripts/DC/ConfigDC.ps1"
    destination = "c:/temp/script.ps1"

    connection {
      type     = "winrm"
      host     = "${azurerm_network_interface.vm_nic.private_ip_address}"
      user     = "${var.admUser}"
      password = "${var.admPass}"
      insecure = "true"
	    port = 5986
      https = "true"
    }
  }

    provisioner "file" {
    source      = "./RemoteScripts/DC/configDC-WaitDomain.ps1"
    destination = "c:/temp/script.ps1"

    connection {
      type     = "winrm"
      host     = "${azurerm_network_interface.vm_nic.private_ip_address}"
      user     = "${var.admUser}"
      password = "${var.admPass}"
      insecure = "true"
	    port = 5986
      https = "true"
    }
  }

  provisioner "remote-exec" {
    connection {
      type     = "winrm"
      host     = "${azurerm_network_interface.vm_nic.private_ip_address}"
      user     = "${var.admUser}"
      password = "${var.admPass}"
      insecure = "true"
      port = 5986
      https = "true"
    }

    inline = [
      "powershell -ExecutionPolicy Unrestricted -File c:\\temp\\myscript1.ps1"
    ]
  }

  
tags = "${var.resource_tags}"
}

**The outputs file on the child module:**

output "vm" {
value = "${azurerm_virtual_machine.vm.id}"
}

Debug Output

from log, only find i could find related to those objects.
[DEBUG] ReferenceTransformer: "module.TenantOnBoard.module.Infrastructure.output.vm" references: [module.TenantOnBoard.module.Infrastructure.azurerm_virtual_machine.vm]

Crash Output

azurerm_virtual_machine_extension.vm_cse: resource depends on non-existent module 'Infrastructure.vm'

Expected Behavior

CSE extension from parent root could depend on output of VM once it is ready

Actual Behavior

Fails with above error

Steps to Reproduce

Terraform apply

Important Factoids

I have added ".id" to the return object of the virtual machine resource, that didn't help.

This isn't the only resource that I had this issue with, I also tried doing this the other way around, depend on a virtual_macihne_extension resource while i have on the parent module a virtual machine that is depending on it

[ColinHebert]

depends_on = ["module.Infrastructure.vm"]
You cannot really depend on a specific output of a module. What you would do is use one of the outputs of the module in your parameters (which you are already doing with virtual_machine_name = "${module.Infrastructure.vm}".

How does it behave if you remove the depends on?

[pixelicous]

@ColinHebert My problem is that the resource that depends on the other resource might not have anything to do with the first one.
What i mean is, let's say i have 5 servers and one of them is a domain controller, i bootstrap it and now i want to wait on its creation for the rest of the VMs, once it is ready i can incorporate joining to the domain.
I guess ill have to join the modules together somehow and then use that depends on within the module.

Not sure why it isn't possible to depend on output, seem very logical and easy to graph such thing..
Besides the obvious fact that the error message has nothing to do with the actual error..

[ColinHebert]

Not sure why it isn't possible to depend on output, seem very logical and easy to graph such thing..

Not necessarily in this case, depending on a specific output is very similar to depending on an attribute of a resource. What is usually done in Terraform is depending on the resource itself (not its attribute), and that is done more often than not implicitly by terraform). So if you wanted to absolutely have a depends_on it would be on the module itself rather than one of its outputs.

But as I said, this is expected to be handled by terraform, the fact that your azurerm_virtual_machine.vm has virtual_machine_name = "${module.Infrastructure.vm}" already makes it dependent on the module.

[pixelicous]

@ColinHebert That is actually a mistake. virtual_machine_name = "${module.Infrastructure.vm}" should be virtual_machine_name = "${module.Infrastructure.some_other_vm}" i just changed the names a bit to not use our production naming here.. so i accidentally did referred to it.

That CSE resource isn't running on the same VM.
I am not after being dependent on the whole module, there is a step that takes a very long time while all the rest of the module finishes very quickly, i just want to depend on other specific resource that the rest can continue on without waiting for the whole module to finish

[ColinHebert]

This isn't, as far as I know, quite how modules work though. Someone who knows the system more than I do could jump in, but my current understanding is that modules are meant to be atomic (in a similar way resources are atomic), it can be executed in parallel with other modules, but you can't access its output until the entire module has been applied/resolved.

It seems I was wrong:

Since modules encapsulate other resources, however, the dependency is not on the module as a whole but rather on the server_availability_zone output specifically, which allows Terraform to work on resources in different modules concurrently rather than waiting for the entire module to be complete before proceeding.

Source: https://www.terraform.io/docs/modules/usage.html

So in that case I would go with a null_resource that uses ${module.Infrastructure.vm} and has azurerm_virtual_machine.vm depending on that null_resource.

[apparentlymart]

Great discussion here @pixelicous and @ColinHebert! Thanks.

As you saw in the docs, modules are implemented in a special way compared to resources where the dependencies are on individual variables and outputs rather than the module as a whole. As noted in that text, this allows Terraform to maximize parallelism by getting started on parts of a module where possible, rather than waiting until the entire thing is complete before proceeding.

However, it is true that depends_on doesn't accept outputs and variables today. At first glance it seems like it could, though! Both outputs and variables are nodes in the dependency graph, so we'd need to teach the depends_on parser how to recognize module.foo.bar and var.baz strings properly.

Interestingly based on the error here it seems like Terraform is already attempting to support this, probably by coincidence due to how the dependency graph is built. The fact that it split module. from Infrastructure.vm to produce that error message is telling, but no such support is intended.

We already have #10462 open about the idea of supporting depends_on on modules themselves, which is problematic due to being incompatible with how Terraform models modules in the graph, but supporting dependencies on specific variables and outputs could be an interesting way to meet some of the use-cases for inter-module dependencies without radically changing how Terraform constructs the graph.

The Terraform team at HashiCorp is currently focused on some more general configuration language improvements, but we'll think some more about this and see how well it works once we're ready to start looking at explicit dependencies between modules. Thanks for the suggestion!

[pixelicous]

@apparentlymart Hi, well if we are speaking on maximuizing parallelism then you should support depending on services that are inside a module. Terraform is actually doing this right now, i have two modules, one which is nested. the nested module is creating nics with dynamic ip addresses and outputs the ip and i have a parent module that is setting the same nics with static ip and it utilizes the output ip address from the nested module, so basically it depends on that ip address.. its just not explicit depending on it, the graph does it, it understand it needs that IP address before it can set it as static ip.. so something of this nature can be supported if needed..

[apparentlymart]

Hi all!

Support for module outputs and variables inside depends_on has now been merged to master ready to be included in the forthcoming v0.12.0 release.

I verified this on the v0.12.0-alpha1 prerelease build using the following root module:

module "child" {
  source = "./child"
}

resource "null_resource" "bar" {
  depends_on = [module.child.foo]
}

The child module contains the following:

resource "null_resource" "foo" {
}

output "foo" {
  value = null_resource.foo.id
}

Including module.child.foo in the depends_on list now behaves the same as interpolating that output value into the resource, so the dependency graph for this configuration is the following:

The top-level null_resource.bar depends on module.child.null_resource.foo indirectly through the output.

---

I then updated the root module to the following:

module "child" {
  source = "./child"
}

module "child2" {
  source = "./child2"

  foo_id = module.child.foo
}

This introduces the "child2" module, which has the following configuration:

variable "foo_id" {
}

resource "null_resource" "baz" {
  depends_on = [var.foo_id]
}

Now module.child2.null_resource.baz depends on the foo_id variable, which means it in turn depends on anything that variable depends on.

---

This completes the support for output values and input variables in depends_on, which allows inter-module dependencies to be constructed indirectly. #10462 remains open to cover the further idea of having a module as a whole depend on another module, which would ultimately be a shorthand way to create dependencies for all of a module's variables at once.

Since the feature of using depends_on with outputs and variables is complete and ready for inclusion in the forthcoming v0.12.0, I'm going to close this issue out, leaving #10462 to represent the other related work.

[JnMik]

@apparentlymart Is there a way to create a proper dependency relation if I have no direct access to the child module resources ?

Let's say I want to use an external module (https://github.com/hashicorp/terraform-aws-vault/blob/master/modules/private-tls-cert), which only outputs variables.

The module generate a certificate file and i'm trying to render the file content, but my file() function is run before the module has finish creating the file.

How can I make sure my own resource is triggered after the whole module has finish running ?
Seems I can't properly create a dependency relation using depends_on on the module OR on the output variables.

output "ca_public_key_file_path_content" {
  depends_on = [module.private-tls-cert]
  value =  file("${module.private-tls-cert.ca_public_key_file_path}")
}

Error: Error in function call

  on ../modules/vault/private-tls-cert-generation/output.tf line 21, in output "ca_public_key_file_content":
  21:   value =  file("${module.private-tls-cert.ca_public_key_file_path}")
    |----------------
    | module.private-tls-cert.ca_public_key_file_path is "ca.crt.pem"

Call to function "file" failed: no file exists at ca.crt.pem.

Thanks for your time :)

Edited: Oh well I think I found more informations on my use case here : #22036
I should use data local_file instead of file() function.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.