v1.17.0 - vsphere_virtual_machine : Invalid operation for device '0'

[derrar05aiss]

Hello,

I'm currently unable to deploy VMs using the terraform-provider-vsphere 1.17.0 and I'm getting :

vsphere_virtual_machine.svvm: Creating...
vsphere_virtual_machine.svvm: Still creating... [10s elapsed]
Error: error reconfiguring virtual machine: error reconfiguring virtual machine: Invalid operation for device '0'.

with the version 1.15, the VM is created with the same configuration

Terraform Version

Terraform v0.12.24

vSphere Provider Version

provider.vsphere = 1.17.0

Affected Resource(s)

• vsphere_virtual_machine

Terraform Configuration Files

variable "dns_servers" {}
variable "domain" {}
variable "vsphere_datacenter" {}
variable "vsphere_cluster" {}
variable "vsphere_datastore" {}
variable "vsphere_template" {}
variable "vsphere_folder" {}
variable "vsphere_network" {}

data "vsphere_datacenter" "datacenter" {
        name = var.vsphere_datacenter
}

data "vsphere_resource_pool" "cluster" {
        name = "${var.vsphere_cluster}/Resources"
        datacenter_id = data.vsphere_datacenter.datacenter.id
}

data "vsphere_datastore" "datastore" {
        name = var.vsphere_datastore
        datacenter_id = data.vsphere_datacenter.datacenter.id
}

data "vsphere_virtual_machine" "template" {
        name          = var.vsphere_template
        datacenter_id = data.vsphere_datacenter.datacenter.id
}

data "vsphere_network" "network" {
        name = var.vsphere_network
        datacenter_id = data.vsphere_datacenter.datacenter.id
}

resource "vsphere_virtual_machine" "svvm" {
        name             = "testvm5"
        folder           = ""
        num_cpus         = 1
        memory           = 1024

        memory_hot_add_enabled = true
        cpu_hot_add_enabled    = true
        cpu_hot_remove_enabled = true

        datastore_id     = data.vsphere_datastore.datastore.id
        resource_pool_id = data.vsphere_resource_pool.cluster.id
        guest_id         = data.vsphere_virtual_machine.template.guest_id
        scsi_type        = data.vsphere_virtual_machine.template.scsi_type

        network_interface {
                network_id     = data.vsphere_network.network.id
        }

        disk {
                label            = "disk0"
                size             = data.vsphere_virtual_machine.template.disks.0.size
                eagerly_scrub    = data.vsphere_virtual_machine.template.disks.0.eagerly_scrub
                thin_provisioned = data.vsphere_virtual_machine.template.disks.0.thin_provisioned
        }
        clone {
                template_uuid = data.vsphere_virtual_machine.template.id
                customize {
                        timeout = 30
                        linux_options {
                          host_name = "testvm"
                          domain = "vm.lab.platform-essential.com"
                        }
                }
        }
}

Debug Output

https://gist.github.com/derrar05aiss/f29196d83eb90dc21380b11eca8c39d5

Expected Behavior

VM creation

Actual Behavior

Error while reconfiguring the VM

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply

References

Are there any other GitHub issues (open or closed) or Pull Requests that should
be linked here?

• v1.16.0 - vsphere_virtual_machine : Invalid operation for device '0' #965

[eystsen]

Hi,

I'm getting this error and been trying to find a workaround until the issue is fixed.
Hopefully this observation might help solving it.

tl;dr;
Using terraform with an ubuntu 18.04 created using packer results in the error.
Create the same VM manually and it works fine with terraform.

More info
I'm using Ubuntu 18.04 and 16.04 (both server). Due to netplan they behave a bit differently. But the error is exactly the same. Invalid operation for device '0'.

Reproducing error

1. Create an Ubuntu 18.04 VM with packer (distributed switch port group)
2. This VM works fine by itself. Network also works when booting it.
3. Using this VM as a template in terraform results network being disconnected due to "Invalid operation for device '0'.
4. Any changes to the template does not help (ex. removing network adapter)

Workaround

1. Create the same Ubuntu 18.04 VM manually. Installing it through vcenter.
   (of course something is different between these VMs, but the point is that a minimal installed VM works)
2. Using this VM as a template with terraform works fine

Summary
It's not purely a terraform issue, since terraform works with a VM created manually.
It's also not purely a packer issue, since the VM created by packer works by itself.
The issue can probably be fixed by both packer and terraform.

Versions tested
Packer v1.5.4 and v1.5.5
Terraform v0.12.23 and v0.12.24
vSphere provider v1.17.0

[vfiftyfive]

I've got exactly the same issue from packer created VM. Seems to be ok with VM created manually.
Is anybody looking at this?
provider version: 1.18.1
TF version: 0.12.24

Nic

[FrederikNJS]

We just hit this one on 1.17.3, it looks identical to #965.

Terraform version was 0.12.24

[LennertMertens]

Having this issue with:

Terraform v0.12.24
+ provider.vsphere v1.20.0

I'm wondering if this is might be a Packer or Terraform related issue?

[eystsen]

I'm wondering if this is might be a Packer or Terraform related issue?

It could even be both.
Packer does something that it shouldn't, leaving the VM in an odd state
Terraform should handle that and make it work anyhow

[rmacian]

Same here, I have two Vsphere environments and the error only happens in one of them:
Works on
Vsphere 6.7.0.42000:
provider.vsphere v1.40.0
Packer: 1.6.1
Terraform v0.12.26

Does NOT work
Vsphere 6.7.0.44000:
BUT:

• I can create a VM manually from vsphere with the template generated with packer and runs ok
• I can create a VM from terraform with an image not generated with Packer

[arothste-blk]

I'm having the same problem with
vSphere 7.0.0.10600
packer 1.6.1
terraform 0.13.0 with the v1.23 of the vsphere provider

packer packed VM templates instantiate through the vSphere UI without problem
instantiation with Terraform yield
"error reconfiguring virtual machine: Invalid operation for device '0'"

[rmacian]

Definetively the issue is in the provider as pointed by @derrar05aiss

I can create the VM forcing the provider to 1.15

[ghost]

Yes same issue here : error reconfiguring virtual machine: error reconfiguring virtual machine: Invalid operation for device '0'.
vsphere 7 and hashicorp/vsphere v1.24.3...
packer version 1.6.6

forcing the provider version to 1.15 helps thanks!

[sestegra]

I found the root cause of this issue by investigating differences between 1.15.0 and 1.16.0 releases.

Investigation logs

First, i activated client_debug in vsphere provider to log SOAP requests and responses into ~/.govmomi/ folder.
A difference appears in ReconfigVM_Task request. In fact, an additional edit operation on hard drive is present in the 1.16.0 request. This operation appears at rank 0.

<deviceChange xmlns:XMLSchema-instance="http://www.w3.org/2001/XMLSchema-instance" XMLSchema-instance:type="VirtualDeviceConfigSpec">
    <operation>edit</operation>
    <device XMLSchema-instance:type="VirtualDisk">
        <key>2000</key>
        <deviceInfo XMLSchema-instance:type="Description">
            <label>Hard disk 1</label>
            <summary>20,480,000 KB</summary>
        </deviceInfo>

The related error message is following.

<error>
    <fault xsi:type="InvalidDeviceOperation">
        <property>virtualDeviceSpec.device</property>
        <deviceIndex>0</deviceIndex>
        <badOp>edit</badOp>
    </fault>
    <localizedMessage>Invalid operation for device &apos;0&apos;.</localizedMessage>
</error>

So, the first clue seems to be the disk device.

Then, I decided to compile every commits between 1.15.0 and 1.16.0 and redo the same test with the compiled plugin by using dev_overrides in ~/.terraformrc. I discovered that the commit bc1eb53 reveals the issue. This commit adds the Storage Policy Based Management).

The second clue is the storage_policy_id parameter that has been added in 1.16.0 version.

Solution

In order to get the name of the Storage Policy at VM and disk levels, follow these instructions:

1. Open your inventory and select your template
2. Select Configure and Policies to get the names

See below the updates required to fix the issue.

data "vsphere_storage_policy" "vm_storage_policy" {
  name = <VM storage policy name>
}

data "vsphere_storage_policy" "disk_storage_policy" {
  name = <Disk storage policy name>
}

resource "vsphere_virtual_machine" "vm" {
  name = "vm"
  ...
  storage_policy_id = data.vsphere_storage_policy.vm_storage_policy.id

  disk {
    ...
    storage_policy_id = data.vsphere_storage_policy.disk_storage_policy.id
  }
}

[gitnetofr]

Hi, I have the same issue.

Environment

• vSphere Client version 6.7.0.45000
• Terraform v0.13.3
• Terraform provider vsphere v1.24.3

It crashes when disk size is calculated basing on the template data:

disk {
       size             = data.vsphere_virtual_machine.vsphere_template.disks.0.size
       ...
       }

I got Error: error reconfiguring virtual machine: error reconfiguring virtual machine: Invalid operation for device '0'

However, if I precise the size > data than vsphere_template.disks.0.size, it works well:

disk {
       size             = data.vsphere_virtual_machine.vsphere_template.disks.0.size+1
       ...
       }

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

When I create the VM manually I do not need to expand the disk size, the VM is created without error.

Any idea?

Thanks a lot.
;-)

[isometry]

@gitnetofr : the root issue appears to result from the vsphere_virtual_machine data source and resource types measuring disk size in integer GiB, whilst real-world templates may have non-integer GiB disk sizes (e.g. Packer/vsphere-iso still expects sizes in integer MiB). Integer arithmetic in terraform-provider-vsphere results in these non-integer GiB disk sizes being rounded down to the nearest GiB. Unfortunately, when you clone using the rounded down size, you're implicitly telling vSphere to shrink the disk, which is unsupported, and results in the Invalid operation for device '0' error.

I've opened #1397 with a fix.

[isometry]

I believe this issue was fixed by #1397, included in the 2.0.0 release.

[tenthirtyam]

@iBrandyJackson and @appilon can you review if this one can be closed per the fix in the PR mentioned above?

Ryan

[appilon]

Will close, if the issue persists after upgrading to v2.0.0 of the provider please feel free to comment for re-evaluation

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.