vsphere_entity_permissions - global permissions preventing removing root folder permissions

[slarimore02]

Terraform Version

Terraform v0.15.1

vSphere Provider Version

1.26.0

Affected Resource(s)

• vsphere_entity_permissions

Terraform Configuration Files

terraform {
  required_version = ">= 0.13.6"

  required_providers {
    vsphere = {
      source  = "hashicorp/vsphere"
      version = "~> 1.26.0"
    }
  }
}
provider "vsphere" {
  user                 = var.vsphere_user
  password             = var.vsphere_password
  vsphere_server       = var.vsphere_server
  allow_unverified_ssl = true
}

data "vsphere_folder" "root" {
  path = "/"
}
resource "vsphere_entity_permissions" "avi_root" {
  entity_id = data.vsphere_folder.root.id
  entity_type = "Folder"
  
  permissions {
    user_or_group = var.vsphere_avi_user
    propagate = true
    is_group = false
    role_id = vsphere_role.avi_root.id
  }
  lifecycle {
    ignore_changes = [ permissions ]
  }
}
resource "vsphere_role" "avi_root" {
  name = "avi_root"
  role_privileges = [
    "Datastore.AllocateSpace", 
    "Network.Assign", 
    "Host.Config.Network", 
    "VirtualMachine.Config.AddNewDisk", 
    "VirtualMachine.Config.AdvancedConfig", 
    "Resource.AssignVMToPool", 
    "VApp.Import"  
  ]
}

Debug Output

https://gist.github.com/slarimore02/dadd86f45cad33bda0905200ce177d84

Expected Behavior

When running terraform destroy the vsphere_entity_permissions resource should be removed for the root vSphere folder. The provider should only remove/create permissions defined in the vsphere_entity_permissions resource not existing Global Permissions.

Actual Behavior

When running terraform destroy the vsphere_entity_permissions resource is not deleted. The root folder inherits all of the users/permissions defined in the Global Permissions and the provider cannot find those additional permissions to remove.

The error below is returned:

Error: error while deleting permission for the user/group VSPHERE.LOCAL\Administrator ServerFaultCode: The object or item referred to could not be found.

Steps to Reproduce

1. terraform apply
2. list inherited global permissions with terraform show
3. terraform destroy.

Important Factoids

vSphere version 7.0.0.10100

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[pezhore]

Adding to this, I think the permissions are removed on non-root folders as well.

[eric-gunn]

This has been a pain point for us, too.

[tenthirtyam]

An update based on my expereince with this issue.

Terraform Version

>= 1.0.0

vSphere Provider Version:

>= 2.0.0

terraform {
  required_providers {
    vsphere = {
      source  = "hashicorp/vsphere"
      version = ">= 2.0.0"
    }
  }
  required_version = ">= 1.0.0"
}

Affected Resource(s)

vsphere_entity_permissions

Terraform Configuration Files

##################################################################################
# VERSIONS
##################################################################################

terraform {
  required_providers {
    vsphere = {
      source  = "hashicorp/vsphere"
      version = ">= 2.0.0"
    }
  }
  required_version = ">= 1.0.0"
}

##################################################################################
# PROVIDERS
##################################################################################

provider "vsphere" {
  vsphere_server       = var.vsphere_server
  user                 = var.vsphere_username
  password             = var.vsphere_password
  allow_unverified_ssl = var.vsphere_insecure
}

##################################################################################
# VARIABLES
##################################################################################

# Credentials

variable "vsphere_server" {}

variable "vsphere_username" {
  sensitive   = true
}

variable "vsphere_password" {
  sensitive   = true
}

variable "vsphere_insecure" {}

vsphere_server   = "sfo-m01-vc01.rainpole.io"

vsphere_username = "[email protected]"

vsphere_password = "***********!"

vsphere_insecure = false

# Reduce Service Account Permissions

variable "vsphere_folder_path" {}
variable "vsphere_entity_type" {}

vsphere_folder_path    = "/"
vsphere_entity_type    = "Folder"

variable "vra_vsphere_user" {}
variable "vra_vsphere_role_label" {}

vra_vsphere_user       = "RAINPOLE\\svc-vra-vsphere"
vra_vsphere_role_label = "No access"

variable "vro_vsphere_user" {}
variable "vro_vsphere_role_label" {}

vro_vsphere_user       = "RAINPOLE\\svc-vro-vsphere"
vro_vsphere_role_label = "No access"

##################################################################################
# DATA
##################################################################################

data "vsphere_folder" "this" {
  path = var.vsphere_folder_path
}

data "vsphere_role" "vra_vsphere_role" {
  label = var.vra_vsphere_role_label
}

data "vsphere_role" "vro_vsphere_role" {
  label = var.vro_vsphere_role_label
}

##################################################################################
# RESOURCES
##################################################################################

resource "vsphere_entity_permissions" "vra_vsphere_user" {
  entity_id   = data.vsphere_folder.this.id
  entity_type = var.vsphere_entity_type
  permissions {
    user_or_group = var.vra_vsphere_user
    role_id       = data.vsphere_role.vra_vsphere_role.id
    is_group      = false
    propagate     = true
  }
}

resource "vsphere_entity_permissions" "vro_vsphere_user" {
  entity_id   = data.vsphere_folder.this.id
  entity_type = var.vsphere_entity_type
  permissions {
    user_or_group = var.vro_vsphere_user
    role_id       = data.vsphere_role.vro_vsphere_role.id
    is_group      = false
    propagate     = true
  }
}

When applying the plan I see that 2 permissions will change....

PS D:\Terraform\vsphere-global-permissions> terraform plan
vsphere_entity_permissions.vro_vsphere_user: Refreshing state... [id=group-d1]
vsphere_entity_permissions.vra_vsphere_user: Refreshing state... [id=group-d1]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the
following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # vsphere_entity_permissions.vra_vsphere_user will be updated in-place
  ~ resource "vsphere_entity_permissions" "vra_vsphere_user" {
        id          = "group-d1"
        # (2 unchanged attributes hidden)

      ~ permissions {
          ~ role_id       = "-562474839" -> "-5"
            # (3 unchanged attributes hidden)
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "802812196" -> null
          - user_or_group = "RAINPOLE\\svc-vrli-vsphere" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-5" -> null
          - user_or_group = "RAINPOLE\\svc-vro-vsphere" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "1823778329" -> null
          - user_or_group = "RAINPOLE\\svc-vrops-mpsd" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "1823778329" -> null
          - user_or_group = "RAINPOLE\\svc-vrops-vsan" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-1007234473" -> null
          - user_or_group = "RAINPOLE\\svc-vrops-vsphere" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-974337264" -> null
          - user_or_group = "RAINPOLE\\svc-vrslcm-vsphere" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "RAINPOLE\\ug-vc-admins" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "VSPHERE.LOCAL\\Administrator" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "VSPHERE.LOCAL\\Administrators" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "11" -> null
          - user_or_group = "VSPHERE.LOCAL\\AutoUpdate" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "1937985690" -> null
          - user_or_group = "VSPHERE.LOCAL\\NsxAdministrators" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "42755698" -> null
          - user_or_group = "VSPHERE.LOCAL\\NsxAuditors" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "1379913370" -> null
          - user_or_group = "VSPHERE.LOCAL\\NsxViAdministrators" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "1006" -> null
          - user_or_group = "VSPHERE.LOCAL\\RegistryAdministrators" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "VSPHERE.LOCAL\\SDDCAdmins" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "VSPHERE.LOCAL\\svc-sfo-m01-nsx01-sfo-m01-vc01" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "VSPHERE.LOCAL\\svc-sfo-w01-nsx01-sfo-w01-vc01" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-974337264" -> null
          - user_or_group = "VSPHERE.LOCAL\\svc-vrslcm-vsphere" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "1002" -> null
          - user_or_group = "VSPHERE.LOCAL\\SyncUsers" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "-7" -> null
          - user_or_group = "VSPHERE.LOCAL\\TrustedAdmins" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "VSPHERE.LOCAL\\vpxd-3654cf7c-6db4-43a5-bd82-c9b05fc78fd0" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "VSPHERE.LOCAL\\vpxd-a6f76a8e-7bf4-465c-a013-d9b6814676db" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "VSPHERE.LOCAL\\vpxd-extension-3654cf7c-6db4-43a5-bd82-c9b05fc78fd0" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "VSPHERE.LOCAL\\vpxd-extension-a6f76a8e-7bf4-465c-a013-d9b6814676db" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-562474839" -> null
          - user_or_group = "VSPHERE.LOCAL\\vra-svc-d4deab82-b6b2-40e2-9b0a-8f99dd0a3527" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "1003" -> null
          - user_or_group = "VSPHERE.LOCAL\\vsphere-webclient-3654cf7c-6db4-43a5-bd82-c9b05fc78fd0" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "1003" -> null
          - user_or_group = "VSPHERE.LOCAL\\vsphere-webclient-a6f76a8e-7bf4-465c-a013-d9b6814676db" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "1003" -> null
          - user_or_group = "VSPHERE.LOCAL\\vSphereClientSolutionUsers" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "1004" -> null
          - user_or_group = "VSPHERE.LOCAL\\WorkloadStorage" -> null
        }
    }

  # vsphere_entity_permissions.vro_vsphere_user will be updated in-place
  ~ resource "vsphere_entity_permissions" "vro_vsphere_user" {
        id          = "group-d1"
        # (2 unchanged attributes hidden)

      ~ permissions {
          ~ role_id       = "-562474839" -> "-5"
          ~ user_or_group = "RAINPOLE\\svc-vra-vsphere" -> "RAINPOLE\\svc-vro-vsphere"
            # (2 unchanged attributes hidden)
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "802812196" -> null
          - user_or_group = "RAINPOLE\\svc-vrli-vsphere" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-5" -> null
          - user_or_group = "RAINPOLE\\svc-vro-vsphere" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "1823778329" -> null
          - user_or_group = "RAINPOLE\\svc-vrops-mpsd" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "1823778329" -> null
          - user_or_group = "RAINPOLE\\svc-vrops-vsan" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-1007234473" -> null
          - user_or_group = "RAINPOLE\\svc-vrops-vsphere" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-974337264" -> null
          - user_or_group = "RAINPOLE\\svc-vrslcm-vsphere" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "RAINPOLE\\ug-vc-admins" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "VSPHERE.LOCAL\\Administrator" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "VSPHERE.LOCAL\\Administrators" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "11" -> null
          - user_or_group = "VSPHERE.LOCAL\\AutoUpdate" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "1937985690" -> null
          - user_or_group = "VSPHERE.LOCAL\\NsxAdministrators" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "42755698" -> null
          - user_or_group = "VSPHERE.LOCAL\\NsxAuditors" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "1379913370" -> null
          - user_or_group = "VSPHERE.LOCAL\\NsxViAdministrators" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "1006" -> null
          - user_or_group = "VSPHERE.LOCAL\\RegistryAdministrators" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "VSPHERE.LOCAL\\SDDCAdmins" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "VSPHERE.LOCAL\\svc-sfo-m01-nsx01-sfo-m01-vc01" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "VSPHERE.LOCAL\\svc-sfo-w01-nsx01-sfo-w01-vc01" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-974337264" -> null
          - user_or_group = "VSPHERE.LOCAL\\svc-vrslcm-vsphere" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "1002" -> null
          - user_or_group = "VSPHERE.LOCAL\\SyncUsers" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "-7" -> null
          - user_or_group = "VSPHERE.LOCAL\\TrustedAdmins" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "VSPHERE.LOCAL\\vpxd-3654cf7c-6db4-43a5-bd82-c9b05fc78fd0" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "VSPHERE.LOCAL\\vpxd-a6f76a8e-7bf4-465c-a013-d9b6814676db" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "VSPHERE.LOCAL\\vpxd-extension-3654cf7c-6db4-43a5-bd82-c9b05fc78fd0" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-1" -> null
          - user_or_group = "VSPHERE.LOCAL\\vpxd-extension-a6f76a8e-7bf4-465c-a013-d9b6814676db" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "-562474839" -> null
          - user_or_group = "VSPHERE.LOCAL\\vra-svc-d4deab82-b6b2-40e2-9b0a-8f99dd0a3527" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "1003" -> null
          - user_or_group = "VSPHERE.LOCAL\\vsphere-webclient-3654cf7c-6db4-43a5-bd82-c9b05fc78fd0" -> null
        }
      - permissions {
          - is_group      = false -> null
          - propagate     = true -> null
          - role_id       = "1003" -> null
          - user_or_group = "VSPHERE.LOCAL\\vsphere-webclient-a6f76a8e-7bf4-465c-a013-d9b6814676db" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "1003" -> null
          - user_or_group = "VSPHERE.LOCAL\\vSphereClientSolutionUsers" -> null
        }
      - permissions {
          - is_group      = true -> null
          - propagate     = true -> null
          - role_id       = "1004" -> null
          - user_or_group = "VSPHERE.LOCAL\\WorkloadStorage" -> null
        }
    }

Plan: 0 to add, 2 to change, 0 to destroy.

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if
you run "terraform apply" now.

However, when applied, it completes but with an erroneous error for another role and permission....

Plan: 0 to add, 2 to change, 0 to destroy.
vsphere_entity_permissions.vra_vsphere_user: Modifying... [id=group-d1]
vsphere_entity_permissions.vro_vsphere_user: Modifying... [id=group-d1]
╷
│ Error: error while deleting permission for the user/group RAINPOLE\svc-vrli-vsphere ServerFaultCode: The object or item referred to could not be found.
│
│   with vsphere_entity_permissions.vra_vsphere_user,
│   on main.tf line 32, in resource "vsphere_entity_permissions" "vra_vsphere_user":
│   32: resource "vsphere_entity_permissions" "vra_vsphere_user" {
│
╵
╷
│ Error: error while deleting permission for the user/group RAINPOLE\svc-vra-vsphere ServerFaultCode: The object or item referred to could not be found.
│
│   with vsphere_entity_permissions.vro_vsphere_user,
│   on main.tf line 43, in resource "vsphere_entity_permissions" "vro_vsphere_user":
│   43: resource "vsphere_entity_permissions" "vro_vsphere_user" {
│
╵

The results are corrects and the users are set to the "No access" role.

However, if I re-apply the plan, the first of the two returns to the Global Permissions settings but the second is skipped. If applied a tird time, it resets to te correct state. And additioanal runs flip between the two.

Next, if I run a terraform destroy, it will remove the state as excepted but with additional errrors as mentioned in the initial summary of this issue.

Plan: 0 to add, 0 to change, 2 to destroy.
vsphere_entity_permissions.vro_vsphere_user: Destroying... [id=group-d1]
vsphere_entity_permissions.vra_vsphere_user: Destroying... [id=group-d1]
╷
│ Error: error while deleting permission for the user/group RAINPOLE\svc-vrli-vsphere ServerFaultCode: The object or item referred to could not be found.
│
│
╵
╷
│ Error: error while deleting permission for the user/group RAINPOLE\svc-vra-vsphere ServerFaultCode: The object or item referred to could not be found.
│

cc @sumitAgrawal007

Ryan

[andyanfieldroad]

I'm also experiencing this issue. Has anyone found a fix or workaround?

[dipeshsharma]

This bug has been open for almost two years now. Just wondering how people are working around this without a fix in place?

[slarimore02]

Manually cleaning up the role (or through some other automation) and then removing the role/vsphere_entity_permissions from the terraform state: terraform state rm vsphere_entity_permissions.avi_root vsphere_role.avi_root

[pezhore]

This bug has been open for almost two years now. Just wondering how people are working around this without a fix in place?

My team has just stopped trying to use this resource - we look at alternative methods of provisioning access.

[tenthirtyam]

This bug has been open for almost two years now. Just wondering how people are working around this without a fix in place?

My team has just stopped trying to use this resource - we look at alternative methods of provisioning access.

There are underlying issues with the implementation that require further development efforts on this resource (and other IAM-related items). However, there are upstream dependencies \that would need to be exposed by govmomi. On the VMware side (this provider is maintained by HashiCorp) we are looking to contribute these features/fixes in the future based on prioritization.

Ryan Johnson
Senior Staff Solutions Architect | Product Engineering @ VMware, Inc.