Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Resource elasticstack_fleet_integration_policy fails for CSPM integration #921

Open
tehbooom opened this issue Nov 19, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@tehbooom
Copy link
Member

Describe the bug
Applying the CSPM integration to a policy works but returns an error. The integration is added to the policy the provider returns the error
panic: interface conversion: interface {} is nil, not map[string]interface {}

To Reproduce
Steps to reproduce the behavior:

  1. TF configuration used
resource "elasticstack_fleet_integration_policy" "cspm" {
  name                = "cspm"
  namespace           = "default"
  description         = "GCP Cloud Security Posture Management"
  agent_policy_id     = elasticstack_fleet_agent_policy.gcp.policy_id
  integration_name    = elasticstack_fleet_integration.cspm.name
  integration_version = elasticstack_fleet_integration.cspm.version
  vars_json = jsonencode({
    "posture" : "cspm",
    "deployment" : "gcp"
  })
  input {
    input_id = "kspm-cloudbeat/cis_k8s"
    enabled  = false
    streams_json = jsonencode({
      "cloud_security_posture.findings" : {
        "enabled" : false,
        "vars" : {}
      }
    })
  }
  input {
    input_id = "kspm-cloudbeat/cis_eks"
    enabled  = false
    streams_json = jsonencode({
      "cloud_security_posture.findings" : {
        "enabled" : false,
        "vars" : {}
      }
    })
  }
  input {
    input_id = "cspm-cloudbeat/cis_aws"
    enabled  = false
    streams_json = jsonencode({
      "cloud_security_posture.findings" : {
        "enabled" : false,
        "vars" : {
          "aws.account_type" : "organization-account"
        }
      }
    })
  }
  input {
    input_id = "cspm-cloudbeat/cis_gcp"
    enabled  = true
    streams_json = jsonencode({
      "cloud_security_posture.findings" : {
        "enabled" : true,
        "setup_access" : "manual",
        "vars" : {
          "gcp.account_type" : "single-account",
          "gcp.project_id" : var.project_id,
          "gcp.credentials.type" : "credentials-json",
          "gcp.credentials.json" : var.monitor_json_key
        },
      }
    })
  }
  input {
    input_id = "cspm-cloudbeat/cis_azure"
    enabled  = false
    streams_json = jsonencode({
      "cloud_security_posture.findings" : {
        "enabled" : false,
        "vars" : {}
      }
    })
  }
  input {
    input_id = "vuln_mgmt-cloudbeat/vuln_mgmt_aws"
    enabled  = false
    streams_json = jsonencode({
      "cloud_security_posture.vulnerabilities" : {
        "enabled" : false
      }
    })
  }
}
  1. Run terraform apply
  2. See the error in the output
module.fleet_policies.elasticstack_fleet_integration_policy.cspm: Creating...
╷
│ Error: Plugin did not respond
│
│ The plugin encountered an error, and failed to respond to the plugin6.(*GRPCProvider).ApplyResourceChange call. The plugin logs may contain more details.
╵

Stack trace from the terraform-provider-elasticstack_v0.11.9 plugin:

panic: interface conversion: interface {} is nil, not map[string]interface {}

goroutine 29 [running]:
github.com/elastic/terraform-provider-elasticstack/internal/fleet/integration_policy.HandleReqRespSecrets({0x1a9ad78, 0xc0001b4780}, {0xc00042bec8, 0x0, 0x0, 0xc00017c180, {0xc00014ec54, 0x4}, 0xc00042bee0, {{0xc0004a06d8, ...}, ...}, ...}, ...)
        github.com/elastic/terraform-provider-elasticstack/internal/fleet/integration_policy/secrets.go:174 +0x405
github.com/elastic/terraform-provider-elasticstack/internal/fleet/integration_policy.(*integrationPolicyResource).Create(0xc00017c078, {0x1a9ad78, 0xc0001b4780}, {{{{0x1aa1758, 0xc00039c450}, {0x165e380, 0xc0005ae3f0}}, {0x1aa4840, 0xc000498280}}, {{{0x1aa1758, ...}, ...}, ...}, ...}, ...)
        github.com/elastic/terraform-provider-elasticstack/internal/fleet/integration_policy/create.go:37 +0x43a
github.com/hashicorp/terraform-plugin-framework/internal/fwserver.(*Server).CreateResource(0xc0002661e0, {0x1a9ad78, 0xc0001b4780}, 0xc0004554b0, 0xc000455488)
        github.com/hashicorp/[email protected]/internal/fwserver/server_createresource.go:101 +0x578
github.com/hashicorp/terraform-plugin-framework/internal/fwserver.(*Server).ApplyResourceChange(0xc0002661e0, {0x1a9ad78, 0xc0001b4780}, 0xc0006b2af0, 0xc0004555a0)
        github.com/hashicorp/[email protected]/internal/fwserver/server_applyresourcechange.go:57 +0x4aa
github.com/hashicorp/terraform-plugin-framework/internal/proto6server.(*Server).ApplyResourceChange(0xc0002661e0, {0x1a9ad78?, 0xc0001b4360?}, 0xc0006b2a50)
        github.com/hashicorp/[email protected]/internal/proto6server/server_applyresourcechange.go:55 +0x38e
github.com/hashicorp/terraform-plugin-mux/tf6muxserver.(*muxServer).ApplyResourceChange(0xc000376460, {0x1a9ad78?, 0xc0001b4090?}, 0xc0006b2a50)
        github.com/hashicorp/[email protected]/tf6muxserver/mux_server_ApplyResourceChange.go:36 +0x193
github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server.(*server).ApplyResourceChange(0xc00036c780, {0x1a9ad78?, 0xc00043de00?}, 0xc0000d2a80)
        github.com/hashicorp/[email protected]/tfprotov6/tf6server/server.go:865 +0x3bc
github.com/hashicorp/terraform-plugin-go/tfprotov6/internal/tfplugin6._Provider_ApplyResourceChange_Handler({0x1802e60, 0xc00036c780}, {0x1a9ad78, 0xc00043de00}, 0xc00019db00, 0x0)
        github.com/hashicorp/[email protected]/tfprotov6/internal/tfplugin6/tfplugin6_grpc.pb.go:545 +0x1a6
google.golang.org/grpc.(*Server).processUnaryRPC(0xc00043a000, {0x1a9ad78, 0xc00043dd70}, {0x1aa2e00, 0xc0003616c0}, 0xc000784a20, 0xc00043c4b0, 0x275f2f8, 0x0)
        google.golang.org/[email protected]/server.go:1394 +0xe2b
google.golang.org/grpc.(*Server).handleStream(0xc00043a000, {0x1aa2e00, 0xc0003616c0}, 0xc000784a20)
        google.golang.org/[email protected]/server.go:1805 +0xe8b
google.golang.org/grpc.(*Server).serveStreams.func2.1()
        google.golang.org/[email protected]/server.go:1029 +0x7f
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 14
        google.golang.org/[email protected]/server.go:1040 +0x125

Error: The terraform-provider-elasticstack_v0.11.9 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

Expected behavior
Terraform apply to run without an error

Debug output
Run terraform command with TF_LOG=trace and provide extended information on TF operations. Please ensure you redact any base64 encoded credentials from your output.
eg

2024-11-19T14:56:51.049Z [TRACE] terraform.contextPlugins: Schema for provider "registry.terraform.io/elastic/elasticstack" is in the global cache
2024-11-19T14:56:51.049Z [INFO]  Starting apply for module.fleet_policies.elasticstack_fleet_integration_policy.cspm
2024-11-19T14:56:51.049Z [TRACE] terraform.contextPlugins: Schema for provider "registry.terraform.io/elastic/elasticstack" is in the global cache
2024-11-19T14:56:51.049Z [TRACE] terraform.contextPlugins: Schema for provider "registry.terraform.io/elastic/elasticstack" is in the global cache
2024-11-19T14:56:51.049Z [TRACE] terraform.contextPlugins: Schema for provider "registry.terraform.io/elastic/elasticstack" is in the global cache
2024-11-19T14:56:51.049Z [TRACE] terraform.contextPlugins: Schema for provider "registry.terraform.io/elastic/elasticstack" is in the global cache
2024-11-19T14:56:51.049Z [TRACE] terraform.contextPlugins: Schema for provider "registry.terraform.io/elastic/elasticstack" is in the global cache
2024-11-19T14:56:51.049Z [TRACE] terraform.contextPlugins: Schema for provider "registry.terraform.io/elastic/elasticstack" is in the global cache
2024-11-19T14:56:51.050Z [TRACE] terraform.contextPlugins: Schema for provider "registry.terraform.io/elastic/elasticstack" is in the global cache
2024-11-19T14:56:51.050Z [DEBUG] module.fleet_policies.elasticstack_fleet_integration_policy.cspm: applying the planned Create change
2024-11-19T14:56:51.050Z [TRACE] GRPCProvider.v6: ApplyResourceChange
2024-11-19T14:56:51.050Z [TRACE] GRPCProvider.v6: GetProviderSchema
2024-11-19T14:56:51.050Z [TRACE] GRPCProvider.v6: returning cached schema: EXTRA_VALUE_AT_END=registry.terraform.io/elastic/elasticstack
2024-11-19T14:56:51.051Z [TRACE] provider.terraform-provider-elasticstack_v0.11.9: Received request: @module=sdk.proto tf_resource_type=elasticstack_fleet_integration_policy @caller=github.com/hashicorp/[email protected]/tfprotov6/tf6server/server.go:852 tf_proto_version=6.6 tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_req_id=338f302e-3354-801e-0274-285ae357f76d tf_rpc=ApplyResourceChange timestamp=2024-11-19T14:56:51.051Z
2024-11-19T14:56:51.051Z [TRACE] provider.terraform-provider-elasticstack_v0.11.9: Sending request downstream: tf_req_id=338f302e-3354-801e-0274-285ae357f76d tf_resource_type=elasticstack_fleet_integration_policy @caller=github.com/hashicorp/[email protected]/tfprotov6/internal/tf6serverlogging/downstream_request.go:22 @module=sdk.proto tf_proto_version=6.6 tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_rpc=ApplyResourceChange timestamp=2024-11-19T14:56:51.051Z
2024-11-19T14:56:51.051Z [TRACE] provider.terraform-provider-elasticstack_v0.11.9: calling downstream server: tf_mux_provider="*proto6server.Server" tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/[email protected]/internal/logging/mux.go:19 @module=sdk.mux timestamp=2024-11-19T14:56:51.051Z
2024-11-19T14:56:51.051Z [TRACE] provider.terraform-provider-elasticstack_v0.11.9: Checking ResourceTypes lock: @module=sdk.framework tf_mux_provider="*proto6server.Server" tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_req_id=338f302e-3354-801e-0274-285ae357f76d tf_resource_type=elasticstack_fleet_integration_policy @caller=github.com/hashicorp/[email protected]/internal/fwserver/server.go:510 tf_rpc=ApplyResourceChange timestamp=2024-11-19T14:56:51.051Z
2024-11-19T14:56:51.052Z [TRACE] provider.terraform-provider-elasticstack_v0.11.9: ApplyResourceChange received no PriorState, running CreateResource: tf_mux_provider="*proto6server.Server" tf_resource_type=elasticstack_fleet_integration_policy tf_rpc=ApplyResourceChange @module=sdk.framework @caller=github.com/hashicorp/[email protected]/internal/fwserver/server_applyresourcechange.go:45 tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_req_id=338f302e-3354-801e-0274-285ae357f76d timestamp=2024-11-19T14:56:51.052Z
2024-11-19T14:56:51.052Z [TRACE] provider.terraform-provider-elasticstack_v0.11.9: Resource implements ResourceWithConfigure: @module=sdk.framework tf_mux_provider="*proto6server.Server" tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_req_id=338f302e-3354-801e-0274-285ae357f76d tf_resource_type=elasticstack_fleet_integration_policy tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/[email protected]/internal/fwserver/server_createresource.go:47 timestamp=2024-11-19T14:56:51.052Z
2024-11-19T14:56:51.052Z [TRACE] provider.terraform-provider-elasticstack_v0.11.9: Calling provider defined Resource Configure: @caller=github.com/hashicorp/[email protected]/internal/fwserver/server_createresource.go:54 tf_rpc=ApplyResourceChange tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_req_id=338f302e-3354-801e-0274-285ae357f76d tf_resource_type=elasticstack_fleet_integration_policy @module=sdk.framework tf_mux_provider="*proto6server.Server" timestamp=2024-11-19T14:56:51.052Z
2024-11-19T14:56:51.052Z [TRACE] provider.terraform-provider-elasticstack_v0.11.9: Called provider defined Resource Configure: tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_resource_type=elasticstack_fleet_integration_policy tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/[email protected]/internal/fwserver/server_createresource.go:56 @module=sdk.framework tf_mux_provider="*proto6server.Server" tf_req_id=338f302e-3354-801e-0274-285ae357f76d timestamp=2024-11-19T14:56:51.052Z
2024-11-19T14:56:51.052Z [TRACE] provider.terraform-provider-elasticstack_v0.11.9: Calling provider defined Resource Create: tf_mux_provider="*proto6server.Server" tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/[email protected]/internal/fwserver/server_createresource.go:100 @module=sdk.framework tf_req_id=338f302e-3354-801e-0274-285ae357f76d tf_resource_type=elasticstack_fleet_integration_policy timestamp=2024-11-19T14:56:51.052Z
2024-11-19T14:56:51.054Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9: Fleet API Request Details:
---[ REQUEST ]---------------------------------------
POST /kibana/api/fleet/package_policies?format=simplified HTTP/1.1
Host: test-host
User-Agent: Go-http-client/1.1
Content-Length: 3606
Authorization: ***************************************************
Content-Type: application/json
Kbn-Xsrf: true
Accept-Encoding: gzip

{
 "description": "GCP Cloud Security Posture Management",
 "inputs": {
  "cspm-cloudbeat/cis_aws": {
   "enabled": false,
   "streams": {
    "cloud_security_posture.findings": {
     "enabled": false,
     "vars": {
      "aws.account_type": "organization-account"
     }
    }
   }
  },
  "cspm-cloudbeat/cis_azure": {
   "enabled": false,
   "streams": {
    "cloud_security_posture.findings": {
     "enabled": false,
     "vars": {}
    }
   }
  },
  "cspm-cloudbeat/cis_gcp": {
   "enabled": true,
   "streams": {
    "cloud_security_posture.findings": {
     "enabled": true,
     "vars": {
      "gcp.account_type": "single-account",
      "gcp.credentials.json": "{}"
      "gcp.credentials.type": "credentials-json",
      "gcp.project_id": "test-project"
     }
    }
   }
  },
  "kspm-cloudbeat/cis_eks": {
   "enabled": false,
   "streams": {
    "cloud_security_posture.findings": {
     "enabled": false,
     "vars": {}
    }
   }
  },
  "kspm-cloudbeat/cis_k8s": {
   "enabled": false,
   "streams": {
    "cloud_security_posture.findings": {
     "enabled": false,
     "vars": {}
    }
   }
  },
  "vuln_mgmt-cloudbeat/vuln_mgmt_aws": {
   "enabled": false,
   "streams": {
    "cloud_security_posture.vulnerabilities": {
     "enabled": false
    }
   }
  }
 },
 "name": "cspm",
 "namespace": "default",
 "package": {
  "name": "cloud_security_posture",
  "version": "1.10.1"
 },
 "policy_id": "0b02a1d0-93d6-4a1a-89f9-c9d42a12055b",
 "vars": {
  "deployment": "gcp",
  "posture": "cspm"
 }
}
-----------------------------------------------------: @caller=github.com/elastic/terraform-provider-elasticstack/internal/utils/http_log.go:40 tf_mux_provider="*proto6server.Server" tf_resource_type=elasticstack_fleet_integration_policy tf_rpc=ApplyResourceChange @module=elasticstack tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_req_id=338f302e-3354-801e-0274-285ae357f76d timestamp=2024-11-19T14:56:51.053Z
2024-11-19T14:56:56.026Z [TRACE] dag/walk: vertex "root" is waiting for "module.fleet_policies (close)"
2024-11-19T14:56:56.027Z [TRACE] dag/walk: vertex "provider[\"registry.terraform.io/elastic/elasticstack\"] (close)" is waiting for "module.fleet_policies.elasticstack_fleet_integration_policy.cspm"
2024-11-19T14:56:56.028Z [TRACE] dag/walk: vertex "module.fleet_policies (close)" is waiting for "module.fleet_policies.elasticstack_fleet_integration_policy.cspm"
2024-11-19T14:56:56.460Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9: Fleet API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Cache-Control: private, no-cache, no-store, must-revalidate
Content-Security-Policy: script-src 'report-sample' 'self'; worker-src 'report-sample' 'self' blob:; style-src 'report-sample' 'self' 'unsafe-inline'
Content-Type: application/json; charset=utf-8
Cross-Origin-Opener-Policy: same-origin
Date: Tue, 19 Nov 2024 14:56:56 GMT
Elastic-Api-Version: 2023-10-31
Kbn-License-Sig: ba54a3a8df3e6b6a7b56f6824b00a8ae5e6982ab0ba11a13f538dc96b4345ce6
Kbn-Name: kibana-test
Permissions-Policy: camera=(), display-capture=(), fullscreen=(self), geolocation=(), microphone=(), web-share=()
Referrer-Policy: strict-origin-when-cross-origin
Server: envoy
Vary: accept-encoding
X-Content-Type-Options: nosniff
X-Envoy-Upstream-Service-Time: 5394

f52
{
 "item": {
  "id": "d3ca59ef-4db1-4717-86d7-75593c5331cc",
  "version": "WzQzNjIsMV0=",
  "name": "cspm",
  "namespace": "default",
  "description": "GCP Cloud Security Posture Management",
  "package": {
   "name": "cloud_security_posture",
   "title": "Security Posture Management",
   "version": "1.10.1"
  },
  "enabled": true,
  "policy_id": "0b02a1d0-93d6-4a1a-89f9-c9d42a12055b",
  "inputs": {
   "kspm-cloudbeat/cis_k8s": {
    "enabled": false,
    "streams": {
     "cloud_security_posture.findings": {
      "enabled": false,
      "vars": {}
     }
    }
   },
   "kspm-cloudbeat/cis_eks": {
    "enabled": false,
    "streams": {
     "cloud_security_posture.findings": {
      "enabled": false,
      "vars": {}
     }
    }
   },
   "cspm-cloudbeat/cis_aws": {
    "enabled": false,
    "streams": {
     "cloud_security_posture.findings": {
      "enabled": false,
      "vars": {
       "aws.account_type": "organization-account"
      }
     }
    }
   },
   "cspm-cloudbeat/cis_gcp": {
    "enabled": true,
    "streams": {
     "cloud_security_posture.findings": {
      "enabled": true,
      "vars": {
       "gcp.account_type": "single-account",
       "gcp.project_id": "project",
       "gcp.credentials.type": "credentials-json",
       "gcp.credentials.json": "{}"
      }
     }
    }
   },
   "cspm-cloudbeat/cis_azure": {
    "enabled": false,
    "streams": {
     "cloud_security_posture.findings": {
      "enabled": false,
      "vars": {}
     }
    }
   },
   "vuln_mgmt-cloudbeat/vuln_mgmt_aws": {
    "enabled": false,
    "streams": {
     "cloud_security_posture.vulnerabilities": {
      "enabled": false
     }
    }
   }
  },
  "vars": {
   "posture": {
    "value": "cspm",
    "type": "text"
   },
   "deployment": {
    "value": "gcp",
    "type": "text"
   }
  },
  "revision": 1,
  "created_at": "2024-11-19T14:56:51.971Z",
  "created_by": "system",
  "updated_at": "2024-11-19T14:56:51.971Z",
  "updated_by": "system"
 }
}
0


-----------------------------------------------------: @caller=github.com/elastic/terraform-provider-elasticstack/internal/utils/http_log.go:52 tf_mux_provider="*proto6server.Server" tf_req_id=338f302e-3354-801e-0274-285ae357f76d tf_rpc=ApplyResourceChange @module=elasticstack tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_resource_type=elasticstack_fleet_integration_policy timestamp=2024-11-19T14:56:56.460Z
2024-11-19T14:56:56.460Z [TRACE] provider.terraform-provider-elasticstack_v0.11.9: Served request: tf_proto_version=6.6 tf_rpc=ApplyResourceChange @caller=runtime/panic.go:785 tf_req_id=338f302e-3354-801e-0274-285ae357f76d tf_resource_type=elasticstack_fleet_integration_policy @module=sdk.proto tf_provider_addr=registry.terraform.io/elastic/elasticstack timestamp=2024-11-19T14:56:56.460Z
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9: panic: interface conversion: interface {} is nil, not map[string]interface {}
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9: goroutine 52 [running]:
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9: github.com/elastic/terraform-provider-elasticstack/internal/fleet/integration_policy.HandleReqRespSecrets({0x1a9ad78, 0xc0006f4f90}, {0xc0004937f0, 0x0, 0x0, 0xc00006e740, {0xc0003a46d4, 0x4}, 0xc000493808, {{0xc00003afa8, ...}, ...}, ...}, ...)
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9:      github.com/elastic/terraform-provider-elasticstack/internal/fleet/integration_policy/secrets.go:174 +0x405
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9: github.com/elastic/terraform-provider-elasticstack/internal/fleet/integration_policy.(*integrationPolicyResource).Create(0xc00006e520, {0x1a9ad78, 0xc0006f4f90}, {{{{0x1aa1758, 0xc0004c0f90}, {0x165e380, 0xc00039e5d0}}, {0x1aa4840, 0xc00048e370}}, {{{0x1aa1758, ...}, ...}, ...}, ...}, ...)
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9:      github.com/elastic/terraform-provider-elasticstack/internal/fleet/integration_policy/create.go:37 +0x43a
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9: github.com/hashicorp/terraform-plugin-framework/internal/fwserver.(*Server).CreateResource(0xc0002661e0, {0x1a9ad78, 0xc0006f4f90}, 0xc00054f4b0, 0xc00054f488)
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9:      github.com/hashicorp/[email protected]/internal/fwserver/server_createresource.go:101 +0x578
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9: github.com/hashicorp/terraform-plugin-framework/internal/fwserver.(*Server).ApplyResourceChange(0xc0002661e0, {0x1a9ad78, 0xc0006f4f90}, 0xc0002eadc0, 0xc00054f5a0)
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9:      github.com/hashicorp/[email protected]/internal/fwserver/server_applyresourcechange.go:57 +0x4aa
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9: github.com/hashicorp/terraform-plugin-framework/internal/proto6server.(*Server).ApplyResourceChange(0xc0002661e0, {0x1a9ad78?, 0xc0006f4ea0?}, 0xc0002ead20)
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9:      github.com/hashicorp/[email protected]/internal/proto6server/server_applyresourcechange.go:55 +0x38e
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9: github.com/hashicorp/terraform-plugin-mux/tf6muxserver.(*muxServer).ApplyResourceChange(0xc000378460, {0x1a9ad78?, 0xc0006f4ba0?}, 0xc0002ead20)
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9:      github.com/hashicorp/[email protected]/tf6muxserver/mux_server_ApplyResourceChange.go:36 +0x193
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9: github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server.(*server).ApplyResourceChange(0xc00036a820, {0x1a9ad78?, 0xc0006f41e0?}, 0xc00021a2a0)
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9:      github.com/hashicorp/[email protected]/tfprotov6/tf6server/server.go:865 +0x3bc
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9: github.com/hashicorp/terraform-plugin-go/tfprotov6/internal/tfplugin6._Provider_ApplyResourceChange_Handler({0x1802e60, 0xc00036a820}, {0x1a9ad78, 0xc0006f41e0}, 0xc0005da200, 0x0)
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9:      github.com/hashicorp/[email protected]/tfprotov6/internal/tfplugin6/tfplugin6_grpc.pb.go:545 +0x1a6
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9: google.golang.org/grpc.(*Server).processUnaryRPC(0xc00043a000, {0x1a9ad78, 0xc0006f4150}, {0x1aa2e00, 0xc000363520}, 0xc00062e000, 0xc00043c4b0, 0x275f2f8, 0x0)
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9:      google.golang.org/[email protected]/server.go:1394 +0xe2b
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9: google.golang.org/grpc.(*Server).handleStream(0xc00043a000, {0x1aa2e00, 0xc000363520}, 0xc00062e000)
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9:      google.golang.org/[email protected]/server.go:1805 +0xe8b
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9: google.golang.org/grpc.(*Server).serveStreams.func2.1()
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9:      google.golang.org/[email protected]/server.go:1029 +0x7f
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9: created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 14
2024-11-19T14:56:56.463Z [DEBUG] provider.terraform-provider-elasticstack_v0.11.9:      google.golang.org/[email protected]/server.go:1040 +0x125
2024-11-19T14:56:56.465Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/elastic/elasticstack/0.11.9/linux_amd64/terraform-provider-elasticstack_v0.11.9 pid=1250806 error="exit status 2"
2024-11-19T14:56:56.465Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2024-11-19T14:56:56.465Z [ERROR] plugin6.(*GRPCProvider).ApplyResourceChange: error="rpc error: code = Unavailable desc = error reading from server: EOF"
2024-11-19T14:56:56.465Z [TRACE] maybeTainted: module.fleet_policies.elasticstack_fleet_integration_policy.cspm encountered an error during creation, so it is now marked as tainted
2024-11-19T14:56:56.465Z [TRACE] terraform.contextPlugins: Schema for provider "registry.terraform.io/elastic/elasticstack" is in the global cache
2024-11-19T14:56:56.465Z [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState to workingState for module.fleet_policies.elasticstack_fleet_integration_policy.cspm
2024-11-19T14:56:56.465Z [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState: removing state object for module.fleet_policies.elasticstack_fleet_integration_policy.cspm
2024-11-19T14:56:56.465Z [TRACE] evalApplyProvisioners: module.fleet_policies.elasticstack_fleet_integration_policy.cspm is tainted, so skipping provisioning
2024-11-19T14:56:56.465Z [TRACE] maybeTainted: module.fleet_policies.elasticstack_fleet_integration_policy.cspm was already tainted, so nothing to do
2024-11-19T14:56:56.465Z [TRACE] terraform.contextPlugins: Schema for provider "registry.terraform.io/elastic/elasticstack" is in the global cache
2024-11-19T14:56:56.465Z [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState to workingState for module.fleet_policies.elasticstack_fleet_integration_policy.cspm
2024-11-19T14:56:56.465Z [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState: removing state object for module.fleet_policies.elasticstack_fleet_integration_policy.cspm
2024-11-19T14:56:56.465Z [DEBUG] State storage *remote.State declined to persist a state snapshot
2024-11-19T14:56:56.465Z [ERROR] vertex "module.fleet_policies.elasticstack_fleet_integration_policy.cspm" error: Plugin did not respond
2024-11-19T14:56:56.465Z [TRACE] vertex "module.fleet_policies.elasticstack_fleet_integration_policy.cspm": visit complete, with errors
2024-11-19T14:56:56.465Z [TRACE] dag/walk: upstream of "provider[\"registry.terraform.io/elastic/elasticstack\"] (close)" errored, so skipping
2024-11-19T14:56:56.465Z [TRACE] dag/walk: upstream of "module.fleet_policies (close)" errored, so skipping
2024-11-19T14:56:56.465Z [TRACE] dag/walk: upstream of "root" errored, so skipping
2024-11-19T14:56:56.466Z [DEBUG] states/remote: state read serial is: 47; serial is: 47
2024-11-19T14:56:56.466Z [DEBUG] states/remote: state read lineage is: 9269388d-053b-d881-f5f1-a8d91718cab2; lineage is: 9269388d-053b-d881-f5f1-a8d91718cab2
╷
│ Error: Plugin did not respond
│
│ The plugin encountered an error, and failed to respond to the plugin6.(*GRPCProvider).ApplyResourceChange call. The plugin logs may contain more details.
╵

Stack trace from the terraform-provider-elasticstack_v0.11.9 plugin:

panic: interface conversion: interface {} is nil, not map[string]interface {}

goroutine 52 [running]:
github.com/elastic/terraform-provider-elasticstack/internal/fleet/integration_policy.HandleReqRespSecrets({0x1a9ad78, 0xc0006f4f90}, {0xc0004937f0, 0x0, 0x0, 0xc00006e740, {0xc0003a46d4, 0x4}, 0xc000493808, {{0xc00003afa8, ...}, ...}, ...}, ...)
        github.com/elastic/terraform-provider-elasticstack/internal/fleet/integration_policy/secrets.go:174 +0x405
github.com/elastic/terraform-provider-elasticstack/internal/fleet/integration_policy.(*integrationPolicyResource).Create(0xc00006e520, {0x1a9ad78, 0xc0006f4f90}, {{{{0x1aa1758, 0xc0004c0f90}, {0x165e380, 0xc00039e5d0}}, {0x1aa4840, 0xc00048e370}}, {{{0x1aa1758, ...}, ...}, ...}, ...}, ...)
        github.com/elastic/terraform-provider-elasticstack/internal/fleet/integration_policy/create.go:37 +0x43a
github.com/hashicorp/terraform-plugin-framework/internal/fwserver.(*Server).CreateResource(0xc0002661e0, {0x1a9ad78, 0xc0006f4f90}, 0xc00054f4b0, 0xc00054f488)
        github.com/hashicorp/[email protected]/internal/fwserver/server_createresource.go:101 +0x578
github.com/hashicorp/terraform-plugin-framework/internal/fwserver.(*Server).ApplyResourceChange(0xc0002661e0, {0x1a9ad78, 0xc0006f4f90}, 0xc0002eadc0, 0xc00054f5a0)
        github.com/hashicorp/[email protected]/internal/fwserver/server_applyresourcechange.go:57 +0x4aa
github.com/hashicorp/terraform-plugin-framework/internal/proto6server.(*Server).ApplyResourceChange(0xc0002661e0, {0x1a9ad78?, 0xc0006f4ea0?}, 0xc0002ead20)
        github.com/hashicorp/[email protected]/internal/proto6server/server_applyresourcechange.go:55 +0x38e
github.com/hashicorp/terraform-plugin-mux/tf6muxserver.(*muxServer).ApplyResourceChange(0xc000378460, {0x1a9ad78?, 0xc0006f4ba0?}, 0xc0002ead20)
        github.com/hashicorp/[email protected]/tf6muxserver/mux_server_ApplyResourceChange.go:36 +0x193
github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server.(*server).ApplyResourceChange(0xc00036a820, {0x1a9ad78?, 0xc0006f41e0?}, 0xc00021a2a0)
        github.com/hashicorp/[email protected]/tfprotov6/tf6server/server.go:865 +0x3bc
github.com/hashicorp/terraform-plugin-go/tfprotov6/internal/tfplugin6._Provider_ApplyResourceChange_Handler({0x1802e60, 0xc00036a820}, {0x1a9ad78, 0xc0006f41e0}, 0xc0005da200, 0x0)
        github.com/hashicorp/[email protected]/tfprotov6/internal/tfplugin6/tfplugin6_grpc.pb.go:545 +0x1a6
google.golang.org/grpc.(*Server).processUnaryRPC(0xc00043a000, {0x1a9ad78, 0xc0006f4150}, {0x1aa2e00, 0xc000363520}, 0xc00062e000, 0xc00043c4b0, 0x275f2f8, 0x0)
        google.golang.org/[email protected]/server.go:1394 +0xe2b
google.golang.org/grpc.(*Server).handleStream(0xc00043a000, {0x1aa2e00, 0xc000363520}, 0xc00062e000)
        google.golang.org/[email protected]/server.go:1805 +0xe8b
google.golang.org/grpc.(*Server).serveStreams.func2.1()
        google.golang.org/[email protected]/server.go:1029 +0x7f
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 14
        google.golang.org/[email protected]/server.go:1040 +0x125

Error: The terraform-provider-elasticstack_v0.11.9 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

2024-11-19T14:56:56.659Z [DEBUG] provider: plugin exited

Screenshots
If applicable, add screenshots to help explain your problem.

Versions (please complete the following information):

  • OS: Linux
  • Terraform Version 1.7.3
  • Provider version 0.11.9
  • Elasticsearch Version 8.13.2

Additional context
Add any other context about the problem here.

@tehbooom tehbooom added the bug Something isn't working label Nov 19, 2024
@tehbooom
Copy link
Member Author

Can confirm this works in 0.11.6 but fails when upgrading the provider

@tehbooom
Copy link
Member Author

tehbooom commented Dec 17, 2024

With what is currently in main I now get the following error

╷
│ Error: Provider produced inconsistent result after apply
│
│ When applying changes to elasticstack_fleet_integration_policy.cspm, provider "provider[\"registry.terraform.io/elastic/elasticstack\"]" produced an unexpected new value: .input[3].streams_json: inconsistent
│ values for sensitive attribute.
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Using the following terraform

resource "elasticstack_fleet_integration" "cspm" {
  name         = "cloud_security_posture"
  version      = "1.10.1"
  force        = true
  skip_destroy = true
}

resource "elasticstack_fleet_agent_policy" "gcp" {
  name            = "gcp"
  namespace       = "default"
  description     = "Collect logs and metrics from GSS Project"
  monitor_logs    = true
  monitor_metrics = true
  sys_monitoring  = true
  skip_destroy    = false
}

resource "elasticstack_fleet_integration_policy" "cspm" {
  name                = "cspm"
  namespace           = "default"
  description         = "GCP Cloud Security Posture Management"
  agent_policy_id     = elasticstack_fleet_agent_policy.gcp.policy_id
  integration_name    = elasticstack_fleet_integration.cspm.name
  integration_version = elasticstack_fleet_integration.cspm.version
  vars_json = jsonencode({
    "posture" : "cspm",
    "deployment" : "gcp"
  })
  input {
    input_id = "kspm-cloudbeat/cis_k8s"
    enabled  = false
    streams_json = jsonencode({
      "cloud_security_posture.findings" : {
        "enabled" : false,
        "vars" : {}
      }
    })
  }
  input {
    input_id = "kspm-cloudbeat/cis_eks"
    enabled  = false
    streams_json = jsonencode({
      "cloud_security_posture.findings" : {
        "enabled" : false,
        "vars" : {}
      }
    })
  }
  input {
    input_id = "cspm-cloudbeat/cis_aws"
    enabled  = false
    streams_json = jsonencode({
      "cloud_security_posture.findings" : {
        "enabled" : false,
        "vars" : {
          "aws.account_type" : "organization-account"
        }
      }
    })
  }
  input {
    input_id = "cspm-cloudbeat/cis_gcp"
    enabled  = true
    streams_json = jsonencode({
      "cloud_security_posture.findings" : {
        "enabled" : true,
        "setup_access" : "manual",
        "vars" : {
          "gcp.account_type" : "single-account",
          "gcp.project_id" : "test",
          "gcp.credentials.type" : "credentials-json",
          "gcp.credentials.json" : "test"
        },
      }
    })
  }
  input {
    input_id = "cspm-cloudbeat/cis_azure"
    enabled  = false
    streams_json = jsonencode({
      "cloud_security_posture.findings" : {
        "enabled" : false,
        "vars" : {}
      }
    })
  }
  input {
    input_id = "vuln_mgmt-cloudbeat/vuln_mgmt_aws"
    enabled  = false
    streams_json = jsonencode({
      "cloud_security_posture.vulnerabilities" : {
        "enabled" : false
      }
    })
  }
}
terraform {
  required_version = ">= 1.0.0"
  required_providers {
    elasticstack = {
      source  = "elastic/elasticstack"
      version = "0.12.12"
    }
  }
}
provider "elasticstack" {
  elasticsearch {
    username  = "elastic"
    password  = "test"
    endpoints = ["https://elasticstack"]
  }
  kibana {
    endpoints = ["https://elasticstack"]
  }
}

@dm-bil
Copy link

dm-bil commented Dec 19, 2024

facing the same issue, I am on v0.11.11 of the elasticstack provider

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants