[Bug] Error: json: cannot unmarshal array into Go value of type map[string]interface {} #700

Alfaxomo · 2024-08-07T15:23:50Z

Describe the bug
I am trying to create elasticsearch index templates using the elasticstack_elasticsearch_index_template resource. When running terraform plan, it reports that it will create the index templates correctly. However, when running apply I get the following error for all index templates:

│ Error: json: cannot unmarshal array into Go value of type map[string]interface {}
│
│   with elasticstack_elasticsearch_index_template.index_templates["the_index_template_name"],
│   on IndexTemplates.tf line 13, in resource "elasticstack_elasticsearch_index_template" "index_templates":
│   13: resource "elasticstack_elasticsearch_index_template" "index_templates" {
│

To Reproduce
Steps to reproduce the behavior:

TF configuration used '...'

locals.tf

locals {
  features = {
    feature0 = {
      component_template = "true"
      contextkey         = "feature0"
      ilm_policy        = "policy0"
    }
    feature1 = {
      component_template = "true"
      contextkey         = "feature1"
      ilm_policy        = "policy0"
    }
    feature2 = {
      component_template = "true"
      contextkey         = "feature2"
      ilm_policy        = "policy2"
    }
    feature3 = {
      component_template = "false"
      contextkey         = "feature3"
      ilm_policy        = "policy3"
    }
    feature4 = {
      component_template = "true"
      contextkey         = "feature4"
      ilm_policy        = "policy4"
    }
    feature5 = {
      component_template = "false"
      contextkey         = "feature5"
      ilm_policy        = "policy5"
    }
  }
}

index_templates.tf

resource "elasticstack_elasticsearch_component_template" "default_policy_component" {

  name = "policy0"
  template {
    settings = file("component_templates/policy0_settings.json")
    mappings = file("component_templates/policy0_mappings.json")
  }
  metadata = jsonencode({
    description = "Terraform managed resource by DevOps Team"
  })
}

resource "elasticstack_elasticsearch_index_template" "index_templates" {

  for_each = local.features
  
      composed_of = each.value.component_template ? [elasticstack_elasticsearch_component_template.default_policy_component.name] : []
    index_patterns = ["${each.value.contextkey}_*"]
    name           = each.value.contextkey
    template {
      settings = jsonencode({
        index = {
          lifecycle = {
            name = "${each.value.ilm_policy}"
            rollover_alias = "${each.value.contextkey}_write"
          }
        }
      })
      mappings = each.value.component_template == false ? file("index_templates/${each.value.contextkey}_mappings.json") : "[]"
    }

  metadata = jsonencode({
    description = "Terraform managed resource by DevOps Team"
  })

}

TF operations to execute to get the error '...' [e.g terraform plan,terraform apply, terraform destroy]

terraform plan

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # elasticstack_elasticsearch_index_template.index_templates["feature0"] will be created
  + resource "elasticstack_elasticsearch_index_template" "index_templates" {
      + composed_of    = [
          + "policy0",
        ]
      + id             = (known after apply)
      + index_patterns = [
          + "feature0_*",
        ]
      + metadata       = jsonencode(
            {
              + description = "Terraform managed resource by DevOps Team"
            }
        )
      + name           = "feature0"

      + template {
          + mappings = jsonencode([])
          + settings = jsonencode(
                {
                  + index = {
                      + lifecycle = {
                          + name           = "policy0"
                          + rollover_alias = "feature0_write"
                        }
                    }
                }
            )
        }
    }

  # elasticstack_elasticsearch_index_template.index_templates["feature1"] will be created
  + resource "elasticstack_elasticsearch_index_template" "index_templates" {
      + composed_of    = [
          + "policy0",
        ]
      + id             = (known after apply)
      + index_patterns = [
          + "feature1_*",
        ]
      + metadata       = jsonencode(
            {
              + description = "Terraform managed resource by DevOps Team"
            }
        )
      + name           = "feature1"

      + template {
          + mappings = jsonencode([])
          + settings = jsonencode(
                {
                  + index = {
                      + lifecycle = {
                          + name           = "policy0"
                          + rollover_alias = "feature1_write"
                        }
                    }
                }
            )
        }
    }

  # elasticstack_elasticsearch_index_template.index_templates["feature2"] will be created
  + resource "elasticstack_elasticsearch_index_template" "index_templates" {
      + composed_of    = [
          + "policy0",
        ]
      + id             = (known after apply)
      + index_patterns = [
          + "feature2_*",
        ]
      + metadata       = jsonencode(
            {
              + description = "Terraform managed resource by DevOps Team"
            }
        )
      + name           = "feature2"

      + template {
          + mappings = jsonencode([])
          + settings = jsonencode(
                {
                  + index = {
                      + lifecycle = {
                          + name           = "policy2"
                          + rollover_alias = "feature2_write"
                        }
                    }
                }
            )
        }
    }

  # elasticstack_elasticsearch_index_template.index_templates["feature3"] will be created
  + resource "elasticstack_elasticsearch_index_template" "index_templates" {
      + composed_of    = []
      + id             = (known after apply)
      + index_patterns = [
          + "feature3_*",
        ]
      + metadata       = jsonencode(
            {
              + description = "Terraform managed resource by DevOps Team"
            }
        )
      + name           = "feature3"

      + template {
          + mappings = jsonencode([])
          + settings = jsonencode(
                {
                  + index = {
                      + lifecycle = {
                          + name           = "policy3"
                          + rollover_alias = "feature3_write"
                        }
                    }
                }
            )
        }
    }

  # elasticstack_elasticsearch_index_template.index_templates["feature4"] will be created
  + resource "elasticstack_elasticsearch_index_template" "index_templates" {
      + composed_of    = [
          + "policy0",
        ]
      + id             = (known after apply)
      + index_patterns = [
          + "feature4_*",
        ]
      + metadata       = jsonencode(
            {
              + description = "Terraform managed resource by DevOps Team"
            }
        )
      + name           = "feature4"

      + template {
          + mappings = jsonencode([])
          + settings = jsonencode(
                {
                  + index = {
                      + lifecycle = {
                          + name           = "policy4"
                          + rollover_alias = "feature4_write"
                        }
                    }
                }
            )
        }
    }

  # elasticstack_elasticsearch_index_template.index_templates["feature5"] will be created
  + resource "elasticstack_elasticsearch_index_template" "index_templates" {
      + composed_of    = []
      + id             = (known after apply)
      + index_patterns = [
          + "feature5_*",
        ]
      + metadata       = jsonencode(
            {
              + description = "Terraform managed resource by DevOps Team"
            }
        )
      + name           = "feature5"

      + template {
          + mappings = jsonencode([])
          + settings = jsonencode(
                {
                  + index = {
                      + lifecycle = {
                          + name           = "policy5"
                          + rollover_alias = "feature5_write"
                        }
                    }
                }
            )
        }
    }

Plan: 6 to add, 0 to change, 0 to destroy.

────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # elasticstack_elasticsearch_index_template.index_templates["feature0"] will be created
  + resource "elasticstack_elasticsearch_index_template" "index_templates" {
      + composed_of    = [
          + "policy0",
        ]
      + id             = (known after apply)
      + index_patterns = [
          + "feature0_*",
        ]
      + metadata       = jsonencode(
            {
              + description = "Terraform managed resource by DevOps Team"
            }
        )
      + name           = "feature0"

      + template {
          + mappings = jsonencode([])
          + settings = jsonencode(
                {
                  + index = {
                      + lifecycle = {
                          + name           = "policy0"
                          + rollover_alias = "feature0_write"
                        }
                    }
                }
            )
        }
    }

  # elasticstack_elasticsearch_index_template.index_templates["feature1"] will be created
  + resource "elasticstack_elasticsearch_index_template" "index_templates" {
      + composed_of    = [
          + "policy0",
        ]
      + id             = (known after apply)
      + index_patterns = [
          + "feature1_*",
        ]
      + metadata       = jsonencode(
            {
              + description = "Terraform managed resource by DevOps Team"
            }
        )
      + name           = "feature1"

      + template {
          + mappings = jsonencode([])
          + settings = jsonencode(
                {
                  + index = {
                      + lifecycle = {
                          + name           = "policy0"
                          + rollover_alias = "feature1_write"
                        }
                    }
                }
            )
        }
    }

  # elasticstack_elasticsearch_index_template.index_templates["feature2"] will be created
  + resource "elasticstack_elasticsearch_index_template" "index_templates" {
      + composed_of    = [
          + "policy0",
        ]
      + id             = (known after apply)
      + index_patterns = [
          + "feature2_*",
        ]
      + metadata       = jsonencode(
            {
              + description = "Terraform managed resource by DevOps Team"
            }
        )
      + name           = "feature2"

      + template {
          + mappings = jsonencode([])
          + settings = jsonencode(
                {
                  + index = {
                      + lifecycle = {
                          + name           = "policy2"
                          + rollover_alias = "feature2_write"
                        }
                    }
                }
            )
        }
    }

  # elasticstack_elasticsearch_index_template.index_templates["feature3"] will be created
  + resource "elasticstack_elasticsearch_index_template" "index_templates" {
      + composed_of    = []
      + id             = (known after apply)
      + index_patterns = [
          + "feature3_*",
        ]
      + metadata       = jsonencode(
            {
              + description = "Terraform managed resource by DevOps Team"
            }
        )
      + name           = "feature3"

      + template {
          + mappings = jsonencode([])
          + settings = jsonencode(
                {
                  + index = {
                      + lifecycle = {
                          + name           = "policy3"
                          + rollover_alias = "feature3_write"
                        }
                    }
                }
            )
        }
    }

  # elasticstack_elasticsearch_index_template.index_templates["feature4"] will be created
  + resource "elasticstack_elasticsearch_index_template" "index_templates" {
      + composed_of    = [
          + "policy0",
        ]
      + id             = (known after apply)
      + index_patterns = [
          + "feature4_*",
        ]
      + metadata       = jsonencode(
            {
              + description = "Terraform managed resource by DevOps Team"
            }
        )
      + name           = "feature4"

      + template {
          + mappings = jsonencode([])
          + settings = jsonencode(
                {
                  + index = {
                      + lifecycle = {
                          + name           = "policy4"
                          + rollover_alias = "feature4_write"
                        }
                    }
                }
            )
        }
    }

  # elasticstack_elasticsearch_index_template.index_templates["feature5"] will be created
  + resource "elasticstack_elasticsearch_index_template" "index_templates" {
      + composed_of    = []
      + id             = (known after apply)
      + index_patterns = [
          + "feature5_*",
        ]
      + metadata       = jsonencode(
            {
              + description = "Terraform managed resource by DevOps Team"
            }
        )
      + name           = "feature5"

      + template {
          + mappings = jsonencode([])
          + settings = jsonencode(
                {
                  + index = {
                      + lifecycle = {
                          + name           = "policy5"
                          + rollover_alias = "feature5_write"
                        }
                    }
                }
            )
        }
    }

Plan: 6 to add, 0 to change, 0 to destroy.
elasticstack_elasticsearch_index_template.index_templates["feature0"]: Creating...
elasticstack_elasticsearch_index_template.index_templates["feature5"]: Creating...
elasticstack_elasticsearch_index_template.index_templates["feature1"]: Creating...
elasticstack_elasticsearch_index_template.index_templates["feature4"]: Creating...
elasticstack_elasticsearch_index_template.index_templates["feature3"]: Creating...
elasticstack_elasticsearch_index_template.index_templates["feature2"]: Creating...
╷
│ Error: json: cannot unmarshal array into Go value of type map[string]interface {}
│
│   with elasticstack_elasticsearch_index_template.index_templates["feature3"],
│   on IndexTemplates.tf line 13, in resource "elasticstack_elasticsearch_index_template" "index_templates":
│   13: resource "elasticstack_elasticsearch_index_template" "index_templates" {
│
╵
╷
│ Error: json: cannot unmarshal array into Go value of type map[string]interface {}
│
│   with elasticstack_elasticsearch_index_template.index_templates["feature1"],
│   on IndexTemplates.tf line 13, in resource "elasticstack_elasticsearch_index_template" "index_templates":
│   13: resource "elasticstack_elasticsearch_index_template" "index_templates" {
│
╵
╷
│ Error: json: cannot unmarshal array into Go value of type map[string]interface {}
│
│   with elasticstack_elasticsearch_index_template.index_templates["feature4"],
│   on IndexTemplates.tf line 13, in resource "elasticstack_elasticsearch_index_template" "index_templates":
│   13: resource "elasticstack_elasticsearch_index_template" "index_templates" {
│
╵
╷
│ Error: json: cannot unmarshal array into Go value of type map[string]interface {}
│
│   with elasticstack_elasticsearch_index_template.index_templates["feature2"],
│   on IndexTemplates.tf line 13, in resource "elasticstack_elasticsearch_index_template" "index_templates":
│   13: resource "elasticstack_elasticsearch_index_template" "index_templates" {
│
╵
╷
│ Error: json: cannot unmarshal array into Go value of type map[string]interface {}
│
│   with elasticstack_elasticsearch_index_template.index_templates["feature0"],
│   on IndexTemplates.tf line 13, in resource "elasticstack_elasticsearch_index_template" "index_templates":
│   13: resource "elasticstack_elasticsearch_index_template" "index_templates" {
│
╵
╷
│ Error: json: cannot unmarshal array into Go value of type map[string]interface {}
│
│   with elasticstack_elasticsearch_index_template.index_templates["feature5"],
│   on IndexTemplates.tf line 13, in resource "elasticstack_elasticsearch_index_template" "index_templates":
│   13: resource "elasticstack_elasticsearch_index_template" "index_templates" {
│
╵

See the error in the output '...'

Expected behavior
Creation of index templates as outlined in the plan and apply output

Debug output

2024-08-07T11:11:43.145-0400 [ERROR] provider.terraform-provider-elasticstack_v0.11.4: Response contains error diagnostic: tf_proto_version=6.6 tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_req_id=8f12acff-d24f-c971-a85e-c00d549755b2 tf_resource_type=elasticstack_elasticsearch_index_template tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/[email protected]/tfprotov6/internal/diag/diagnostics.go:58 @module=sdk.proto diagnostic_detail= diagnostic_severity=ERROR diagnostic_summary="json: cannot unmarshal array into Go value of type map[string]interface {}" timestamp=2024-08-07T11:11:43.144-0400
2024-08-07T11:11:43.152-0400 [ERROR] provider.terraform-provider-elasticstack_v0.11.4: Response contains error diagnostic: tf_rpc=ApplyResourceChange @module=sdk.proto diagnostic_detail= tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_req_id=4073dd4f-96c6-d0c0-1f5a-e357c5464b5a tf_proto_version=6.6 tf_resource_type=elasticstack_elasticsearch_index_template @caller=github.com/hashicorp/[email protected]/tfprotov6/internal/diag/diagnostics.go:58 diagnostic_severity=ERROR diagnostic_summary="json: cannot unmarshal array into Go value of type map[string]interface {}" timestamp=2024-08-07T11:11:43.152-0400
2024-08-07T11:11:43.158-0400 [ERROR] provider.terraform-provider-elasticstack_v0.11.4: Response contains error diagnostic: tf_proto_version=6.6 diagnostic_summary="json: cannot unmarshal array into Go value of type map[string]interface {}" @caller=github.com/hashicorp/[email protected]/tfprotov6/internal/diag/diagnostics.go:58 @module=sdk.proto diagnostic_detail= diagnostic_severity=ERROR tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_req_id=462046c6-9fd7-6c0c-df93-3b5ef5ccc321 tf_resource_type=elasticstack_elasticsearch_index_template tf_rpc=ApplyResourceChange timestamp=2024-08-07T11:11:43.158-0400
2024-08-07T11:11:43.160-0400 [ERROR] provider.terraform-provider-elasticstack_v0.11.4: Response contains error diagnostic: @caller=github.com/hashicorp/[email protected]/tfprotov6/internal/diag/diagnostics.go:58 diagnostic_summary="json: cannot unmarshal array into Go value of type map[string]interface {}" tf_req_id=dc445c01-7dbe-7f7a-276e-e5576b575d78 tf_rpc=ApplyResourceChange tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_resource_type=elasticstack_elasticsearch_index_template @module=sdk.proto diagnostic_detail= diagnostic_severity=ERROR tf_proto_version=6.6 timestamp=2024-08-07T11:11:43.159-0400
2024-08-07T11:11:43.164-0400 [ERROR] vertex "elasticstack_elasticsearch_index_template.index_templates[\"feature4\"]" error: json: cannot unmarshal array into Go value of type map[string]interface {}
2024-08-07T11:11:43.169-0400 [ERROR] vertex "elasticstack_elasticsearch_index_template.index_templates[\"feature1\"]" error: json: cannot unmarshal array into Go value of type map[string]interface {}
2024-08-07T11:11:43.175-0400 [ERROR] vertex "elasticstack_elasticsearch_index_template.index_templates[\"feature2\"]" error: json: cannot unmarshal array into Go value of type map[string]interface {}
2024-08-07T11:11:43.178-0400 [ERROR] provider.terraform-provider-elasticstack_v0.11.4: Response contains error diagnostic: diagnostic_summary="json: cannot unmarshal array into Go value of type map[string]interface {}" tf_req_id=2d54c8ae-1fac-e1d9-dcd1-412789e5dca2 tf_resource_type=elasticstack_elasticsearch_index_template tf_rpc=ApplyResourceChange tf_provider_addr=registry.terraform.io/elastic/elasticstack @caller=github.com/hashicorp/[email protected]/tfprotov6/internal/diag/diagnostics.go:58 @module=sdk.proto diagnostic_detail= diagnostic_severity=ERROR tf_proto_version=6.6 timestamp=2024-08-07T11:11:43.178-0400
2024-08-07T11:11:43.179-0400 [ERROR] provider.terraform-provider-elasticstack_v0.11.4: Response contains error diagnostic: diagnostic_severity=ERROR diagnostic_summary="json: cannot unmarshal array into Go value of type map[string]interface {}" tf_proto_version=6.6 tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_rpc=ApplyResourceChange tf_req_id=5e038265-e75a-c31d-952e-e183a0485a49 tf_resource_type=elasticstack_elasticsearch_index_template @caller=github.com/hashicorp/[email protected]/tfprotov6/internal/diag/diagnostics.go:58 @module=sdk.proto diagnostic_detail= timestamp=2024-08-07T11:11:43.178-0400
2024-08-07T11:11:43.181-0400 [ERROR] vertex "elasticstack_elasticsearch_index_template.index_templates[\"feature5\"]" error: json: cannot unmarshal array into Go value of type map[string]interface {}
2024-08-07T11:11:43.187-0400 [ERROR] vertex "elasticstack_elasticsearch_index_template.index_templates[\"feature0\"]" error: json: cannot unmarshal array into Go value of type map[string]interface {}
2024-08-07T11:11:43.192-0400 [ERROR] vertex "elasticstack_elasticsearch_index_template.index_templates[\"feature3\"]" error: json: cannot unmarshal array into Go value of type map[string]interface {}

Screenshots
If applicable, add screenshots to help explain your problem.

Versions (please complete the following information):

OS: Ubuntu Linux (WSL) bookworm/sid
Terraform Version: v1.3.7
Provider version: 0.11.4
Elasticsearch Version: 8.14.3

Additional context
Add any other context about the problem here.

The text was updated successfully, but these errors were encountered:

tobio · 2024-08-20T10:42:09Z

@Alfaxomo each of these template definitions has mappings = jsonencode([]). Mappings must be a JSON object, not an array. You can get past the error you're seeing by changing this to mappings = jsonencode({}) or just completely removing the mappings field altogether.

I'm opening a PR to improve the validation here to hopefully make the cause clearer.

Alfaxomo · 2024-08-21T19:40:28Z

@tobio thank you!

Alfaxomo added the bug Something isn't working label Aug 7, 2024

tobio mentioned this issue Aug 20, 2024

Validate that mappings are a JSON object, not just valid json #719

Merged

tobio closed this as completed in #719 Aug 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug] Error: json: cannot unmarshal array into Go value of type map[string]interface {} #700

[Bug] Error: json: cannot unmarshal array into Go value of type map[string]interface {} #700

Alfaxomo commented Aug 7, 2024

tobio commented Aug 20, 2024

Alfaxomo commented Aug 21, 2024

[Bug] Error: json: cannot unmarshal array into Go value of type map[string]interface {} #700

[Bug] Error: json: cannot unmarshal array into Go value of type map[string]interface {} #700

Comments

Alfaxomo commented Aug 7, 2024

locals.tf

index_templates.tf

terraform plan

terraform apply

tobio commented Aug 20, 2024

Alfaxomo commented Aug 21, 2024