elasticsearch.config.plugins does not work

[arbourd]

example.tf

  elasticsearch {
    config {
      plugins = ["analysis-icu"]
    }

    topology {
      size       = "1g"
      zone_count = 2
    }
  }

terrform diff

~ elasticsearch {
      # (7 unchanged attributes hidden)

    + config {
        + plugins = [
            + "analysis-icu",
          ]
      }

      # (1 unchanged block hidden)
  }

Readiness Checklist

• I am running the latest version
• I checked the documentation and found no answer
• I checked to make sure that this issue has not already been filed
• I am reporting the issue to the correct repository (for multi-repository projects)

Expected Behavior

Adds analysis-icu like it says it does.

Current Behavior

Doesn't add analysis-icu, and if it already is present on the cluster, it will remove it.

Your Environment

• Version used: 0.1.0-beta
  I did not build from source, this is your binary running in Terraform Cloud.

[arbourd]

It seems that the topology specific option is not being set, but the overall stack option is. How do we set the topology specific option?

[arbourd]

Actually, it appears that after recreating the topology, analysis-icu is there but not in the GUI. I imagine because the frontend looks at a different chunk of the API response.

[wouter-toppy]

@arbourd i have the same issue, its not setting it on the the instance. how did you solve this?

[arbourd]

@wouter-toppy are you using the Terraform provider?

If you create a new instance with the TF provider and it is properly set it will work but it won't appear on the web UI.

[wouter-toppy]

@arbourd

im using this:

terraform {
  required_version = ">= 0.13"
  required_providers {
    ec = {
      source  = "elastic/ec"
      version = "0.1.0-beta"
    }
  }
}

provider "ec" {}

resource "ec_deployment" "elastic_search" {
  name      = "${local.prefix}-${lower(var.environment)}-deployment"

  version                 = var.elasticsearch_version
  region                  = var.elasticsearch_region
  deployment_template_id  = var.elasticsearch_deployment_template_id

  elasticsearch {
    topology {
      instance_configuration_id = var.elasticsearch_instance_configuration_id
      size                      = var.elasticsearch_size
      zone_count                = var.elasticsearch_zone_count
    }

    config {
      plugins = var.elasticsearch_plugins
    }
  }
}

where my vars are these:

  elasticsearch_version                   = "7.11.0"

  elasticsearch_region                    = "gcp-europe-west4"
  elasticsearch_deployment_template_id    = "gcp-io-optimized"
  elasticsearch_instance_configuration_id = "gcp.data.highio.1"

  elasticsearch_size                      = "2g"
  elasticsearch_zone_count                = "1"
  elasticsearch_plugins                   = [
    "analysis-icu",
    "analysis-phonetic",
    "ingest-geoip",
    "ingest-user-agent",
  ]

but if its true what you said that the plugins are enables but not visible in the GUI its fine for now. is there any way to verify that the plugins are actually enabled?

[arbourd]

is there any way to verify that the plugins are actually enabled?

you can make an API call directly to get the settings. the easiest way i found to do this is the Equivalent API request button when editing a deployment. It will show you a JSON blob with the current settings. In our case:

"elasticsearch": {
  "version": "6.8.14",
  "enabled_built_in_plugins": [
    "analysis-icu"
  ]
}

[wouter-toppy]

@arbourd yeah, I already saw that entry there, but when I set the plugins via the GUI is see the plugin namens on more places in the json, but I guess than its working propertly ;-)

Thanks for the explaination ;-)

[arbourd]

According to this comment the provider sets configuration at a global space, but the UI does it per instance. So, if you edit it on the GUI it reapplies with per-instance configuration everywhere.

I think the fix they might do is improve the UI so our changes are reflected.