[BUG] Opensearch index template creation fails via terraform

[pastibog]

What is the bug?

Attempting to create an index template via terraform results in an [type=x_content_parse_exception]. The resource definition is the exact one taken from here: https://registry.terraform.io/providers/opensearch-project/opensearch/latest/docs/resources/index_template

How can one reproduce the bug?

Terraform file:

terraform {
  required_providers {
    opensearch = {
      source = "opensearch-project/opensearch"
      version = "2.0.0-beta.1"
    }
  }
}
resource "opensearch_index_template" "template_1" {
  name = "template_1"
  body = <<EOF
{
  "template": "te*",
  "settings": {
    "number_of_shards": 1
  },
  "mappings": {
    "type1": {
      "_source": {
        "enabled": false
      },
      "properties": {
        "host_name": {
          "type": "keyword"
        },
        "created_at": {
          "type": "date",
          "format": "EEE MMM dd HH:mm:ss Z YYYY"
        }
      }
    }
  }
}
EOF
}

Nothing has been changed in the resource definition, it is the exact one from the provider page.

What is the expected behavior?

The index template is created.

What is your host/environment?

AWS OpenSearch 2.7 domain
Terraform v1.4.5
opensearch-project/opensearch 2.0.0-beta.1

[jordarlu]

cc : @phillbaker , and @prudhvigodithi ... if you guys can help take a look . thanks!!

[huziahmetovsv]

Same for me:
Onprem OpenSearch 2.9.0
Terraform v1.5.7
opensearch-project/opensearch 2.0.0-beta.1

[prudhvigodithi]

Hey @pastibog @huziahmetovsv works for me, please check the following main.tf file. I have tested with index_template and composable_index_template.

main.tf

## Import the provider

terraform {
  required_providers {
    opensearch = {
      source = "opensearch-project/opensearch"
      version = "2.0.0-beta.1"
    }
  }
}

## Configure the provider
provider "opensearch" {
  url = "http://127.0.0.1:9200"
  username = "admin"
  password = "admin"
  insecure = true
}

## Create the resource 
resource "opensearch_composable_index_template" "template_1" {
  name = "template_1"
  body = <<EOF
{
  "index_patterns": ["te*", "bar*", "logs*"],
  "template": {
    "settings": {
      "index": {
        "number_of_shards": 1
      }
    },
    "mappings": {
      "properties": {
        "host_name": {
          "type": "keyword"
        },
        "created_at": {
          "type": "date",
          "format": "EEE MMM dd HH:mm:ss Z yyyy"
        }
      }
    },
    "aliases": {
      "mydata": { }
    }
  },
  "priority": 200,
  "version": 3
}
EOF
}

resource "opensearch_index_template" "template_2" {
  name = "template_2"
  body = <<EOF
  {
	"index_patterns": [
	  "logs-2020-01-*"
	],
	"template": {
	  "aliases": {
		"my_logs": {}
	  },
	  "mappings": {
		"properties": {
		  "timestamp": {
			"type": "date",
			"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
		  },
		  "value": {
			"type": "double"
		  }
		}
	  }
	}
  }
EOF
}

API test index_template

curl http://127.0.0.1:9200/_index_template/template_2 -u admin:admin | jq '.'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   292  100   292    0     0    566      0 --:--:-- --:--:-- --:--:--   570
{
  "index_templates": [
    {
      "name": "template_2",
      "index_template": {
        "index_patterns": [
          "logs-2020-01-*"
        ],
        "template": {
          "mappings": {
            "properties": {
              "value": {
                "type": "double"
              },
              "timestamp": {
                "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis",
                "type": "date"
              }
            }
          },
          "aliases": {
            "my_logs": {}
          }
        },
        "composed_of": []
      }
    }
  ]
}

API test composable_index_template

curl http://127.0.0.1:9200/_index_template/template_1 -u admin:admin | jq '.'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   355  100   355    0     0    689      0 --:--:-- --:--:-- --:--:--   692
{
  "index_templates": [
    {
      "name": "template_1",
      "index_template": {
        "index_patterns": [
          "te*",
          "bar*",
          "logs*"
        ],
        "template": {
          "settings": {
            "index": {
              "number_of_shards": "1"
            }
          },
          "mappings": {
            "properties": {
              "created_at": {
                "format": "EEE MMM dd HH:mm:ss Z yyyy",
                "type": "date"
              },
              "host_name": {
                "type": "keyword"
              }
            }
          },
          "aliases": {
            "mydata": {}
          }
        },
        "composed_of": [],
        "priority": 200,
        "version": 3
      }
    }
  ]
}

Some fields have changed with 2.x for index_templates, please refer https://opensearch.org/docs/latest/im-plugin/index-templates/#composable-index-templates.

Adding @phillbaker @bbarani @peterzhuamazon

Thank you

[huziahmetovsv]

Thanx, works for me

[pastibog]

Thanks, will try this.

[prudhvigodithi]

Thanks for the confirmation @huziahmetovsv, closing this issue, @pastibog @huziahmetovsv please feel free re-open if required.
Thank you