Advice for action.auto_create_index

[codefromthecrypt]

@varuna mentioned his site disabled action.auto_create_index and thought it would be helpful to document folks to check on this before using Zipkin in a shared cluster. There's some question of where to document that and what to document. For example, this seems to be a normal (non AWS) feature, but not sure the version range it applies to

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#index-creation

Particularly, there's recent activity on this topic elastic/elasticsearch#64208

any advice @openzipkin/elasticsearch?

[xeraa]

That feature has been around at least since 5.0 (though I suspect longer).

The concept does not apply to the new data streams, but you aren't using those yet (I assume).

And the new allow_auto_create allows overriding it with the v2 templates, but any templates should be under your control so it's up to you — otherwise I would expect a lot of weird things to happen if the wrong templates are being applied.

[codefromthecrypt]

hmm maybe I'm misunderstanding. We intentionally create templates, so then I think we would explicitly not want "allow_auto_create", then, based on your description.

[xeraa]

Sorry if I was unclear — trying to juggle too many things in parallel...

allow_auto_create only applies to v2 templates, but I think Zipkin is only using v1 (legacy) templates, right?
So with v1, if you only explicitly create the template but not the indices, auto_create_index: false is a problem. Either you'd need to keep it at true (the default) or switch it to a pattern that allows the Zipkin indices to be created automatically. Unless you can create all Zipkin indices explicitly with a PUT zipkin-index-of-the-day.

Once you switch over to v2 templates, allow_auto_create in the template sounds like the better solution. Though with things still being a bit in motion (allow_auto_create is IMO only coming in Elasticsearch 7.11), it might make sense to wait until we're getting closer to 8.0 to look at the bigger / more complete picture?

As to the original question around documentation: It might make sense to document that action.auto_create_index: false creates a problem the Zipkin tries to create indices.

[codefromthecrypt]

thanks @xeraa

@varuna @jorgheymans one of you want to try to document that the default of "action.auto_create_index: true" should not changed to false?

the other note.. yeah I agree cleaner to do any new template structuring in a major release as for no other reason easier to communicated.