ES 7.x: [types removal] Specifying types in bulk requests is deprecated.

[DandyDeveloper]

(check apply)

• read the contribution guideline

Problem

Deprecated types is present in documents managed by latest fluentd version (Possibly the fluentd_elasticsearch plugin):

Same issue described here in fluentbit: fluent/fluent-bit#1359

warning: 299 Elasticsearch-7.8.0-757314695644ea9a1dc2fecd26d1a43856725e65 "[types removal] Specifying types in bulk requests is deprecated."
warning: 299 Elasticsearch-7.8.0-757314695644ea9a1dc2fecd26d1a43856725e65 "[types removal] Specifying types in bulk requests is deprecated."
warning: 299 Elasticsearch-7.8.0-757314695644ea9a1dc2fecd26d1a43856725e65 "[types removal] Specifying types in bulk requests is deprecated."

Steps to replicate

Forward a document via the elasticsearch output plugin on any 7.X instance.

Expected Behavior or What you need to ask

Correct doc type used in bulk requests to ES API.

Using Fluentd and ES plugin versions

• Fluentd 1.11-1 (Docker Image) / Latest ES Plugin

• Bare Metal
• ES version (optional) 7.8

[cosmo0920]

https://github.com/uken/fluent-plugin-elasticsearch#suppress_type_name ?

[DandyDeveloper]

@cosmo0920 ... Yeah that has done the job...

I'll RTFM next time. Thank you!

[happy-machine]

This isnt working for me? fluent/fluentd:v1.11-1 with ws 7.7.1, im getting this error even after i add supress_type_name true for example:

<match blaster.winston*>
  @type elasticsearch
  suppress_type_name true
<\match>

[cosmo0920]

Could you check unused parameter warning is raised from Fluentd on container? Thanks in advance.

[happy-machine]

It's in the container logs for FluentD? Is that what you mean

[cdekok]

Does not work for me either, also the fix should probably not be to suppress the error as in ES v8 the functionality will be removed.
https://www.elastic.co/guide/en/elasticsearch/reference/master/removal-of-types.html#_why_are_mapping_types_being_removed

[moreinhardt]

@cdekok I was scared this only suppresses the warning too. But a look in the code (via related PRs) shows this is handled properly 😉

https://github.com/uken/fluent-plugin-elasticsearch/blob/master/lib/fluent/plugin/out_elasticsearch.rb#L909..L912

[cdekok]

yes i see that's good to know and setting the default es version to 7 seemed to help the error to go away. This was on a slightly older version though perhaps that was the issue.