-
Notifications
You must be signed in to change notification settings - Fork 306
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Specifying types in bulk requests is deprecated #915
Comments
Currently at LS 7.10, end the ES (also 7.10) logs are flouding my filesystem. No idea what the previous comment means. The flouding ("[types removal] Specifying types in bulk requests is deprecated.") started after upgrading ES from 7.9 to 7.10. |
I have the same problem after upgrading to 7.10 |
Same problem here. Flooding can be stopped by changing log level of deprecation warnings. From: https://www.elastic.co/guide/en/elasticsearch/reference/current/logging.html You can disable it in the config/log4j2.properties file by setting the deprecation log level to error like this:
This is merely a workaround and I would appreciate a fix. |
Any updates on this issue? |
Could you share some information about this problem please, which |
Hi @andsel, We are sending Beats data through Logstash for some additional data massaging. The entire stack is 7.10.2. That includes Beats, Logstash, Elasticsearch, Kibana... Beats output:
Logstash output:
Nearly all our data is from Beats modules, using the included Beats index templates and pipelines. None of our non-Beats sources have a "_type" field explicitly added. It seems (I don't know where it is being added to prove) either Beats or Logstash are adding a field:value of "_type" : "_doc" to every event ingested. Basically, I am not adding a "_type" field anywhere, so it must be part of the stack itself? Would throwing something in the Logstash pipeline like this do the trick?
Just to be 100% sure I wasn't adding a "_type" field anywhere, I ran Here is just few of the literally millions of these events we receive per day. Even the event saying not to use the "_type" field is for some reason still using the "_type" field, LOL! Again, I am not adding this myself. |
UPDATE: I tried removing the "_type" field in my pipeline as noted above, so that should rule out the Beats agent adding it, but it is still in every event. |
@MakoWish |
It's assigned by
with value:
|
Thank you for the response! Will that be disabled automatically with v8? Or is there a way to disable it now? I would like to stop the literally millions of messages we receive daily. |
A reproducer for this:
|
Logstash output Elasticsearch plugin uses the As a fix on Elasticsearch side, instead of changing the level of deprecation logger:
|
Reference to Elasticsearch issue elastic/elasticsearch#69188 |
…is printed for each request (index/bulk) that contains the deprecated index _type. This commit removes the implicit set of type when connected to Elasticsearch 7.x unless the user doesn't specify explicitly the type. PR logstash-plugins#994 Fixes logstash-plugins#915
This has been fixed with version |
Version: logstash-output-elasticsearch 10.2.3 (shipping with Logstash 7.5.2)
Starting from Elasticsearch 7.0, types should also no longer appear in the body of bulk requests.
Including a type causes a deprecation warning to get logged in Elasticsearch:
The plugin shouldn't use types anymore.
The text was updated successfully, but these errors were encountered: