-
Notifications
You must be signed in to change notification settings - Fork 24.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prevent fielddata loading for _id #43599
Comments
Pinging @elastic/es-search |
I ran into this today too. There is no way to disable loading of field data in the jvm heap. Because Either field data should be allowed to be disabled via configuration or field data should always never be loaded for the |
Thanks to @jasontedor and @SpeechlessWick (and others!) for discussing this with us yesterday at the AMA booth. Just adding this note here to echo the need for some sort of protection against the _id field causing fielddata Heap consumption. There have been multiple instances in both Test and Prod clusters where some user has performed some action (we assume this to be a sort or aggregation) which has caused all of our Data nodes in the cluster to consume large amounts of Heap for fielddata. Our only solution at this time is to simply check or audit the fielddata usage for "_id" and then manually clear the cache when/if we see it consuming large amounts. |
I opened #49166 to disallow the loading of the |
This change adds a dynamic cluster setting named `indices.id_field_data.enabled`. When set to `false` any attempt to load the fielddata for the `_id` field will fail with an exception. The default value in this change is set to `false` in order to prevent fielddata usage on this field for future versions but it will be set to `true` when backporting to 7x. When the setting is set to true (manually or by default in 7x) the loading will also issue a deprecation warning since we want to disallow fielddata entirely when elastic#26472 is implemented. Closes elastic#43599
) This change adds a dynamic cluster setting named `indices.id_field_data.enabled`. When set to `false` any attempt to load the fielddata for the `_id` field will fail with an exception. The default value in this change is set to `false` in order to prevent fielddata usage on this field for future versions but it will be set to `true` when backporting to 7x. When the setting is set to true (manually or by default in 7x) the loading will also issue a deprecation warning since we want to disallow fielddata entirely when #26472 is implemented. Closes #43599
) This change adds a dynamic cluster setting named `indices.id_field_data.enabled`. When set to `false` any attempt to load the fielddata for the `_id` field will fail with an exception. The default value in this change is set to `false` in order to prevent fielddata usage on this field for future versions but it will be set to `true` when backporting to 7x. When the setting is set to true (manually or by default in 7x) the loading will also issue a deprecation warning since we want to disallow fielddata entirely when #26472 is implemented. Closes #43599
Describe the feature: Need a way to prevent fielddata memory issues due to aggregating / sorting on
_id
The text was updated successfully, but these errors were encountered: