-
Notifications
You must be signed in to change notification settings - Fork 66
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[REQUEST]: Discourage cloning index templates - offer easier option to customize settings if not using namespaces + Warning/Disclaimer page #1395
Comments
@lucabelluccini to get us started, I've opened this PR to add the new warnings you suggest for the Best practices for integration assets page. Please let me know whatever you think we should change. For the first part, about updating the Customize data retention policies tutorial with two different options, I haven't worked much with component templates, etc., and would really appreciate if you or anyone can suggest exactly what steps should be added/changed. |
Hello @kilfoyle The key is the current guide we have at https://www.elastic.co/guide/en/fleet/current/data-streams-ilm-tutorial.html is focusing on customizing the ILM policy for a specific namespace. We need to discourage it as we want the majority of users want to customize the ILM policy for all namespaces. And customizing the ILM policy for all namespaces should not involve cloning the index template. It's a pitfall a lot of users are following generating very painful recovery procedures. |
If you need assistance please book a slot next week so we can go over it together. |
Thanks @lucabelluccini! I updated my pull request so that it adds a prominent warning in that "Customize data retention policies" tutorial. If you can have a look at my comment here, and let me know if we need to rephrase things, I'd appreciate it. Whatever changes you have in mind I'm happy to add in. |
Description
We're aware of the issues cloning index templates of Fleet integration leads to.
We need to attempt to warn users / push users to not use index template cloning or edit the index & component templates.
The page https://www.elastic.co/guide/en/fleet/current/data-streams-ilm-tutorial.html should be reviewed to display 2 options:
@custom
component template to override the ILM policy or edit the default ILM policy coming with the integration) - point to https://www.elastic.co/guide/en/fleet/current/data-streams.html#data-streams-ilmFurther enhance https://www.elastic.co/guide/en/fleet/current/integrations-assets-best-practices.html
Never edit managed assets of Fleet (e.g. the Index template, the
@package
component templates and/or ingest pipelines bundled with integrations). Any change will be overwritten upon upgrade/install.When the user creates a
@custom
component template for a package integration, while allowed and legit, the user can incur in some risks which might prevent the correct ingestion of data.@package
component template of the integration switches from a "normal" data stream to TSDB or LogsDB and some settings/mappings introduced by the user are not compatible with such indexing modes.keyword
totext
- this might prevent the aggregation on such field on built-in dashboards.When users create a
@custom
ingest pipeline as we document here, Elastic is not responsible of the correct indexing/behavior of the ingest pipeline. It becomes a custom processing of the data and it's user responsibility to test it.When a user clones the index template of an integration package, the user can incur in the risk of not picking up index template major changes upon upgrades. The "structure" of the index template will be "frozen" at the moment it was cloned.
ingest-dev/3617
Cloning index templates to add/inject additional component templates cannot be tested by Elastic, so we cannot guarantee it can work in future release and it can break at any minor release.
Resources
Referenced above
Collaboration
TBD. The docs and product team will work together to determine the best path forward.
Point of contact.
Main contact: @lucabelluccini
I can help with a draft.
The text was updated successfully, but these errors were encountered: