-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Fleet] Packages with custom dataset use the fallback index templates #160775
Comments
Pinging @elastic/fleet (Team:Fleet) |
Any updates on this? I had a support case opened, and they pointed me here, but this looks to have gone stale. |
@MakoWish this isn't stale, it has been prioritised in on of our upcoming sprint but we have other priorities to deal with for now. |
I think the solution here is to move to input packages where this problem is solved, I can see there is a |
Isn't it a bug in fleet that the package's index template is not used when custom dataset is used? If so, probably we should fix for all packages. |
It is a bug, or just a bad feature I am not sure, dataset customisation has always been quite broken in integrations. But Input packages were created to allow users to customize the destination of their data, if we want to allow integrations to customize dataset and have the index templates match it would be a new feature, one which I don't think we should invest time in when input packages exist. |
The |
@MakoWish sorry I missed this. Custom Logs has been moved to an input type package, this means that fleet creates the index template dynamically for the given dataset. Whereas Windows event logs does not. I have created an issue to move custom windows logs to an input package here elastic/integrations#7820 |
Even if we migrate these packages to input packages, this is still an issue for integration packages, right? Or should we disallow the use of custom datasets in integration packages? |
Kibana version:
Seen at least in 8.6, 8.7 and 8.8.
Describe the bug:
When the data stream of an integration package is configured to use a custom dataset, it may end up using the fallback index templates for logs and metrics, instead of the template installed with the package, missing important mappings or metadata.
This can cause issues like:
@custom
pipeline is referenced, so no custom processing can be added.This is confirmed at least with:
logs.logs
.prometheus.collector
.winlog.winlog
.Steps to reproduce:
<package>.<data_stream>
.logs
ormetrics
built-in data streams.Expected behavior:
The data stream uses the index template installed with the package. This is what happens with input packages or with integration packages that don't use custom datasets.
Screenshots (if relevant):
The text was updated successfully, but these errors were encountered: