[RFC] Data source categorization field values #954
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: Mathieu Martin <[email protected]>
|
Thanks for opening @jamiehynds - the content looks good for stage 0! We'll need to adjust the commits and branching ( It might be simplest to start fresh with a second PR which we copy/paste your contents from above into it. I'd put some steps together: https://gist.github.com/ebeahan/c479f03f3b84caf895f63b894fed1af9. All the steps are using the GitHub site. We can also run through the steps together some time as well. |
|
Closing this one out, new PR here: #958 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
0000: Data Source Categorization Fields
Elastic currently supports ingestion of data from 180+ sources, and growing. However, we do not have a coherent way to categorise these sources. This has resulted in a disconnect in how we categorize these sources from the Elastic website, in-product experiences and ECS.
The fieldset we use to describe the data source is up for discussion, data_stream.category is a possibility. Here are proposed allowed values:
Usage
Categorization fields in ECS can govern how we categorize these data source, but only a limited set of event.category values are supported by the schema today. The event categorisation fields are catered to individual events, but don't categorise the data source. Expanding the values we support, allows us to align the user experience from ECS, Ingest Manager and the Elastic Website (elastic.co/integrations). Some additional context here: #845 (comment).
These categories could also be used to categorise detection rules, to map data sources to corresponding rules. This would improve our onboarding experience by suggesting detection rules to users based on the sources they are ingesting data from.
People
The following are the people that consulted on the contents of this RFC.
References
#901
#845
RFC Pull Requests