[Security Solution] Migrate to fields API #136163
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
YulNaumenko
changed the title
YulNaumenko
added
v8.4.0
release_note:skip
…-ref HEAD~1..HEAD --fix'
…m/YulNaumenko/kibana into security-migrate-to-fields-api-pr
andrew-goldstein
approved these changes
Jul 12, 2022
semd
approved these changes
Jul 12, 2022
jamster10
approved these changes
Jul 12, 2022
MadameSheema
approved these changes
Jul 12, 2022
💛 Build succeeded, but was flakyFailed CI StepsTest Failures
Metrics [docs]Module Count
Async chunks
Page load bundle
History
To update your PR or re-run it, just comment with: cc @YulNaumenko |
dhurley14
approved these changes
Jul 12, 2022
4 tasks
kibanamachine
added a commit
that referenced
this pull request
Nov 28, 2022
…tions to use parameters (#145889) (#146414) # Backport This will backport the following commits from `main` to `8.5`: - [[Security Solution][Investigations][Timeline] - Update getExceptions to use parameters (#145889)](#145889) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Michael Olorunnisola","email":"[email protected]"},"sourceCommit":{"committedDate":"2022-11-28T15:08:48Z","message":"[Security Solution][Investigations][Timeline] - Update getExceptions to use parameters (#145889)\n\n## Summary\r\n\r\nFixes: https://github.com/elastic/kibana/issues/136772\r\n\r\nThe issue was introduced by a couple of changes:\r\n\r\nFirst:\r\nhttps://github.com//pull/136163/files#diff-02d33a1ed6679f7775dc01941ca21b085d7c008ecffe5e029f5967407a5e5b13L23\r\nin 8.4.\r\n\r\nThe bug: A filter on the timeline UI relied on the `exceptions_list`\r\nfield provided on `_source` to auto-generate a filter when investigating\r\nin timeline labelled `Not Exceptions` which would filter out the\r\nexceptions from the timeline. This PR resolves that issue by pulling the\r\n`exceptions_list` field from `kibana.alert.rule.parameters`.\r\n\r\nSecond:\r\nhttps://github.com//pull/133254/files#diff-0f69b69fd9cefef6ed04a048d7df86b7e385e816bdf17309212437dc3f69726cL74\r\n\r\nThe filter actually stopped being passed to timeline entirely because of\r\nthe above change.\r\n\r\nWith the fixes in place:\r\n\r\n\r\nhttps://user-images.githubusercontent.com/17211684/203111748-7a0c2eb5-a46f-4f88-9d77-3628204625ac.mov","sha":"b32c8b9df89188cdcb149bd1d9494d3f99999ad6","branchLabelMapping":{"^v8.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","backport","release_note:fix","Team:Threat Hunting:Investigations","v8.5.0","v8.6.0","v8.7.0"],"number":145889,"url":"https://github.com/elastic/kibana/pull/145889","mergeCommit":{"message":"[Security Solution][Investigations][Timeline] - Update getExceptions to use parameters (#145889)\n\n## Summary\r\n\r\nFixes: https://github.com/elastic/kibana/issues/136772\r\n\r\nThe issue was introduced by a couple of changes:\r\n\r\nFirst:\r\nhttps://github.com//pull/136163/files#diff-02d33a1ed6679f7775dc01941ca21b085d7c008ecffe5e029f5967407a5e5b13L23\r\nin 8.4.\r\n\r\nThe bug: A filter on the timeline UI relied on the `exceptions_list`\r\nfield provided on `_source` to auto-generate a filter when investigating\r\nin timeline labelled `Not Exceptions` which would filter out the\r\nexceptions from the timeline. This PR resolves that issue by pulling the\r\n`exceptions_list` field from `kibana.alert.rule.parameters`.\r\n\r\nSecond:\r\nhttps://github.com//pull/133254/files#diff-0f69b69fd9cefef6ed04a048d7df86b7e385e816bdf17309212437dc3f69726cL74\r\n\r\nThe filter actually stopped being passed to timeline entirely because of\r\nthe above change.\r\n\r\nWith the fixes in place:\r\n\r\n\r\nhttps://user-images.githubusercontent.com/17211684/203111748-7a0c2eb5-a46f-4f88-9d77-3628204625ac.mov","sha":"b32c8b9df89188cdcb149bd1d9494d3f99999ad6"}},"sourceBranch":"main","suggestedTargetBranches":["8.5","8.6"],"targetPullRequestStates":[{"branch":"8.5","label":"v8.5.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.6","label":"v8.6.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.7.0","labelRegex":"^v8.7.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/145889","number":145889,"mergeCommit":{"message":"[Security Solution][Investigations][Timeline] - Update getExceptions to use parameters (#145889)\n\n## Summary\r\n\r\nFixes: https://github.com/elastic/kibana/issues/136772\r\n\r\nThe issue was introduced by a couple of changes:\r\n\r\nFirst:\r\nhttps://github.com//pull/136163/files#diff-02d33a1ed6679f7775dc01941ca21b085d7c008ecffe5e029f5967407a5e5b13L23\r\nin 8.4.\r\n\r\nThe bug: A filter on the timeline UI relied on the `exceptions_list`\r\nfield provided on `_source` to auto-generate a filter when investigating\r\nin timeline labelled `Not Exceptions` which would filter out the\r\nexceptions from the timeline. This PR resolves that issue by pulling the\r\n`exceptions_list` field from `kibana.alert.rule.parameters`.\r\n\r\nSecond:\r\nhttps://github.com//pull/133254/files#diff-0f69b69fd9cefef6ed04a048d7df86b7e385e816bdf17309212437dc3f69726cL74\r\n\r\nThe filter actually stopped being passed to timeline entirely because of\r\nthe above change.\r\n\r\nWith the fixes in place:\r\n\r\n\r\nhttps://user-images.githubusercontent.com/17211684/203111748-7a0c2eb5-a46f-4f88-9d77-3628204625ac.mov","sha":"b32c8b9df89188cdcb149bd1d9494d3f99999ad6"}}]}] BACKPORT--> Co-authored-by: Michael Olorunnisola <[email protected]>
kibanamachine
added a commit
that referenced
this pull request
Nov 28, 2022
…tions to use parameters (#145889) (#146415) # Backport This will backport the following commits from `main` to `8.6`: - [[Security Solution][Investigations][Timeline] - Update getExceptions to use parameters (#145889)](#145889) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Michael Olorunnisola","email":"[email protected]"},"sourceCommit":{"committedDate":"2022-11-28T15:08:48Z","message":"[Security Solution][Investigations][Timeline] - Update getExceptions to use parameters (#145889)\n\n## Summary\r\n\r\nFixes: https://github.com/elastic/kibana/issues/136772\r\n\r\nThe issue was introduced by a couple of changes:\r\n\r\nFirst:\r\nhttps://github.com//pull/136163/files#diff-02d33a1ed6679f7775dc01941ca21b085d7c008ecffe5e029f5967407a5e5b13L23\r\nin 8.4.\r\n\r\nThe bug: A filter on the timeline UI relied on the `exceptions_list`\r\nfield provided on `_source` to auto-generate a filter when investigating\r\nin timeline labelled `Not Exceptions` which would filter out the\r\nexceptions from the timeline. This PR resolves that issue by pulling the\r\n`exceptions_list` field from `kibana.alert.rule.parameters`.\r\n\r\nSecond:\r\nhttps://github.com//pull/133254/files#diff-0f69b69fd9cefef6ed04a048d7df86b7e385e816bdf17309212437dc3f69726cL74\r\n\r\nThe filter actually stopped being passed to timeline entirely because of\r\nthe above change.\r\n\r\nWith the fixes in place:\r\n\r\n\r\nhttps://user-images.githubusercontent.com/17211684/203111748-7a0c2eb5-a46f-4f88-9d77-3628204625ac.mov","sha":"b32c8b9df89188cdcb149bd1d9494d3f99999ad6","branchLabelMapping":{"^v8.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","backport","release_note:fix","Team:Threat Hunting:Investigations","v8.5.0","v8.6.0","v8.7.0"],"number":145889,"url":"https://github.com/elastic/kibana/pull/145889","mergeCommit":{"message":"[Security Solution][Investigations][Timeline] - Update getExceptions to use parameters (#145889)\n\n## Summary\r\n\r\nFixes: https://github.com/elastic/kibana/issues/136772\r\n\r\nThe issue was introduced by a couple of changes:\r\n\r\nFirst:\r\nhttps://github.com//pull/136163/files#diff-02d33a1ed6679f7775dc01941ca21b085d7c008ecffe5e029f5967407a5e5b13L23\r\nin 8.4.\r\n\r\nThe bug: A filter on the timeline UI relied on the `exceptions_list`\r\nfield provided on `_source` to auto-generate a filter when investigating\r\nin timeline labelled `Not Exceptions` which would filter out the\r\nexceptions from the timeline. This PR resolves that issue by pulling the\r\n`exceptions_list` field from `kibana.alert.rule.parameters`.\r\n\r\nSecond:\r\nhttps://github.com//pull/133254/files#diff-0f69b69fd9cefef6ed04a048d7df86b7e385e816bdf17309212437dc3f69726cL74\r\n\r\nThe filter actually stopped being passed to timeline entirely because of\r\nthe above change.\r\n\r\nWith the fixes in place:\r\n\r\n\r\nhttps://user-images.githubusercontent.com/17211684/203111748-7a0c2eb5-a46f-4f88-9d77-3628204625ac.mov","sha":"b32c8b9df89188cdcb149bd1d9494d3f99999ad6"}},"sourceBranch":"main","suggestedTargetBranches":["8.5","8.6"],"targetPullRequestStates":[{"branch":"8.5","label":"v8.5.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.6","label":"v8.6.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.7.0","labelRegex":"^v8.7.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/145889","number":145889,"mergeCommit":{"message":"[Security Solution][Investigations][Timeline] - Update getExceptions to use parameters (#145889)\n\n## Summary\r\n\r\nFixes: https://github.com/elastic/kibana/issues/136772\r\n\r\nThe issue was introduced by a couple of changes:\r\n\r\nFirst:\r\nhttps://github.com//pull/136163/files#diff-02d33a1ed6679f7775dc01941ca21b085d7c008ecffe5e029f5967407a5e5b13L23\r\nin 8.4.\r\n\r\nThe bug: A filter on the timeline UI relied on the `exceptions_list`\r\nfield provided on `_source` to auto-generate a filter when investigating\r\nin timeline labelled `Not Exceptions` which would filter out the\r\nexceptions from the timeline. This PR resolves that issue by pulling the\r\n`exceptions_list` field from `kibana.alert.rule.parameters`.\r\n\r\nSecond:\r\nhttps://github.com//pull/133254/files#diff-0f69b69fd9cefef6ed04a048d7df86b7e385e816bdf17309212437dc3f69726cL74\r\n\r\nThe filter actually stopped being passed to timeline entirely because of\r\nthe above change.\r\n\r\nWith the fixes in place:\r\n\r\n\r\nhttps://user-images.githubusercontent.com/17211684/203111748-7a0c2eb5-a46f-4f88-9d77-3628204625ac.mov","sha":"b32c8b9df89188cdcb149bd1d9494d3f99999ad6"}}]}] BACKPORT--> Co-authored-by: Michael Olorunnisola <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The things were done under the current PR:
Replaces
_sourceusage with fields API for the next files queries:x-pack/plugins/timelines/server/search_strategy/timeline/factory/events/all/query.events_all.dsl.tsx-pack/plugins/timelines/server/search_strategy/timeline/factory/events/details/query.events_details.dsl.tsx-pack/plugins/timelines/server/search_strategy/timeline/factory/events/last_event_time/query.events_last_event_time.dsl.tsImpact:
current query is used by
useTimelineLastEventTimeandLastEventTimecomponent.Extended CTI query with
timestampformat definition andinclude_unmappedfieldsx-pack/plugins/security_solution/server/search_strategy/security_solution/factory/cti/event_enrichment/query.ts, removeddocValueFieldsusage. Addedstored_fields: ['*']as a bonus :-)Cleaned up unused plugin router registration in
x-pack/plugins/timelines/server/plugin.tsand deleted filex-pack/plugins/timelines/server/routes/index.tsRemoved
docValueFieldsin favor of usage a new fields API in the next files related to this PR (those files changes is docValueFields cleanup only):x-pack/plugins/security_solution/public/cases/pages/index.tsxx-pack/plugins/security_solution/public/common/components/events_viewer/index.tsxx-pack/plugins/security_solution/public/common/components/last_event_time/index.tsxx-pack/plugins/security_solution/public/common/containers/events/last_event_time/index.tsx-pack/plugins/security_solution/public/detections/components/alerts_table/actions.tsxx-pack/plugins/security_solution/public/detections/components/rules/rule_preview/preview_histogram.tsxx-pack/plugins/security_solution/public/hosts/pages/details/index.tsxx-pack/plugins/security_solution/public/hosts/pages/hosts.tsxx-pack/plugins/security_solution/public/network/pages/network.tsxx-pack/plugins/security_solution/public/timelines/components/side_panel/event_details/index.tsxx-pack/plugins/security_solution/public/timelines/components/side_panel/hooks/use_detail_panel.tsxx-pack/plugins/security_solution/public/timelines/components/side_panel/index.tsxx-pack/plugins/security_solution/public/timelines/components/timeline/eql_tab_content/index.tsxx-pack/plugins/security_solution/public/timelines/components/timeline/notes_tab_content/index.tsxx-pack/plugins/security_solution/public/timelines/components/timeline/pinned_tab_content/index.tsxx-pack/plugins/security_solution/public/timelines/components/timeline/query_tab_content/index.tsxx-pack/plugins/security_solution/public/timelines/containers/details/index.tsxx-pack/plugins/security_solution/public/timelines/containers/index.tsxx-pack/plugins/security_solution/public/users/pages/details/index.tsxx-pack/plugins/security_solution/public/users/pages/details/types.tsx-pack/plugins/security_solution/public/users/pages/users.tsxx-pack/plugins/timelines/common/search_strategy/timeline/index.tsx-pack/plugins/timelines/public/components/t_grid/integrated/index.tsxx-pack/plugins/timelines/public/components/t_grid/standalone/index.tsxx-pack/plugins/timelines/public/container/index.tsxx-pack/plugins/timelines/public/mock/t_grid.tsxOpened a follow up issue to clean up
docValueFieldsacross the rest ofsecurity_solutionandtimelinesplugins.Removed function
getDataFromSourceHitsfromx-pack/plugins/timelines/common/utils/field_formatters.ts, x-pack/plugins/security_solution/common/utils/field_formatters.tsand replaced it's usage withgetDataFromFieldsHits. Still have this function duplicated but this will be changed by migrating to the new table soon.For
x-pack/plugins/timelines/server/search_strategy/timeline/factory/events/details/query.events_details.dsl.tschanged thethreat.enrichmentsrepresentation from string like JSON stringify to multipleTimelineEventsDetailsItemobjects. This helped to unify this representation between table and details/flyout page.All the data without mappings (ecsFieldMap, technicalRuleFieldMap or experimentalRuleFieldMap) defined will be parsed now as a string objects array: