[Security Solution] Timeline fails to show hits for docs missing chosen sort by fields #129603
Assignees
Labels
bug
Fixes for quality problems that affect the customer experience
Feature:Timeline
Security Solution Timeline feature
impact:high
Addressing this issue will have a high level of impact on the quality/strength of our product.
QA:Validated
Issue has been validated by QA
Team: SecuritySolution
Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.
Team:Threat Hunting:Investigations
Security Solution Investigations Team
v8.1.0
Comments
dainperkins
added
bug
MikePaquette
added
the
Team: SecuritySolution
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
MikePaquette
added
the
impact:high
Summary@dainperkins
Details
The failures in the attached zip are also displayed in the
The There's no error toaster in this scenario, because Elasticsearch "failed open" and returned a To that end, there's an open UX issue for displaying an indicator whenever partial results are displayed for all queries, including Timeline, here: https://github.com/elastic/security-team/issues/1244 |
|
Excellent - that definitely addresses the issue based on the dev-tools runs of the various tests I had been using from the timeline queries |
MindyRS
added
the
Team:Threat Hunting:Investigations
|
To ensure the shard failures described in this issue are reproducible when verifying the fix, please prepare the environment by executing the following steps:
Expected result
Actual result
Expected result
Actual result
Expected result
"sort": [
{
"@timestamp": {
"order": "desc",
"unmapped_type": "number"
}
},
{
"event.action": {
"order": "desc",
"unmapped_type": "keyword"
}
}
],Actual result
"sort": [
{
"@timestamp": {
"order": "desc",
"unmapped_type": "number"
}
},
{
"event.action": {
"order": "desc",
"unmapped_type": "string"
}
}
],
Expected result
Actual result
per the example below: {
"took": 1,
"timed_out": false,
"_shards": {
"total": 4,
"successful": 3,
"skipped": 0,
"failed": 1,
"failures": [
{
"shard": 0,
"index": "logs-ti_test",
"node": "NCRcGeDqSlKQiuPWVFvMEg",
"reason": {
"type": "illegal_argument_exception",
"reason": "No mapper found for type [string]"
}
}
]
}, |
andrew-goldstein
added a commit
to andrew-goldstein/kibana
that referenced
this issue
May 16, 2022
… in timelines and the events / alerts tables: - <elastic#129603> - <elastic#123912> - <elastic#131625> The `unmapped_type` property [addition](https://github.com/elastic/kibana/pull/87241/files#diff-52fd5870dcd5f783f9fc8ac3a18a8674d83ac6136e09fe0e0bcae30427d61c3fR55) to the `sort` parameter of requests was using the `type` field metadata from `BrowserFields`, but the `type` metadata (for some fields) contains the value `string`, which is not a [valid field data type](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html). The fix for the issues above: - Populates the `sort` property of requests with values from the `esTypes` `BrowserFields` metadata (instead of `type`) - The `esTypes` metadata may specify more than one field value type. When `esTypes` contains more than one type, and `keyword` is one of the types, the `sort` property of the request will prefer `keyword` over other the other types - When the field metadata has an empty `esTypes` collection, the `sort` property of the request will default to using `"unmapped_type": "keyword"` - The field type displayed in tooltips when hovering over columns in a timeline now displays values from `esTypes` instead of `type` To reproduce issue <elastic#129603> and to verify the fix: 1) Open Kibana `Dev tools` 2) Execute the following query to delete any exiting `logs-ti_test` index: ``` DELETE logs-ti_test ``` 3) Execute the following query to create an index named `logs-ti_test`, which has the following properities: - Dynamic mappings are disabled via `"dynamic": false` - It does NOT contain a mapping for `event.action` (we will sort by this field in later steps) - It contains a mapping for the non-ECS `testing` field ``` PUT logs-ti_test { "mappings": { "dynamic": false, "properties": { "@timestamp": { "type": "date" }, "event": { "properties": { "category": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "dataset": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "kind": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "type": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } }, "host": { "properties": { "name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } }, "testing": { "type": "keyword", "ignore_above": 1024 }, "threat": { "properties": { "indicator": { "properties": { "file": { "properties": { "hash": { "properties": { "md5": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } } } } } } } } } } } ``` 4) Execute the following query to add a new document to the `logs-ti_test` index, and note that: - It does NOT contain a `event.action` field - It contains a value for the non-ECS `testing` field ``` POST logs-ti_test/_doc/ { "@timestamp": "2022-05-12T00:00:14.725Z", "host": { "name": "foozle" }, "threat": { "indicator": { "file": { "hash": { "md5": "a4f87cbcd2a4241da77b6bf0c5d9e8553fec991f" } } } }, "event": { "kind": "enrichment", "type": "indicator", "dataset": "ti_*", "category": "threat" }, "testing": "simulated threat intel data" } ``` 5) Navigate to the Security > Hosts page 6) Select `Last 1 year` from the date picker 7) Click the `Events` tab 8) Enter the following KQL query in the search bar at the top of the page: ``` host.name: foozle ``` 9) Hover over the `foozle` entry in the `host.name` column in the Events table, and click the `Add to timeline investigation` cell action 10) Open the timeline 11) Hover over the `event.action` field **Expected result** - The tooltip displays type `keyword` for the `event.action` field **Actual result** - The tooltip displays type `string` for the `event.action` field 12) Click the `event.action` column to add a secondary sort **Expected result** - The table is sorted by `@timestamp` and `event.action` - The table contents are (still) visible **Actual result** - The table is sorted by `@timestamp` and `event.action` - The contents of the table are now empty 13) Click the timeline's `Inspect` button 14) In the `Inspect Timeline` dialog, click the `Request` tab 15) Scroll down to the `sort` property of the request **Expected result** - The `event.action` field contains a `"unmapped_type": "keyword"` property, per the example below: ```json "sort": [ { "@timestamp": { "order": "desc", "unmapped_type": "number" } }, { "event.action": { "order": "desc", "unmapped_type": "keyword" } } ], ``` **Actual result** - The request's `event.action` field contains a `"unmapped_type": "string"` property, per the example below: ```json "sort": [ { "@timestamp": { "order": "desc", "unmapped_type": "number" } }, { "event.action": { "order": "desc", "unmapped_type": "string" } } ], ``` 16) In the `Inspect Timeline` dialog, click the `Response` tab **Expected result** - The response contains `0` `failed` shards / no failures **Actual result** - The response contains failures for the `logs-ti_test` index, with the following reason: ``` "reason": "No mapper found for type [string]" ``` per the example below: ```json { "took": 1, "timed_out": false, "_shards": { "total": 4, "successful": 3, "skipped": 0, "failed": 1, "failures": [ { "shard": 0, "index": "logs-ti_test", "node": "NCRcGeDqSlKQiuPWVFvMEg", "reason": { "type": "illegal_argument_exception", "reason": "No mapper found for type [string]" } } ] }, ```
andrew-goldstein
added a commit
that referenced
this issue
May 17, 2022
…132190) ## [Security Solution] Fixes sorting issues related to unmapped fields This PR fixes the following issues related to sorting unmapped fields in timelines and the events / alerts tables: - <#129603> - <#123912> - <#131625> The `unmapped_type` property [addition](https://github.com/elastic/kibana/pull/87241/files#diff-52fd5870dcd5f783f9fc8ac3a18a8674d83ac6136e09fe0e0bcae30427d61c3fR55) to the `sort` parameter of requests was using the `type` field metadata from `BrowserFields`, but the `type` metadata (for some fields) contains the value `string`, which is not a [valid field data type](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html). The fix for the issues above: - Populates the `sort` property of requests with values from the `esTypes` `BrowserFields` metadata (instead of `type`) - The `esTypes` metadata may specify more than one field value type. When `esTypes` contains more than one type, and `keyword` is one of the types, the `sort` property of the request will prefer `keyword` over other the other types - When the field metadata has an empty `esTypes` collection, the `sort` property of the request will default to using `"unmapped_type": "keyword"` - The field type displayed in tooltips when hovering over columns in a timeline now displays values from `esTypes` instead of `type` ### Desk testing To reproduce issue <#129603> and to verify the fix: 1) Open Kibana `Dev tools` 2) Execute the following query to delete any exiting `logs-ti_test` index: ``` DELETE logs-ti_test ``` 3) Execute the following query to create an index named `logs-ti_test`, which has the following properities: - Dynamic mappings are disabled via `"dynamic": false` - It does NOT contain a mapping for `event.action` (we will sort by this field in later steps) - It contains a mapping for the non-ECS `testing` field ``` PUT logs-ti_test { "mappings": { "dynamic": false, "properties": { "@timestamp": { "type": "date" }, "event": { "properties": { "category": { "type": "keyword" }, "dataset": { "type": "keyword" }, "kind": { "type": "keyword" }, "type": { "type": "keyword" } } }, "host": { "properties": { "name": { "type": "keyword" } } }, "testing": { "type": "keyword", "ignore_above": 1024 }, "threat": { "properties": { "indicator": { "properties": { "file": { "properties": { "hash": { "properties": { "md5": { "type": "keyword" } } } } } } } } } } } } ``` 4) Execute the following query to add a new document to the `logs-ti_test` index, and note that: - It does NOT contain a `event.action` field - It contains a value for the non-ECS `testing` field ``` POST logs-ti_test/_doc/ { "@timestamp": "2022-05-12T00:00:14.725Z", "host": { "name": "foozle" }, "threat": { "indicator": { "file": { "hash": { "md5": "a4f87cbcd2a4241da77b6bf0c5d9e8553fec991f" } } } }, "event": { "kind": "enrichment", "type": "indicator", "dataset": "ti_*", "category": "threat" }, "testing": "simulated threat intel data" } ``` 5) Navigate to the Security > Hosts page 6) Select `Last 1 year` from the date picker 7) Click the `Events` tab 8) Enter the following KQL query in the search bar at the top of the page: ``` host.name: foozle ``` 9) Hover over the `foozle` entry in the `host.name` column in the Events table, and click the `Add to timeline investigation` cell action 10) Open the timeline 11) Hover over the `event.action` field **Expected result** - The tooltip displays type `keyword` for the `event.action` field **Actual result** - The tooltip displays type `string` for the `event.action` field 12) Click the `event.action` column to add a secondary sort **Expected result** - The table is sorted by `@timestamp` and `event.action` - The table contents are (still) visible **Actual result** - The table is sorted by `@timestamp` and `event.action` - The contents of the table are now empty 13) Click the timeline's `Inspect` button 14) In the `Inspect Timeline` dialog, click the `Request` tab 15) Scroll down to the `sort` property of the request **Expected result** - The `event.action` field contains a `"unmapped_type": "keyword"` property, per the example below: ```json "sort": [ { "@timestamp": { "order": "desc", "unmapped_type": "date" } }, { "event.action": { "order": "desc", "unmapped_type": "keyword" } } ], ``` **Actual result** - The request's `event.action` field contains a `"unmapped_type": "string"` property, per the example below: ```json "sort": [ { "@timestamp": { "order": "desc", "unmapped_type": "number" } }, { "event.action": { "order": "desc", "unmapped_type": "string" } } ], ``` 16) In the `Inspect Timeline` dialog, click the `Response` tab **Expected result** - The response contains `0` `failed` shards / no failures **Actual result** - The response contains failures for the `logs-ti_test` index, with the following reason: ``` "reason": "No mapper found for type [string]" ``` per the example below: ```json { "took": 1, "timed_out": false, "_shards": { "total": 4, "successful": 3, "skipped": 0, "failed": 1, "failures": [ { "shard": 0, "index": "logs-ti_test", "node": "NCRcGeDqSlKQiuPWVFvMEg", "reason": { "type": "illegal_argument_exception", "reason": "No mapper found for type [string]" } } ] }, ```
andrew-goldstein
added a commit
to andrew-goldstein/kibana
that referenced
this issue
May 17, 2022
…lastic#132190) This PR fixes the following issues related to sorting unmapped fields in timelines and the events / alerts tables: - <elastic#129603> - <elastic#123912> - <elastic#131625> The `unmapped_type` property [addition](https://github.com/elastic/kibana/pull/87241/files#diff-52fd5870dcd5f783f9fc8ac3a18a8674d83ac6136e09fe0e0bcae30427d61c3fR55) to the `sort` parameter of requests was using the `type` field metadata from `BrowserFields`, but the `type` metadata (for some fields) contains the value `string`, which is not a [valid field data type](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html). The fix for the issues above: - Populates the `sort` property of requests with values from the `esTypes` `BrowserFields` metadata (instead of `type`) - The `esTypes` metadata may specify more than one field value type. When `esTypes` contains more than one type, and `keyword` is one of the types, the `sort` property of the request will prefer `keyword` over other the other types - When the field metadata has an empty `esTypes` collection, the `sort` property of the request will default to using `"unmapped_type": "keyword"` - The field type displayed in tooltips when hovering over columns in a timeline now displays values from `esTypes` instead of `type` To reproduce issue <elastic#129603> and to verify the fix: 1) Open Kibana `Dev tools` 2) Execute the following query to delete any exiting `logs-ti_test` index: ``` DELETE logs-ti_test ``` 3) Execute the following query to create an index named `logs-ti_test`, which has the following properities: - Dynamic mappings are disabled via `"dynamic": false` - It does NOT contain a mapping for `event.action` (we will sort by this field in later steps) - It contains a mapping for the non-ECS `testing` field ``` PUT logs-ti_test { "mappings": { "dynamic": false, "properties": { "@timestamp": { "type": "date" }, "event": { "properties": { "category": { "type": "keyword" }, "dataset": { "type": "keyword" }, "kind": { "type": "keyword" }, "type": { "type": "keyword" } } }, "host": { "properties": { "name": { "type": "keyword" } } }, "testing": { "type": "keyword", "ignore_above": 1024 }, "threat": { "properties": { "indicator": { "properties": { "file": { "properties": { "hash": { "properties": { "md5": { "type": "keyword" } } } } } } } } } } } } ``` 4) Execute the following query to add a new document to the `logs-ti_test` index, and note that: - It does NOT contain a `event.action` field - It contains a value for the non-ECS `testing` field ``` POST logs-ti_test/_doc/ { "@timestamp": "2022-05-12T00:00:14.725Z", "host": { "name": "foozle" }, "threat": { "indicator": { "file": { "hash": { "md5": "a4f87cbcd2a4241da77b6bf0c5d9e8553fec991f" } } } }, "event": { "kind": "enrichment", "type": "indicator", "dataset": "ti_*", "category": "threat" }, "testing": "simulated threat intel data" } ``` 5) Navigate to the Security > Hosts page 6) Select `Last 1 year` from the date picker 7) Click the `Events` tab 8) Enter the following KQL query in the search bar at the top of the page: ``` host.name: foozle ``` 9) Hover over the `foozle` entry in the `host.name` column in the Events table, and click the `Add to timeline investigation` cell action 10) Open the timeline 11) Hover over the `event.action` field **Expected result** - The tooltip displays type `keyword` for the `event.action` field **Actual result** - The tooltip displays type `string` for the `event.action` field 12) Click the `event.action` column to add a secondary sort **Expected result** - The table is sorted by `@timestamp` and `event.action` - The table contents are (still) visible **Actual result** - The table is sorted by `@timestamp` and `event.action` - The contents of the table are now empty 13) Click the timeline's `Inspect` button 14) In the `Inspect Timeline` dialog, click the `Request` tab 15) Scroll down to the `sort` property of the request **Expected result** - The `event.action` field contains a `"unmapped_type": "keyword"` property, per the example below: ```json "sort": [ { "@timestamp": { "order": "desc", "unmapped_type": "date" } }, { "event.action": { "order": "desc", "unmapped_type": "keyword" } } ], ``` **Actual result** - The request's `event.action` field contains a `"unmapped_type": "string"` property, per the example below: ```json "sort": [ { "@timestamp": { "order": "desc", "unmapped_type": "number" } }, { "event.action": { "order": "desc", "unmapped_type": "string" } } ], ``` 16) In the `Inspect Timeline` dialog, click the `Response` tab **Expected result** - The response contains `0` `failed` shards / no failures **Actual result** - The response contains failures for the `logs-ti_test` index, with the following reason: ``` "reason": "No mapper found for type [string]" ``` per the example below: ```json { "took": 1, "timed_out": false, "_shards": { "total": 4, "successful": 3, "skipped": 0, "failed": 1, "failures": [ { "shard": 0, "index": "logs-ti_test", "node": "NCRcGeDqSlKQiuPWVFvMEg", "reason": { "type": "illegal_argument_exception", "reason": "No mapper found for type [string]" } } ] }, ``` (cherry picked from commit f2c8b2c)
andrew-goldstein
added a commit
that referenced
this issue
May 17, 2022
…elds (#132190) (#132357) ## [Security Solution] Fixes sorting issues related to unmapped fields This PR fixes the following issues related to sorting unmapped fields in timelines and the events / alerts tables: - <#129603> - <#123912> - <#131625> The `unmapped_type` property [addition](https://github.com/elastic/kibana/pull/87241/files#diff-52fd5870dcd5f783f9fc8ac3a18a8674d83ac6136e09fe0e0bcae30427d61c3fR55) to the `sort` parameter of requests was using the `type` field metadata from `BrowserFields`, but the `type` metadata (for some fields) contains the value `string`, which is not a [valid field data type](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html). The fix for the issues above: - Populates the `sort` property of requests with values from the `esTypes` `BrowserFields` metadata (instead of `type`) - The `esTypes` metadata may specify more than one field value type. When `esTypes` contains more than one type, and `keyword` is one of the types, the `sort` property of the request will prefer `keyword` over other the other types - When the field metadata has an empty `esTypes` collection, the `sort` property of the request will default to using `"unmapped_type": "keyword"` - The field type displayed in tooltips when hovering over columns in a timeline now displays values from `esTypes` instead of `type` ### Desk testing To reproduce issue <#129603> and to verify the fix: 1) Open Kibana `Dev tools` 2) Execute the following query to delete any exiting `logs-ti_test` index: ``` DELETE logs-ti_test ``` 3) Execute the following query to create an index named `logs-ti_test`, which has the following properities: - Dynamic mappings are disabled via `"dynamic": false` - It does NOT contain a mapping for `event.action` (we will sort by this field in later steps) - It contains a mapping for the non-ECS `testing` field ``` PUT logs-ti_test { "mappings": { "dynamic": false, "properties": { "@timestamp": { "type": "date" }, "event": { "properties": { "category": { "type": "keyword" }, "dataset": { "type": "keyword" }, "kind": { "type": "keyword" }, "type": { "type": "keyword" } } }, "host": { "properties": { "name": { "type": "keyword" } } }, "testing": { "type": "keyword", "ignore_above": 1024 }, "threat": { "properties": { "indicator": { "properties": { "file": { "properties": { "hash": { "properties": { "md5": { "type": "keyword" } } } } } } } } } } } } ``` 4) Execute the following query to add a new document to the `logs-ti_test` index, and note that: - It does NOT contain a `event.action` field - It contains a value for the non-ECS `testing` field ``` POST logs-ti_test/_doc/ { "@timestamp": "2022-05-12T00:00:14.725Z", "host": { "name": "foozle" }, "threat": { "indicator": { "file": { "hash": { "md5": "a4f87cbcd2a4241da77b6bf0c5d9e8553fec991f" } } } }, "event": { "kind": "enrichment", "type": "indicator", "dataset": "ti_*", "category": "threat" }, "testing": "simulated threat intel data" } ``` 5) Navigate to the Security > Hosts page 6) Select `Last 1 year` from the date picker 7) Click the `Events` tab 8) Enter the following KQL query in the search bar at the top of the page: ``` host.name: foozle ``` 9) Hover over the `foozle` entry in the `host.name` column in the Events table, and click the `Add to timeline investigation` cell action 10) Open the timeline 11) Hover over the `event.action` field **Expected result** - The tooltip displays type `keyword` for the `event.action` field **Actual result** - The tooltip displays type `string` for the `event.action` field 12) Click the `event.action` column to add a secondary sort **Expected result** - The table is sorted by `@timestamp` and `event.action` - The table contents are (still) visible **Actual result** - The table is sorted by `@timestamp` and `event.action` - The contents of the table are now empty 13) Click the timeline's `Inspect` button 14) In the `Inspect Timeline` dialog, click the `Request` tab 15) Scroll down to the `sort` property of the request **Expected result** - The `event.action` field contains a `"unmapped_type": "keyword"` property, per the example below: ```json "sort": [ { "@timestamp": { "order": "desc", "unmapped_type": "date" } }, { "event.action": { "order": "desc", "unmapped_type": "keyword" } } ], ``` **Actual result** - The request's `event.action` field contains a `"unmapped_type": "string"` property, per the example below: ```json "sort": [ { "@timestamp": { "order": "desc", "unmapped_type": "number" } }, { "event.action": { "order": "desc", "unmapped_type": "string" } } ], ``` 16) In the `Inspect Timeline` dialog, click the `Response` tab **Expected result** - The response contains `0` `failed` shards / no failures **Actual result** - The response contains failures for the `logs-ti_test` index, with the following reason: ``` "reason": "No mapper found for type [string]" ``` per the example below: ```json { "took": 1, "timed_out": false, "_shards": { "total": 4, "successful": 3, "skipped": 0, "failed": 1, "failures": [ { "shard": 0, "index": "logs-ti_test", "node": "NCRcGeDqSlKQiuPWVFvMEg", "reason": { "type": "illegal_argument_exception", "reason": "No mapper found for type [string]" } } ] }, ```
andrew-goldstein
added a commit
that referenced
this issue
May 18, 2022
…ields (#132190) (#132369) ## [Security Solution] Fixes sorting issues related to unmapped fields This PR fixes the following issues related to sorting unmapped fields in timelines and the events / alerts tables: - <#129603> - <#123912> - <#131625> The `unmapped_type` property [addition](https://github.com/elastic/kibana/pull/87241/files#diff-52fd5870dcd5f783f9fc8ac3a18a8674d83ac6136e09fe0e0bcae30427d61c3fR55) to the `sort` parameter of requests was using the `type` field metadata from `BrowserFields`, but the `type` metadata (for some fields) contains the value `string`, which is not a [valid field data type](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html). The fix for the issues above: - Populates the `sort` property of requests with values from the `esTypes` `BrowserFields` metadata (instead of `type`) - The `esTypes` metadata may specify more than one field value type. When `esTypes` contains more than one type, and `keyword` is one of the types, the `sort` property of the request will prefer `keyword` over other the other types - When the field metadata has an empty `esTypes` collection, the `sort` property of the request will default to using `"unmapped_type": "keyword"` - The field type displayed in tooltips when hovering over columns in a timeline now displays values from `esTypes` instead of `type` ### Desk testing To reproduce issue <#129603> and to verify the fix: 1) Open Kibana `Dev tools` 2) Execute the following query to delete any exiting `logs-ti_test` index: ``` DELETE logs-ti_test ``` 3) Execute the following query to create an index named `logs-ti_test`, which has the following properities: - Dynamic mappings are disabled via `"dynamic": false` - It does NOT contain a mapping for `event.action` (we will sort by this field in later steps) - It contains a mapping for the non-ECS `testing` field ``` PUT logs-ti_test { "mappings": { "dynamic": false, "properties": { "@timestamp": { "type": "date" }, "event": { "properties": { "category": { "type": "keyword" }, "dataset": { "type": "keyword" }, "kind": { "type": "keyword" }, "type": { "type": "keyword" } } }, "host": { "properties": { "name": { "type": "keyword" } } }, "testing": { "type": "keyword", "ignore_above": 1024 }, "threat": { "properties": { "indicator": { "properties": { "file": { "properties": { "hash": { "properties": { "md5": { "type": "keyword" } } } } } } } } } } } } ``` 4) Execute the following query to add a new document to the `logs-ti_test` index, and note that: - It does NOT contain a `event.action` field - It contains a value for the non-ECS `testing` field ``` POST logs-ti_test/_doc/ { "@timestamp": "2022-05-12T00:00:14.725Z", "host": { "name": "foozle" }, "threat": { "indicator": { "file": { "hash": { "md5": "a4f87cbcd2a4241da77b6bf0c5d9e8553fec991f" } } } }, "event": { "kind": "enrichment", "type": "indicator", "dataset": "ti_*", "category": "threat" }, "testing": "simulated threat intel data" } ``` 5) Navigate to the Security > Hosts page 6) Select `Last 1 year` from the date picker 7) Click the `Events` tab 8) Enter the following KQL query in the search bar at the top of the page: ``` host.name: foozle ``` 9) Hover over the `foozle` entry in the `host.name` column in the Events table, and click the `Add to timeline investigation` cell action 10) Open the timeline 11) Hover over the `event.action` field **Expected result** - The tooltip displays type `keyword` for the `event.action` field **Actual result** - The tooltip displays type `string` for the `event.action` field 12) Click the `event.action` column to add a secondary sort **Expected result** - The table is sorted by `@timestamp` and `event.action` - The table contents are (still) visible **Actual result** - The table is sorted by `@timestamp` and `event.action` - The contents of the table are now empty 13) Click the timeline's `Inspect` button 14) In the `Inspect Timeline` dialog, click the `Request` tab 15) Scroll down to the `sort` property of the request **Expected result** - The `event.action` field contains a `"unmapped_type": "keyword"` property, per the example below: ```json "sort": [ { "@timestamp": { "order": "desc", "unmapped_type": "date" } }, { "event.action": { "order": "desc", "unmapped_type": "keyword" } } ], ``` **Actual result** - The request's `event.action` field contains a `"unmapped_type": "string"` property, per the example below: ```json "sort": [ { "@timestamp": { "order": "desc", "unmapped_type": "number" } }, { "event.action": { "order": "desc", "unmapped_type": "string" } } ], ``` 16) In the `Inspect Timeline` dialog, click the `Response` tab **Expected result** - The response contains `0` `failed` shards / no failures **Actual result** - The response contains failures for the `logs-ti_test` index, with the following reason: ``` "reason": "No mapper found for type [string]" ``` per the example below: ```json { "took": 1, "timed_out": false, "_shards": { "total": 4, "successful": 3, "skipped": 0, "failed": 1, "failures": [ { "shard": 0, "index": "logs-ti_test", "node": "NCRcGeDqSlKQiuPWVFvMEg", "reason": { "type": "illegal_argument_exception", "reason": "No mapper found for type [string]" } } ] }, ```
|
Please test the fix for this issue in the following branches / releases: |
|
Hi Team, We have validated above issue on 8.2. branch and it's working fine ✅. Build Details: Screenshots:
We will validate this issue on 7.17 branch and will share our observation as well Thanks! cc: @MadameSheema |
|
Hi Team, We have validated above issue on 8.2.1 BC1 build and found that it's still occurring 🔴. Looks like backport is not merged on 8.2.1 BC1 Below are the observations:
Build info Screenshots:
We will validate this issue on 7.17.4 and will share our observation as well. Thanks! |
|
Hi Team, We have validated above issue on 8.2.1 BC2 Staging build and it's working fine ✅. Below are the observations:
Build info: Screenshots:
We will validate this issue on 7.17.4 and will share our observation as well. Thanks! |
|
Hi Team, We have validated above issue on 7.17.4 build and it's working fine ✅. Below are the observations:
Build info: Screenshots:
Hence, marking this as QA Validated. Thanks! |
ghost
added
the
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
ESS - 8.1.0, Chrome V100, FF 98.0.2
Describe the bug:
Timeline fails to show docs when those docs are missing selected sorting fields
Steps to reproduce:
Expected behavior:
show docs even if missing sort field
Screenshots (if relevant):
Errors in browser console (if relevant):
Provide logs and/or server output (if relevant):
timeline response.json.zip
Any additional context:
https://observe-aws.prod-3.eden.elastic.dev/app/security/timelines/default?sourcerer=(default:(id:security-solution-default,selectedPatterns:!(%27apm-*-transaction*%27,%27filebeat-*%27,%27logs-*%27,%27packetbeat-*%27,%27traces-apm*%27)))&timerange=(global:(linkTo:!(),timerange:(from:%272022-04-06T04:00:00.000Z%27,fromStr:now%2Fd,kind:relative,to:%272022-04-07T03:59:59.999Z%27,toStr:now%2Fd)),timeline:(linkTo:!(),timerange:(from:%272022-04-01T20:00:00.000Z%27,kind:absolute,to:%272022-04-01T22:00:00.000Z%27)))&timeline=(activeTab:query,graphEventId:%27%27,id:dd6c1ef0-b5a3-11ec-aaa2-efdbe68cf756,isOpen:!t)
remove sort on event.action and timeline populates.
The text was updated successfully, but these errors were encountered: