Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Suggestion. Collapse Alerting graphs into one view #120282

Closed
Tracked by #120277
snide opened this issue Dec 2, 2021 · 6 comments
Closed
Tracked by #120277

[Security Solution] Suggestion. Collapse Alerting graphs into one view #120282

snide opened this issue Dec 2, 2021 · 6 comments
Assignees
Labels
enhancement New value added to drive a business result Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Threat Hunting:Investigations Security Solution Investigations Team Team:Threat Hunting Security Solution Threat Hunting Team

Comments

@snide
Copy link
Contributor

snide commented Dec 2, 2021

The alerting graph summaries could be consolidated into one view by providing the count in the label of the histogram. I might be missing something but I don't think you'd ever compare the count of one metric against the histogram against another metric. Elastic charts should give you the ability to apply the counts to the label themselves.

Moreover, there is likely little reason to use a custom legend here. The hover state filtering doesn't seem to be used very often from FullStory, so I'd suggest going with the default Elastic Charts legends.

Also, are these charts using the EC/EUI palettes?

image

@snide snide added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Dec 2, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@michaelolo24 michaelolo24 added enhancement New value added to drive a business result Team:Threat Hunting:Investigations Security Solution Investigations Team labels Dec 15, 2021
@Max-Iskra Max-Iskra self-assigned this Feb 16, 2022
@Max-Iskra
Copy link

Max-Iskra commented Feb 21, 2022

Figma mockups - https://ela.st/collapsed-graphs

  • Graphs are now collapsed into one view
  • Added headings: Rule name and Count into graphs legend table
  • Header now is sticky for legend scrolling

Collapsed Gpaphs View

Collapsed

Scrolling legend

States

Page overview

  • Overall padding cleanup
  • Added sticky pagination at the bottom of the page

Note: See upcoming page updates at: #120291

Collapsed Alerted Gpaphs

cc @snide @paulewing

@snide
Copy link
Contributor Author

snide commented Feb 22, 2022

@Max-Iskra Nice! I might suggest collapsing this even further. Likely the table header isn't neccessary. The signal.rule.name is duplicative with the dropdown, and the count can be assumed.

Visually I might also recommend removing the border on the panel. I don't think it needs the container?

@Max-Iskra
Copy link

Collapsed even further

  • Removed legend title
  • Legend has 5 rows now

Note: In keeping with other pages of a product, the panel has a 1px border. In this case, I think we should create a separate thread if we want to remove the border all over the pages

Collapsed Alerted Gpaphs

cc @snide

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

andrew-goldstein added a commit to andrew-goldstein/kibana that referenced this issue Jul 12, 2022
…ity-team#3221) to the Alerts page:

![alerts_by_risk_score_treemap](https://user-images.githubusercontent.com/4459398/156707243-cd9dc763-7d25-4de2-b53a-04139e334e01.png)

_Above: The new `Alerts by risk score` treemap_

- Alerts are colored by risk score
- Clicking on a cell instantly filters the alerts page
- Users may specify custom fields to group by, or quick-select from our curated sets of fields that pair well together

The features in this PR are enabled via the following feature flag in [experimental_features.ts](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/common/experimental_features.ts):

```ts
  alertsTreemapEnabled: true,
```

When the feature flag is **disabled**, the alerts page looks like `main` today:

![feature_flag_disabled](https://user-images.githubusercontent.com/4459398/156687162-2be47b18-9413-4711-9f50-a9acb5904617.png)

_Above: This branch, when the feature flag is disabled_

When the feature flag is enabled, the layout of the Alerts page incorporates the design detailed in <elastic#120287>, which eliminates unusable space at the top of the Alerts page, and enables users to quickly hide and show the charts.

Users are introduced to the new `Hide charts` feature of the layout via `Step 1` of an [EUI Tour](https://elastic.github.io/eui/#/display/tour):

![tour_step_1](https://user-images.githubusercontent.com/4459398/156707574-9888290d-98e8-48a0-b920-4da6c56abcc5.png)

_Above: `Step 1` of the tour introduces the `Hide charts` feature_

Users may (optionally) try the `Hide charts` feature as part of the tour, per the video below:

<https://user-images.githubusercontent.com/4459398/156709026-ac16c97b-167e-4fce-b0e0-fd16077027b1.mov>

_Above: (video) The interactive tour let's users experience the new `Hide charts` feature_

`Step 2`, the final step of the tour, introduces the view selector:

![tour_step_2](https://user-images.githubusercontent.com/4459398/156707828-a1f742d8-3be2-4322-8bcf-dfd30a147670.png)

_Above: `Step 2` of the tour introduces the view selector_

Per the screenshot and video below, view selection toggles between the default `Trend view` and the new `Alerts by risk score view`:

![view_selection](https://user-images.githubusercontent.com/4459398/156708084-3c387887-ff2e-44f5-b729-7ed9f16e5297.png)

_Above: View selection_

<https://user-images.githubusercontent.com/4459398/156710378-3b9461c1-9779-4d66-8f12-21709d6f25fc.mov>

_Above: (video) View selection toggles between the  `Trend view` and the `Alerts by risk score view`_

We store the user's chart visibility and view preferences in local storage.

When users navigate to other views in Kibana or close the browser, their chart visibility and view preferences are restored the next time they visit the Alerts page.

Every rule, including prebuilt Elastic rules and custom rules created by users, must specify a risk score at rule creation time, per the screenshot below:

![rule_risk_score_configuration](https://user-images.githubusercontent.com/4459398/156712042-19b71f53-f337-4aed-bebf-ce10ea2b9f63.png)

_Above: Every rule has a risk score specified when it's created_

The colors of the alerts displayed in the treemap are determined by the rule's risk score.

This makes it easy to quickly filter the entire alerts page by clicking on the riskiest alerts, per the video below:

<https://user-images.githubusercontent.com/4459398/156714075-75112515-a247-4aa4-bc58-52fd17ce6fa0.mov>

_Above: (video) Users click on the riskiest alerts to filter the page_

Clicking on a cell in the treemap adds two filters, one for each group by field:

![two_filters](https://user-images.githubusercontent.com/4459398/156715601-5b7c0208-2fae-4231-ba3f-3073ef66cccf.png)

_Above: Two filters, (one for each group by field), are added to the page when a cell is clicked_

In the screenshot below, the 2nd filter was removed to reveal all the hosts in `mimikatz process started` alerts:

![2nd_filter_removed](https://user-images.githubusercontent.com/4459398/156716276-d582143a-a665-42f2-b1e2-122925d7399a.png)

_Above: Removing the 2nd filter, `host.name`, revealed all the hosts in the `mimikatz process started` alerts_

We can instantly answer questions like "Did this host trigger alerts for other rules?", and at a glance, grok the severity of those alerts, by removing the first of the two filters:

![1st_filter_removed](https://user-images.githubusercontent.com/4459398/156719888-51852cb5-c2bd-43b0-8a80-d1ee2c121bab.png)

_Above: Removing the first filter, `kibana.alert.rule.name`, revealed other alerts (from different rules) triggered for a specific host_

Users may customize the group by criteria:

<https://user-images.githubusercontent.com/4459398/156720911-8ed26b24-658f-4cc3-85b2-f317e15c430c.mov>

_Above: (video) Entering custom group by criteria_

When users navigate to other views in Kibana or close the browser, the previously-selected group by fields are restored from local storage the next time they visit the Alerts page.

Clicking on the first menu item in the settings gear, `Reset group by fields`, restores the default group by fields, per the video below:

<https://user-images.githubusercontent.com/4459398/156722389-2927d4e4-f8e8-4113-88d7-274a7c1e4bbe.mov>

_Above: (video) Restoring the default group by fields_

Users may quick-select from Elastic-curated sets of fields that pair well together, like parent and child processes, or process and file name:

![quick_select_groups](https://user-images.githubusercontent.com/4459398/156723673-cb405e41-9577-46a2-944b-44d4f1b2d6d7.png)

_Above: Quick-select from Elastic-curated sets of fiels_

Previous versions of the `Count` table were limited to grouping by one field. This PR enhances it to support grouping by two fields, per the screenshot below:

![count_two_group_by_fields](https://user-images.githubusercontent.com/4459398/156725762-1ea9ef4f-8d6c-4847-a637-f847df50bccd.png)

_Above: The `Count` table now supports two group by fields_

The group by criteria of the `Count` and `Alerts by risk score` widgets are linked together, making it easy to toggle between the two views, per the video below:

<https://user-images.githubusercontent.com/4459398/156727296-59fc1f29-e7a2-45eb-a549-5317100ab68a.mov>

_Above: (video) The `Count` and `Alerts by risk score` group by criteria are linked_

The `Trend` chart's legend has been enhanced to display counts, per the design detailed in issue <elastic#120282>

![counts_in_trend_widget_legend](https://user-images.githubusercontent.com/4459398/156728673-d365c88e-8da0-4cc4-a1cc-dbb5939a5c01.png)

_Above: The `Trend` chart legend, enhanced to display counts_

Note: The original intention of adding counts to the `Trend` chart legend, as documented in <elastic#120282>, was to combine the `Trend` and `Count` widgets into a single view via the legend.

In this PR, the `Trend` and `Count` widget queries have diverged; (the latter now supports multiple group by fields). This prevents the `Trend` and `Count` widgets from being combined into a single view.

This PR adds persistence of the `Trend` chart Stack by field to local storage.

When users navigate to other views in Kibana or close the browser, the previously-selected field is restored the next time they visit the Alerts page.
andrew-goldstein added a commit to andrew-goldstein/kibana that referenced this issue Jul 12, 2022
…uping

This PR introduces the new _Treemap_ and _Multi-Dimensional Alert Grouping_ to the Alerts page.

The initial commit was developed as an _ON week_ proof of concept (POC). It has since been updated to incorporate product and UX feedback.

### Alerts treemap

The new _Alerts_ page treemap is shown in the screenshot below:

![treemap](https://user-images.githubusercontent.com/4459398/178233664-c45be7ca-b03e-40b9-b423-aeeaa47461c0.png)

_Above: The new `treemap` in the Alerts page_

- Alerts are colored by risk score
- Clicking on a cell instantly filters the alerts page
- Treemap legend items may be added to filters and Timeline investigations
- The new treemap supports multi-dimensional grouping and filtering
  - Alerts are grouped by `kibana.alert.rule.name` and `host.name` by default

### Multi-Dimensional Alert Grouping

The table on the Alerts page, which previously supported grouping by a single field, has also been enhanced to support multi-dimensional grouping, per the screenshot below:

![alerts-table-multi-dimensional-grouping](https://user-images.githubusercontent.com/4459398/178240710-ecf66799-35a8-4874-8882-5ccfcccc86fe.png)

_Above: The table in the Alerts page, enhanced to support multi-dimensional grouping_

## Filtering the Alerts page by risk score

Every rule, including prebuilt Elastic rules and custom rules created by users, must specify a risk score at rule creation time, per the screenshot below:

![rule_risk_score_configuration](https://user-images.githubusercontent.com/4459398/156712042-19b71f53-f337-4aed-bebf-ce10ea2b9f63.png)

_Above: Every rule has a risk score specified when it's created_

The colors of the alerts displayed in the treemap are determined by the rule's risk score. This makes it easy to quickly filter the entire alerts page by clicking on the riskiest alerts.

Clicking on a cell in the treemap adds two filters, one for each group by field, per the screenshot below:

![two-filters](https://user-images.githubusercontent.com/4459398/178252768-7c66dc5e-8a3c-41d8-95e2-eeca20133127.png)

_Above: Two filters, (one for each group by field), are added to the page when a cell is clicked_

The Alerts page updates instantly when filters are added or removed. In the screenshot below, the 2nd filter was removed to filter the page to all `mimikatz process started` alerts:

![second-filter-removed](https://user-images.githubusercontent.com/4459398/178253726-66905d60-99da-4d76-9ea1-744cb53abd6f.png)

_Above: Removing the 2nd filter, a specific `host.name`, revealed all the hosts in the `mimikatz process started` alerts_

### Switching views

Users may switch between the following views:

- Table
- Trend (default)
- Treemap

per the screenshot below:

![view-selection](https://user-images.githubusercontent.com/4459398/178242001-6868c751-ffa6-486f-b81a-81a4d5912877.png)

_Above: View selection_

The default _Trend_ view is shown in the screenshot below:

![trend-view](https://user-images.githubusercontent.com/4459398/178242769-58d6c800-69db-4e14-87e4-9232f3e35427.png)

_Above: The (default) Trend view_

- The Trend chart's legend has been enhanced to display counts, per the design detailed in issue <elastic#120282>

- The Trend view only supports visualizing a single dimension. Hovering over the disabled `Group by top` select in the Trend view displays the tooltip shown in the screenshot below:

![tooltip](https://user-images.githubusercontent.com/4459398/178243356-9bfe7f54-5b31-4f61-a795-0cfa0a70285b.png)

_Above: The Trend view only supports visualizing a single dimension_

### Collapsing the panel

The panel may optionally be collapsed to save space, per the screenshot below:

![collapsed](https://user-images.githubusercontent.com/4459398/178244282-525d4c9f-a23b-4ec4-8f72-7e813e771687.png)

_Above: The panel (optionally) collapsed_

### User preferences are persisted to local storage

Previously, the _Group by_ selections on the Alerts page were always forgotten when users navigated away from the Alerts page. As a result, users had to re-select their preferred Group by fields every time they visited the page.

We now store all of the following preferences in local storage:

- View selection (Table, Trend, Treemap)
- Group by selections
- Panel collapse state

The preferences above are now restored the next time users return to the Alerts page.

### Group by field selection is synchronized between views

Group by field selection is synchronized between views. For example, if a user changes the Group by fields in the Treemap view and then switches to the Table view, the same Group by fields will be displayed in the Table view.

### Resetting Group by fields to their defaults

Users may reset the Group by fields to their defaults (`kibana.alert.rule.name` and `host.name`) for any visualization via the menu shown in the screenshot below:

![reset-group-by-fields](https://user-images.githubusercontent.com/4459398/178246997-70d89763-40d4-4c93-b0b6-b439fc3e22cd.png)

_Above: Resetting Group by fields to defaults via the menu_
andrew-goldstein added a commit that referenced this issue Jul 13, 2022
…ng (#126896)

## [Security Solution] Alerts Treemap and Multi-Dimensional Alert Grouping

This PR introduces the new _Treemap_ and _Multi-Dimensional Alert Grouping_ to the Alerts page.

The initial commit was developed as an _ON week_ proof of concept (POC). It has since been updated to incorporate product and UX feedback.

### Alerts treemap

The new _Alerts_ page treemap is shown in the screenshot below:

![treemap](https://user-images.githubusercontent.com/4459398/178233664-c45be7ca-b03e-40b9-b423-aeeaa47461c0.png)

_Above: The new `treemap` in the Alerts page_

- Alerts are colored by risk score
- Clicking on a cell instantly filters the _Alerts_ page
- Treemap legend items may be added to filters and Timeline investigations
- The new treemap supports multi-dimensional grouping and filtering
  - Alerts are grouped by `kibana.alert.rule.name` and `host.name` by default

### Multi-Dimensional Alert Grouping

The table on the Alerts page, which previously supported grouping by a single field, has also been enhanced to support multi-dimensional grouping, per the screenshot below:

![alerts-table-multi-dimensional-grouping](https://user-images.githubusercontent.com/4459398/178240710-ecf66799-35a8-4874-8882-5ccfcccc86fe.png)

_Above: The table in the Alerts page, enhanced to support multi-dimensional grouping_

## Filtering the Alerts page by risk score

Every rule, including prebuilt Elastic rules and custom rules created by users, must specify a risk score at rule creation time, per the screenshot below:

![rule_risk_score_configuration](https://user-images.githubusercontent.com/4459398/156712042-19b71f53-f337-4aed-bebf-ce10ea2b9f63.png)

_Above: Every rule has a risk score specified when it's created_

The colors of the alerts displayed in the treemap are determined by the rule's risk score. This makes it easy to quickly filter the entire alerts page by clicking on the riskiest alerts.

Clicking on a cell in the treemap adds two filters, one for each group by field, per the screenshot below:

![two-filters](https://user-images.githubusercontent.com/4459398/178252768-7c66dc5e-8a3c-41d8-95e2-eeca20133127.png)

_Above: Two filters, (one for each group by field), are added to the page when a cell is clicked_

The Alerts page updates instantly when filters are added or removed. In the screenshot below, the 2nd filter was removed to filter the page to all `mimikatz process started` alerts:

![second-filter-removed](https://user-images.githubusercontent.com/4459398/178253726-66905d60-99da-4d76-9ea1-744cb53abd6f.png)

_Above: Removing the 2nd filter, a specific `host.name`, revealed all the hosts in the `mimikatz process started` alerts_

### Switching views

Users may switch between the following views:

- Table
- Trend (default)
- Treemap

per the screenshot below:

![view-selection](https://user-images.githubusercontent.com/4459398/178412669-f437cfe4-f819-45b5-8359-987a2a3c6645.png)

_Above: View selection_

The default _Trend_ view is shown in the screenshot below:

![trend-view](https://user-images.githubusercontent.com/4459398/178242769-58d6c800-69db-4e14-87e4-9232f3e35427.png)

_Above: The (default) Trend view_

- The Trend chart's legend has been enhanced to display counts, per the design detailed in issue <#120282>

- The Trend view only supports visualizing a single dimension. Hovering over the disabled `Group by top` select in the Trend view displays the tooltip shown in the screenshot below:

![tooltip](https://user-images.githubusercontent.com/4459398/178243356-9bfe7f54-5b31-4f61-a795-0cfa0a70285b.png)

_Above: The Trend view only supports visualizing a single dimension_

### Collapsing the panel

The panel may optionally be collapsed to save space, per the screenshot below:

![collapsed](https://user-images.githubusercontent.com/4459398/178244282-525d4c9f-a23b-4ec4-8f72-7e813e771687.png)

_Above: The panel (optionally) collapsed_

### User preferences are persisted to local storage

Previously, the _Group by_ selections on the Alerts page were always forgotten when users navigated away from the Alerts page. As a result, users had to re-select their preferred Group by fields every time they visited the page.

We now store all of the following preferences in local storage:

- View selection (Table, Trend, Treemap)
- Group by selections
- Panel collapse state

The preferences above are now restored the next time users return to the Alerts page.

### Group by field selection is synchronized between views

Group by field selection is synchronized between views. For example, if a user changes the Group by fields in the Treemap view and then switches to the Table view, the same Group by fields will be displayed in the Table view.

### Resetting Group by fields to their defaults

Users may reset the Group by fields to their defaults (`kibana.alert.rule.name` and `host.name`) for any visualization via the menu shown in the screenshot below:

![reset-group-by-fields](https://user-images.githubusercontent.com/4459398/178246997-70d89763-40d4-4c93-b0b6-b439fc3e22cd.png)

_Above: Resetting Group by fields to defaults via the menu_
@PhilippeOberti
Copy link
Contributor

closing as this chart section was updated a few months ago and seemed to have solved the ask here
https://github.com/user-attachments/assets/dcb66a42-6c13-4649-9773-32c51fec8630

If some changes are still desired, please open a different ticket with the correct mocks and acceptance criteria

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New value added to drive a business result Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Threat Hunting:Investigations Security Solution Investigations Team Team:Threat Hunting Security Solution Threat Hunting Team
Projects
None yet
Development

No branches or pull requests

6 participants