Skip to content

Commit

Permalink
[Security Solution] Implement concurrency control for Prebuilt Upgrad…
Browse files Browse the repository at this point in the history
…e workflow (elastic#203604)

**Resolves:** elastic#200134

## Summary

This PR implements concurrency control to make sure user has the recent rule updates data in Rule Upgrade flyout. Any modifications saved in Rule Upgrade flyout are reset upon new `revision` or `version` detected.

## Details

Concurrency control is important to provide better UX. Multiple users work in Kibana in parallel and new prebuilt rules package version can be released in any time. Attempts to upgrade a rule with outdated `revision` and/or `version` results in failed request. Users may experience multiple rule upgrade failure in that case causing a lot of confusion. More experienced users may guess to reload the page to continue.

Typical reasons leading to `revision` and/or `version` change are the following

- Current rule has been edited will bump rule's `revision`. For example the rule currently shown in Rule Upgrade flyout has been edited by someone else.
- Prebuilt rules package got released will give provide rule assets with higher `version`. Rules having upgrades in the currently installed package and in a new one are affected.

This PR mitigates the described issues by implementing concurrency control. It sets up `_review` API endpoint refetch interval to 5 minutes to fetch fresh data. In case a higher `revision` or `version` is detected for some rule this rule's resolved conflicts and customizations performed in Rule Upgrade flyout get cleared.

## Screenshots

- `revision` change (refresh interval was reduced to 30 seconds to make the video shorter)

https://github.com/user-attachments/assets/98d2a22f-9338-482a-a7b2-1e170b9642ce

- `version` change (refresh interval was reduced to 1 minute to make the video shorter)

https://github.com/user-attachments/assets/2b7c23f0-5a50-471e-aa7f-8d9b2aecc957

## How to test locally

There are two cases for testing

- `revision` change
- `version` change

### Test `revision` change

Revision change means the rule has been edited. Use the following steps to test it 

- Ensure the `prebuiltRulesCustomizationEnabled` feature flag is enabled
- Allow internal APIs via adding `server.restrictInternalApis: false` to `kibana.dev.yaml`
- Clear Elasticsearch data
- Run Elasticsearch and Kibana locally (do not open Kibana in a web browser)
- Install an outdated version of the `security_detection_engine` Fleet package
```bash
curl -X POST --user elastic:changeme  -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 2023-10-31" -d '{"force":true}' http://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1
```

- Install prebuilt rules
```bash
curl -X POST --user elastic:changeme  -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 1" -d '{"mode":"ALL_RULES"}' http://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform
```
- Open `Detection Rules (SIEM)` Page -> `Rule Updates`
- Open Rule upgrade flyout for some rule
- Make changes to rule field(s) and save them (do not upgrade the rule)
- Open the other web browser tab with Kibana
- Navigate to the same rule's editing page
- Change any field and save the changes
- Return back to the first tab and wait for data to be refetched (data refresh interval is 5 minutes, wait for `_review` request in the Dev Tool's Network tab)
- Make sure the changes you made for field(s) got reverted

### Test `version` change

Version change means a new package version was released. Do the following to test it

- Ensure the `prebuiltRulesCustomizationEnabled` feature flag is enabled
- Allow internal APIs via adding `server.restrictInternalApis: false` to `kibana.dev.yaml`
- Clear Elasticsearch data
- Run Elasticsearch and Kibana locally (do not open Kibana in a web browser)
- Set `xpack.securitySolution.prebuiltRulesPackageVersion: 8.15.2` in `kibana.dev.yaml`
- Install an outdated version of the `security_detection_engine` Fleet package
```bash
curl -X POST --user elastic:changeme  -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 2023-10-31" -d '{"force":true}' http://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1
```

- Install prebuilt rules
```bash
curl -X POST --user elastic:changeme  -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 1" -d '{"mode":"ALL_RULES"}' http://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform
```
- Open `Detection Rules (SIEM)` Page -> `Rule Updates`
- Open Rule upgrade flyout for a rule having updates in packages `v8.15.2` and `.8.17.1-beta.1` for example `Suspicious Web Browser Sensitive File Access`
- Make changes to rule field(s) and save them (do not upgrade the rule)
- Set `xpack.securitySolution.prebuiltRulesPackageVersion: 8.17.1-beta.1` in `kibana.dev.yaml`
- Open the other web browser tab with Kibana
- Navigate to Security Solution plugin to install the
  OR
  install the package `8.17.1-beta.1` via API request
```bash
curl -X POST --user elastic:changeme  -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 2023-10-31" -d '{"force":true}' http://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.17.1-beta.1
```
- Return back to the first tab and wait for data to be refetched (data refresh interval is 5 minutes, wait for `_review` request in the Dev Tool's Network tab)
- Make sure the changes you made for field(s) got the recent target rule values

Alternatively you can spin up EPR locally and publish package updates with rule's version bumped.
  • Loading branch information
maximpn authored and crespocarlos committed Jan 8, 2025
1 parent 9ad96e8 commit e7310a4
Show file tree
Hide file tree
Showing 4 changed files with 553 additions and 6 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -140,3 +140,37 @@ export const RULE_MODIFIED_BADGE_DESCRIPTION = i18n.translate(
'The rule was edited after installation and field values differs from the values upon installation',
}
);

export const RULE_NEW_REVISION_DETECTED_WARNING = i18n.translate(
'xpack.securitySolution.detectionEngine.upgradeFlyout.ruleNewRevisionDetectedWarning',
{
defaultMessage: 'Installed rule changed',
}
);

export const RULE_NEW_REVISION_DETECTED_WARNING_DESCRIPTION = (ruleName: string) =>
i18n.translate(
'xpack.securitySolution.detectionEngine.upgradeFlyout.ruleNewVersionDetectedWarningDescription',
{
defaultMessage:
'Someone edited the installed rule "{ruleName}". Upgrade resolved conflicts were reset.',
values: { ruleName },
}
);

export const RULE_NEW_VERSION_DETECTED_WARNING = i18n.translate(
'xpack.securitySolution.detectionEngine.upgradeFlyout.ruleNewRevisionDetectedWarning',
{
defaultMessage: 'New prebuilt rules package was installed',
}
);

export const RULE_NEW_VERSION_DETECTED_WARNING_DESCRIPTION = (ruleName: string) =>
i18n.translate(
'xpack.securitySolution.detectionEngine.upgradeFlyout.ruleNewRevisionDetectedWarningDescription',
{
defaultMessage:
'Newer prebuilt rules package were installed in background. It contains a newer rule version for "{ruleName}". Upgrade resolved conflicts were reset.',
values: { ruleName },
}
);
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ import { UpgradeFlyoutSubHeader } from './upgrade_flyout_subheader';
import * as ruleDetailsI18n from '../../../../rule_management/components/rule_details/translations';
import * as i18n from './translations';

const REVIEW_PREBUILT_RULES_UPGRADE_REFRESH_INTERVAL = 5 * 60 * 1000;

export interface UpgradePrebuiltRulesTableState {
/**
* Rule upgrade state after applying `filterOptions`
Expand Down Expand Up @@ -110,6 +112,13 @@ interface UpgradePrebuiltRulesTableContextProviderProps {
children: React.ReactNode;
}

/**
* Provides necessary data and actions for Rules Upgrade table.
*
* It periodically re-fetches prebuilt rules upgrade review data to detect possible cases of:
* - editing prebuilt rules (revision change)
* - releasing a new prebuilt rules package (version change)
*/
export const UpgradePrebuiltRulesTableContextProvider = ({
children,
}: UpgradePrebuiltRulesTableContextProviderProps) => {
Expand All @@ -135,7 +144,7 @@ export const UpgradePrebuiltRulesTableContextProvider = ({
isLoading,
isRefetching,
} = usePrebuiltRulesUpgradeReview({
refetchInterval: false, // Disable automatic refetching since request is expensive
refetchInterval: REVIEW_PREBUILT_RULES_UPGRADE_REFRESH_INTERVAL,
keepPreviousData: true, // Use this option so that the state doesn't jump between "success" and "loading" on page change
});
const { rulesUpgradeState, setRuleFieldResolvedValue } =
Expand Down
Loading

0 comments on commit e7310a4

Please sign in to comment.