Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution][Platform] - 7.16/8.0 Sanity Check #116336

Closed
9 of 21 tasks
yctercero opened this issue Oct 26, 2021 · 5 comments
Closed
9 of 21 tasks

[Security Solution][Platform] - 7.16/8.0 Sanity Check #116336

yctercero opened this issue Oct 26, 2021 · 5 comments
Labels
Team:Detections and Resp Security Detection Response Team Team:Security Solution Platform Security Solution Platform Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v7.16.0 v8.0.0

Comments

@yctercero
Copy link
Contributor

yctercero commented Oct 26, 2021

Sanity check on work done for 7.16/8.0 releases. Merging these two together as they tend to overlap a bit this go around. Feel free to add a comment with different test cases you've tried and any results, bugs found, etc.

7.16 Release Build Ticket

7.16

  • End to End test - basic:

    • Verify rules with actions from 7.15 or earlier, continue to work on upgrade to 7.16
    • Verify that on touch of rule with action from pre 7.16, data structure is updated to new actions
    • Verify that you can export 7.15 rules and import in 7.16
    • Verify that you can create a rule with an action in 7.16 (verify notification fires)
    • Verify that you can export rules in one space and import into another
    • Verify that exceptions are exported on rule export
  • End to End test - trial:

    • Verify user is unable to import rules with actions [license check]

8.0

  • End to End test - basic:
    • Verify rules with actions from 7.16, continue to work on upgrade to 8.0
    • Verify that on touch of rule with action from pre 7.16, data structure is updated to new actions
    • Verify that you can export 7.16 rules and import in 8.0
    • Verify that you can create a rule with an action in 8.0 (verify notification fires)
    • Verify that you can export rules in one space and import into another

Bugs

@yctercero yctercero added v8.0.0 Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v7.16.0 Team:Security Solution Platform Security Solution Platform Team labels Oct 26, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@FrankHassanabad
Copy link
Contributor

FrankHassanabad commented Oct 26, 2021

Bug found by @MadameSheema and @dhurley14 when verifying during BC1:

  • Verify that you can export rules in one space and import into another

Glo tried to import some rules (with actions) into a different space but received a 400 (or 404? I don’t remember) saying the action:some-guid-id does not exist. This was a result of the connector not existing in the space she imported the rule into. This is known at least within our area but I could not find an issue describing if we will just document this behavior or if we are fixing for one of the next BC’s. Anyone know if there is an issue open and what the next steps are?

From me:

This a bug (thanks for finding it) and I will work on this as a fix. Basically I will do a pre-emptive search for all connectors associated with rules to ensure they exist before importing as a validation. If it fails then I will give a clear message that the connector does not exist and I could not import the rule with the missing connector.
We don't support exporting/importing connectors just yet directly from the export button.

I am going to try to make the message more clear to the end user but still do a rejection of the rule if the connector does not exist for the action before the next BC.

@MadameSheema
Copy link
Member

MadameSheema commented Oct 29, 2021

7.16 BC2

  • End to End test - basic:

    • Verify rules with actions from 7.15 or earlier, continue to work on upgrade to 7.16
    • Verify that on touch of rule with action from pre 7.16, data structure is updated to new actions
    • Verify that you can export 7.15 rules and import in 7.16
    • Verify that you can create a rule with an action in 7.16 (verify notification fires)
    • Verify that you can export rules in one space and import into another
    • Verify that exceptions are exported on rule export
  • End to End test - trial:

    • Verify user is unable to import rules with actions [license check]

@yctercero
Copy link
Contributor Author

FrankHassanabad added a commit that referenced this issue Oct 29, 2021
…ssing connections clearer for the end user. (#116490)

## Summary

Fixes issue see on this comment:
#116336 (comment)

* Removes legacy toaster component
* Adds newer toaster component
* Removes issue with the deps array within ReactJS
* Adds utility to give a better network error message to the end user.
* This does effect the timeline component since it shares the same import common component.
* Adds a count of how many rules/timeline items have failed imports
* These error toasters mimic Kibana core's error toaster error message and UI/UX
* Adds e2e tests for imports with actions and error messages for them.

## Rules import error messages now

Before for small toaster:
<img width="417" alt="Screen Shot 2021-10-26 at 6 03 25 PM" src="https://user-images.githubusercontent.com/1151048/139132586-3cf77c73-53ac-4066-b01f-2e91ef2da111.png">

After for small toaster for different error conditions:
<img width="358" alt="Screen Shot 2021-10-26 at 6 00 24 PM" src="https://user-images.githubusercontent.com/1151048/139132679-2eeb1ed3-9f6e-4766-a8ed-8804ce3e6963.png">

<img width="396" alt="Screen Shot 2021-10-26 at 6 01 00 PM" src="https://user-images.githubusercontent.com/1151048/139132742-750cd937-f401-44e8-9a10-c21410073b5d.png">

<img width="379" alt="Screen Shot 2021-10-26 at 6 02 29 PM" src="https://user-images.githubusercontent.com/1151048/139132766-21b58bea-7f46-43a6-a0e9-f01632958eab.png">

Before for when you click "See the full error":
<img width="817" alt="Screen Shot 2021-10-26 at 5 58 47 PM" src="https://user-images.githubusercontent.com/1151048/139132980-de1942d6-7b03-4c08-b34a-1fc4a22d5207.png">

After for when you click "See the full error":
<img width="838" alt="Screen Shot 2021-10-27 at 1 48 16 PM" src="https://user-images.githubusercontent.com/1151048/139136581-af1e331e-ed77-4338-8fb0-c2457acd135f.png">

<img width="802" alt="Screen Shot 2021-10-27 at 1 26 31 PM" src="https://user-images.githubusercontent.com/1151048/139135083-9ca56940-30a8-4f83-9355-312307172834.png">

## timeline

Before:
<img width="441" alt="Screen Shot 2021-10-27 at 1 19 00 PM" src="https://user-images.githubusercontent.com/1151048/139136614-8360d6a6-d182-413e-b5d9-b18e3d70dc24.png">

<img width="827" alt="Screen Shot 2021-10-27 at 1 19 08 PM" src="https://user-images.githubusercontent.com/1151048/139136637-f9203ac2-0eea-4a77-9c53-ac2c20ab32e0.png">

After:
<img width="408" alt="Screen Shot 2021-10-27 at 1 49 45 PM" src="https://user-images.githubusercontent.com/1151048/139136758-7532a8ba-6d73-45e2-adbb-6756ee997289.png">

<img width="820" alt="Screen Shot 2021-10-27 at 1 49 50 PM" src="https://user-images.githubusercontent.com/1151048/139136774-26d4a8a2-caf0-4c6f-94d3-a6cd92b79f5f.png">

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Oct 29, 2021
…ssing connections clearer for the end user. (elastic#116490)

## Summary

Fixes issue see on this comment:
elastic#116336 (comment)

* Removes legacy toaster component
* Adds newer toaster component
* Removes issue with the deps array within ReactJS
* Adds utility to give a better network error message to the end user.
* This does effect the timeline component since it shares the same import common component.
* Adds a count of how many rules/timeline items have failed imports
* These error toasters mimic Kibana core's error toaster error message and UI/UX
* Adds e2e tests for imports with actions and error messages for them.

## Rules import error messages now

Before for small toaster:
<img width="417" alt="Screen Shot 2021-10-26 at 6 03 25 PM" src="https://user-images.githubusercontent.com/1151048/139132586-3cf77c73-53ac-4066-b01f-2e91ef2da111.png">

After for small toaster for different error conditions:
<img width="358" alt="Screen Shot 2021-10-26 at 6 00 24 PM" src="https://user-images.githubusercontent.com/1151048/139132679-2eeb1ed3-9f6e-4766-a8ed-8804ce3e6963.png">

<img width="396" alt="Screen Shot 2021-10-26 at 6 01 00 PM" src="https://user-images.githubusercontent.com/1151048/139132742-750cd937-f401-44e8-9a10-c21410073b5d.png">

<img width="379" alt="Screen Shot 2021-10-26 at 6 02 29 PM" src="https://user-images.githubusercontent.com/1151048/139132766-21b58bea-7f46-43a6-a0e9-f01632958eab.png">

Before for when you click "See the full error":
<img width="817" alt="Screen Shot 2021-10-26 at 5 58 47 PM" src="https://user-images.githubusercontent.com/1151048/139132980-de1942d6-7b03-4c08-b34a-1fc4a22d5207.png">

After for when you click "See the full error":
<img width="838" alt="Screen Shot 2021-10-27 at 1 48 16 PM" src="https://user-images.githubusercontent.com/1151048/139136581-af1e331e-ed77-4338-8fb0-c2457acd135f.png">

<img width="802" alt="Screen Shot 2021-10-27 at 1 26 31 PM" src="https://user-images.githubusercontent.com/1151048/139135083-9ca56940-30a8-4f83-9355-312307172834.png">

## timeline

Before:
<img width="441" alt="Screen Shot 2021-10-27 at 1 19 00 PM" src="https://user-images.githubusercontent.com/1151048/139136614-8360d6a6-d182-413e-b5d9-b18e3d70dc24.png">

<img width="827" alt="Screen Shot 2021-10-27 at 1 19 08 PM" src="https://user-images.githubusercontent.com/1151048/139136637-f9203ac2-0eea-4a77-9c53-ac2c20ab32e0.png">

After:
<img width="408" alt="Screen Shot 2021-10-27 at 1 49 45 PM" src="https://user-images.githubusercontent.com/1151048/139136758-7532a8ba-6d73-45e2-adbb-6756ee997289.png">

<img width="820" alt="Screen Shot 2021-10-27 at 1 49 50 PM" src="https://user-images.githubusercontent.com/1151048/139136774-26d4a8a2-caf0-4c6f-94d3-a6cd92b79f5f.png">

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Oct 29, 2021
…ssing connections clearer for the end user. (elastic#116490)

## Summary

Fixes issue see on this comment:
elastic#116336 (comment)

* Removes legacy toaster component
* Adds newer toaster component
* Removes issue with the deps array within ReactJS
* Adds utility to give a better network error message to the end user.
* This does effect the timeline component since it shares the same import common component.
* Adds a count of how many rules/timeline items have failed imports
* These error toasters mimic Kibana core's error toaster error message and UI/UX
* Adds e2e tests for imports with actions and error messages for them.

## Rules import error messages now

Before for small toaster:
<img width="417" alt="Screen Shot 2021-10-26 at 6 03 25 PM" src="https://user-images.githubusercontent.com/1151048/139132586-3cf77c73-53ac-4066-b01f-2e91ef2da111.png">

After for small toaster for different error conditions:
<img width="358" alt="Screen Shot 2021-10-26 at 6 00 24 PM" src="https://user-images.githubusercontent.com/1151048/139132679-2eeb1ed3-9f6e-4766-a8ed-8804ce3e6963.png">

<img width="396" alt="Screen Shot 2021-10-26 at 6 01 00 PM" src="https://user-images.githubusercontent.com/1151048/139132742-750cd937-f401-44e8-9a10-c21410073b5d.png">

<img width="379" alt="Screen Shot 2021-10-26 at 6 02 29 PM" src="https://user-images.githubusercontent.com/1151048/139132766-21b58bea-7f46-43a6-a0e9-f01632958eab.png">

Before for when you click "See the full error":
<img width="817" alt="Screen Shot 2021-10-26 at 5 58 47 PM" src="https://user-images.githubusercontent.com/1151048/139132980-de1942d6-7b03-4c08-b34a-1fc4a22d5207.png">

After for when you click "See the full error":
<img width="838" alt="Screen Shot 2021-10-27 at 1 48 16 PM" src="https://user-images.githubusercontent.com/1151048/139136581-af1e331e-ed77-4338-8fb0-c2457acd135f.png">

<img width="802" alt="Screen Shot 2021-10-27 at 1 26 31 PM" src="https://user-images.githubusercontent.com/1151048/139135083-9ca56940-30a8-4f83-9355-312307172834.png">

## timeline

Before:
<img width="441" alt="Screen Shot 2021-10-27 at 1 19 00 PM" src="https://user-images.githubusercontent.com/1151048/139136614-8360d6a6-d182-413e-b5d9-b18e3d70dc24.png">

<img width="827" alt="Screen Shot 2021-10-27 at 1 19 08 PM" src="https://user-images.githubusercontent.com/1151048/139136637-f9203ac2-0eea-4a77-9c53-ac2c20ab32e0.png">

After:
<img width="408" alt="Screen Shot 2021-10-27 at 1 49 45 PM" src="https://user-images.githubusercontent.com/1151048/139136758-7532a8ba-6d73-45e2-adbb-6756ee997289.png">

<img width="820" alt="Screen Shot 2021-10-27 at 1 49 50 PM" src="https://user-images.githubusercontent.com/1151048/139136774-26d4a8a2-caf0-4c6f-94d3-a6cd92b79f5f.png">

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
kibanamachine added a commit that referenced this issue Oct 29, 2021
…ssing connections clearer for the end user. (#116490) (#116804)

## Summary

Fixes issue see on this comment:
#116336 (comment)

* Removes legacy toaster component
* Adds newer toaster component
* Removes issue with the deps array within ReactJS
* Adds utility to give a better network error message to the end user.
* This does effect the timeline component since it shares the same import common component.
* Adds a count of how many rules/timeline items have failed imports
* These error toasters mimic Kibana core's error toaster error message and UI/UX
* Adds e2e tests for imports with actions and error messages for them.

## Rules import error messages now

Before for small toaster:
<img width="417" alt="Screen Shot 2021-10-26 at 6 03 25 PM" src="https://user-images.githubusercontent.com/1151048/139132586-3cf77c73-53ac-4066-b01f-2e91ef2da111.png">

After for small toaster for different error conditions:
<img width="358" alt="Screen Shot 2021-10-26 at 6 00 24 PM" src="https://user-images.githubusercontent.com/1151048/139132679-2eeb1ed3-9f6e-4766-a8ed-8804ce3e6963.png">

<img width="396" alt="Screen Shot 2021-10-26 at 6 01 00 PM" src="https://user-images.githubusercontent.com/1151048/139132742-750cd937-f401-44e8-9a10-c21410073b5d.png">

<img width="379" alt="Screen Shot 2021-10-26 at 6 02 29 PM" src="https://user-images.githubusercontent.com/1151048/139132766-21b58bea-7f46-43a6-a0e9-f01632958eab.png">

Before for when you click "See the full error":
<img width="817" alt="Screen Shot 2021-10-26 at 5 58 47 PM" src="https://user-images.githubusercontent.com/1151048/139132980-de1942d6-7b03-4c08-b34a-1fc4a22d5207.png">

After for when you click "See the full error":
<img width="838" alt="Screen Shot 2021-10-27 at 1 48 16 PM" src="https://user-images.githubusercontent.com/1151048/139136581-af1e331e-ed77-4338-8fb0-c2457acd135f.png">

<img width="802" alt="Screen Shot 2021-10-27 at 1 26 31 PM" src="https://user-images.githubusercontent.com/1151048/139135083-9ca56940-30a8-4f83-9355-312307172834.png">

## timeline

Before:
<img width="441" alt="Screen Shot 2021-10-27 at 1 19 00 PM" src="https://user-images.githubusercontent.com/1151048/139136614-8360d6a6-d182-413e-b5d9-b18e3d70dc24.png">

<img width="827" alt="Screen Shot 2021-10-27 at 1 19 08 PM" src="https://user-images.githubusercontent.com/1151048/139136637-f9203ac2-0eea-4a77-9c53-ac2c20ab32e0.png">

After:
<img width="408" alt="Screen Shot 2021-10-27 at 1 49 45 PM" src="https://user-images.githubusercontent.com/1151048/139136758-7532a8ba-6d73-45e2-adbb-6756ee997289.png">

<img width="820" alt="Screen Shot 2021-10-27 at 1 49 50 PM" src="https://user-images.githubusercontent.com/1151048/139136774-26d4a8a2-caf0-4c6f-94d3-a6cd92b79f5f.png">

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios

Co-authored-by: Frank Hassanabad <[email protected]>
kibanamachine added a commit that referenced this issue Oct 29, 2021
…ssing connections clearer for the end user. (#116490) (#116803)

## Summary

Fixes issue see on this comment:
#116336 (comment)

* Removes legacy toaster component
* Adds newer toaster component
* Removes issue with the deps array within ReactJS
* Adds utility to give a better network error message to the end user.
* This does effect the timeline component since it shares the same import common component.
* Adds a count of how many rules/timeline items have failed imports
* These error toasters mimic Kibana core's error toaster error message and UI/UX
* Adds e2e tests for imports with actions and error messages for them.

## Rules import error messages now

Before for small toaster:
<img width="417" alt="Screen Shot 2021-10-26 at 6 03 25 PM" src="https://user-images.githubusercontent.com/1151048/139132586-3cf77c73-53ac-4066-b01f-2e91ef2da111.png">

After for small toaster for different error conditions:
<img width="358" alt="Screen Shot 2021-10-26 at 6 00 24 PM" src="https://user-images.githubusercontent.com/1151048/139132679-2eeb1ed3-9f6e-4766-a8ed-8804ce3e6963.png">

<img width="396" alt="Screen Shot 2021-10-26 at 6 01 00 PM" src="https://user-images.githubusercontent.com/1151048/139132742-750cd937-f401-44e8-9a10-c21410073b5d.png">

<img width="379" alt="Screen Shot 2021-10-26 at 6 02 29 PM" src="https://user-images.githubusercontent.com/1151048/139132766-21b58bea-7f46-43a6-a0e9-f01632958eab.png">

Before for when you click "See the full error":
<img width="817" alt="Screen Shot 2021-10-26 at 5 58 47 PM" src="https://user-images.githubusercontent.com/1151048/139132980-de1942d6-7b03-4c08-b34a-1fc4a22d5207.png">

After for when you click "See the full error":
<img width="838" alt="Screen Shot 2021-10-27 at 1 48 16 PM" src="https://user-images.githubusercontent.com/1151048/139136581-af1e331e-ed77-4338-8fb0-c2457acd135f.png">

<img width="802" alt="Screen Shot 2021-10-27 at 1 26 31 PM" src="https://user-images.githubusercontent.com/1151048/139135083-9ca56940-30a8-4f83-9355-312307172834.png">

## timeline

Before:
<img width="441" alt="Screen Shot 2021-10-27 at 1 19 00 PM" src="https://user-images.githubusercontent.com/1151048/139136614-8360d6a6-d182-413e-b5d9-b18e3d70dc24.png">

<img width="827" alt="Screen Shot 2021-10-27 at 1 19 08 PM" src="https://user-images.githubusercontent.com/1151048/139136637-f9203ac2-0eea-4a77-9c53-ac2c20ab32e0.png">

After:
<img width="408" alt="Screen Shot 2021-10-27 at 1 49 45 PM" src="https://user-images.githubusercontent.com/1151048/139136758-7532a8ba-6d73-45e2-adbb-6756ee997289.png">

<img width="820" alt="Screen Shot 2021-10-27 at 1 49 50 PM" src="https://user-images.githubusercontent.com/1151048/139136774-26d4a8a2-caf0-4c6f-94d3-a6cd92b79f5f.png">

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios

Co-authored-by: Frank Hassanabad <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Detections and Resp Security Detection Response Team Team:Security Solution Platform Security Solution Platform Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v7.16.0 v8.0.0
Projects
None yet
Development

No branches or pull requests

4 participants