Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Switch cypress and api integration test data from es archive to programmatically generated in certain files #120279

Closed
dplumlee opened this issue Dec 2, 2021 · 4 comments
Labels
bug Fixes for quality problems that affect the customer experience Team:Detection Alerts Security Detection Alerts Area Team Team:Detections and Resp Security Detection Response Team Team:Security Solution Platform Security Solution Platform Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. technical debt Improvement of the software architecture and operational architecture

Comments

@dplumlee
Copy link
Contributor

dplumlee commented Dec 2, 2021

Related to #116374

In some of the Security Solution Rules cypress and API integration tests, we rely on es archive data that has been timestamped years ago and just extend the rule look back time to cover the time period. This isn't a reliable practice and while the 50,000 hours value we often use should cover our current test data, there are tests and use cases that can't have a look back time hardcoded.

We should look into a different approach of generating data to create a more realistic testing environment and not have to worry about tests breaking because some arbitrary time has passed

This would also allow us to unskip and update the tests skipped in the linked PR. More detail about the specific tests is listed in its summary

@dplumlee dplumlee added bug Fixes for quality problems that affect the customer experience Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Alerts Security Detection Alerts Area Team labels Dec 2, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@peluja1012 peluja1012 added technical debt Improvement of the software architecture and operational architecture Team:Security Solution Platform Security Solution Platform Team labels Dec 2, 2021
@dplumlee dplumlee changed the title [Security Solution] Switch cypress test data from es archive to programmatically generated in certain files [Security Solution] Switch cypress and api integration test data from es archive to programmatically generated in certain files Dec 7, 2021
@dplumlee
Copy link
Contributor Author

dplumlee commented Dec 7, 2021

I added API integration tests to this ticket as they have a similar issue. No tests were skipped in the linked PR but the inability to have generated data is causing shortages in testing coverage for the same reason as the cypress tests. Namely that we can't rely upon a hardcoded timestamp to retrieve data. Whatever tool we build and integrate to address this should be able to be used in both testing suites

@MindyRS MindyRS added the Team:Detections and Resp Security Detection Response Team label Feb 23, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@marshallmain
Copy link
Contributor

Superseded by #126389

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience Team:Detection Alerts Security Detection Alerts Area Team Team:Detections and Resp Security Detection Response Team Team:Security Solution Platform Security Solution Platform Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. technical debt Improvement of the software architecture and operational architecture
Projects
None yet
Development

No branches or pull requests

5 participants