Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Preparing Cypress to be able to execute on a real serverless environment #166905

Merged

Conversation

MadameSheema
Copy link
Member

@MadameSheema MadameSheema commented Sep 21, 2023

Work done to be prepared for these changes

Summary

This PR provides the changes needed to be able to execute cypress tests in a real serverless environment.

How to test it

  • Create a deployed project (you can use QAF for the creation and management of it. It is a pretty convenient tool since it easily provides you the password and the username)

  • Execute cypress in your machine from x-pack/test/security_solution_cypress:

    • For visual mode:
      CYPRESS_ELASTICSEARCH_URL=<elsUrl> CYPRESS_BASE_URL=<kbUrl>  CYPRESS_ELASTICSEARCH_USERNAME=<username> CYPRESS_ELASTICSEARCH_PASSWORD=<password> CYPRESS_KIBANA_URL=$CYPRESS_BASE_URL yarn cypress:open:cloud:serverless
    • For headless mode:
      CYPRESS_ELASTICSEARCH_URL=<elsUrl> CYPRESS_BASE_URL=<kbUrl>  CYPRESS_ELASTICSEARCH_USERNAME=<username> CYPRESS_ELASTICSEARCH_PASSWORD=<password> CYPRESS_KIBANA_URL=$CYPRESS_BASE_URL yarn cypress:run:cloud:serverless

Known limitations

  • With this initial setup there is no differentiation between different executions
  • Only tests for complete can be executed
  • The data ingested is the one in our archives

To follow up

  • Differentiation between essentials and complete
  • Integration with the proper quality gate
  • Integration with buildkite
  • Do we want to execute all the tests?

Blockers

We need to have cypress at least in 12.15.0 version to be able to execute them on a real serverless since from that version test pages with targeted Content-Security-Policy and Content-Security-Policy-Report-Only header directives can be tested by specifying the allow list via the experimentalCspAllowList configuration option.

#165869

Blocker solved :)

MadameSheema and others added 6 commits September 8, 2023 18:56
@MadameSheema MadameSheema force-pushed the cypress/security-real-environment branch from 56c0e53 to 5a612d9 Compare September 21, 2023 06:42
@MadameSheema MadameSheema changed the title intial setup [Security Solution] Preparing Cypress to be able to execute on a real serverless environment Sep 21, 2023
@MadameSheema
Copy link
Member Author

MadameSheema commented Sep 21, 2023

Closing in favor of: #166905

@MadameSheema MadameSheema self-assigned this Sep 21, 2023
@MadameSheema MadameSheema reopened this Sep 21, 2023
@MadameSheema MadameSheema added Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. release_note:skip Skip the PR/issue when compiling release notes labels Sep 21, 2023
@MadameSheema MadameSheema marked this pull request as ready for review September 21, 2023 08:47
@MadameSheema MadameSheema requested review from a team as code owners September 21, 2023 08:47
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@banderror banderror requested a review from maximpn September 21, 2023 09:23
Copy link
Contributor

@semd semd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@kibana-ci
Copy link
Collaborator

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] Explore - Security Solution Cypress Tests #1 / Cases Creates a new case with timeline and opens the timeline Creates a new case with timeline and opens the timeline
  • [job] [logs] FTR Configs #29 / lens app - Agg based Vis Open in Lens Gauge "before each" hook for "should convert aggregation with params"
  • [job] [logs] FTR Configs #8 / serverless security UI Create Case "before each" hook for "creates a case"

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @MadameSheema

Copy link
Contributor

@maximpn maximpn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -27,6 +27,7 @@ export default defineCypressConfig({
e2e: {
experimentalRunAllSpecs: true,
experimentalMemoryManagement: true,
experimentalCspAllowList: ['default-src', 'script-src', 'script-src-elem'],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit of out context here and just curious why we permit CSP directives now? It looks like an orthogonal change in this PR.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @maximpn!! In order to be able to execute Cypress we were setting the csp parameters when creating the instances for testing in x-pack/test/security_solution_cypress/config.ts. In a real (deployed) serverless environment, we don't have a way of doing it, so the above line helps us to execute the tests without facing any issues.

@MadameSheema MadameSheema merged commit 1f74f23 into elastic:main Sep 22, 2023
@MadameSheema MadameSheema deleted the cypress/security-real-environment branch September 22, 2023 16:26
@kibanamachine kibanamachine added the backport:skip This commit does not require backporting label Sep 22, 2023
joemcelroy pushed a commit to joemcelroy/kibana that referenced this pull request Sep 25, 2023
… serverless environment (elastic#166905)

Co-authored-by: YulNaumenko <[email protected]>
Co-authored-by: Sergi Massaneda <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting release_note:skip Skip the PR/issue when compiling release notes Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.11.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants