Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Security Solution] Preparing Cypress to be able to execute on a real serverless environment #166109

Closed
wants to merge 5 commits into from
Closed

[Security Solution] Preparing Cypress to be able to execute on a real serverless environment #166109

wants to merge 5 commits into from

Conversation

MadameSheema
Copy link
Member

@MadameSheema MadameSheema commented Sep 8, 2023 •
edited
Loading

Work done to be prepared for these changes

Summary

This PR provides the changes needed to be able to execute cypress tests in a real serverless environment.

How to test it

  • Create a deployed project (you can use QAF for the creation and management of it. It is a pretty convenient tool since it easily provides you the password and the username)

  • Execute cypress in your machine from x-pack/test/security_solution_cypress:

    • For visual mode:
      CYPRESS_ELASTICSEARCH_URL=<elsUrl> CYPRESS_BASE_URL=<kbUrl>  CYPRESS_ELASTICSEARCH_USERNAME=<username> CYPRESS_ELASTICSEARCH_PASSWORD=<password> CYPRESS_KIBANA_URL=$CYPRESS_BASE_URL yarn cypress:open:cloud:serverless
    • For headless mode:
      CYPRESS_ELASTICSEARCH_URL=<elsUrl> CYPRESS_BASE_URL=<kbUrl>  CYPRESS_ELASTICSEARCH_USERNAME=<username> CYPRESS_ELASTICSEARCH_PASSWORD=<password> CYPRESS_KIBANA_URL=$CYPRESS_BASE_URL yarn cypress:run:cloud:serverless

Known limitations

  • With this initial setup there is no differentiation between different executions
  • Only tests for complete can be executed
  • The data ingested is the one in our archives

To follow up

  • Differentiation between essentials and complete
  • Integration with the proper quality gate
  • Integration with buildkite
  • Do we want to execute all the tests?

Blockers

We need to have cypress at least in 12.15.0 version to be able to execute them on a real serverless since from that version test pages with targeted Content-Security-Policy and Content-Security-Policy-Report-Only header directives can be tested by specifying the allow list via the experimentalCspAllowList configuration option.

#165869

Blocker solved :)

@MadameSheema MadameSheema added release_note:skip Skip the PR/issue when compiling release notes Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.11.0 labels Sep 8, 2023
@MadameSheema MadameSheema self-assigned this Sep 8, 2023
azasypkin
azasypkin reviewed Sep 12, 2023
View reviewed changes
x-pack/test/security_solution_cypress/cypress/tasks/login.ts Outdated
Comment on lines 109 to 116
const getProviderName = () => {
if (Cypress.env('CLOUD_SERVERLESS')) {
return 'cloud-basic';
} else {
return 'basic';
}
};

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note/issue: would you mind switching to the approach I'm using here, so that the tests can work in any current and future environments, no matter how providers are configured? https://github.com/elastic/kibana/pull/165810/files#diff-c20246118002c8381fa4a7b666148fcea8e1a68afa0c3d3100fe362b5e40ebb2R117-R134

@YulNaumenko YulNaumenko marked this pull request as ready for review September 13, 2023 13:19
@YulNaumenko YulNaumenko requested review from a team as code owners September 13, 2023 13:19
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@YulNaumenko
Merge remote-tracking branch 'upstream/main' into cypress/preparing-f…
c42cf58 
…or-real-environment

# Conflicts:
#	x-pack/test/security_solution_cypress/cypress/tasks/login.ts
#	x-pack/test/security_solution_cypress/package.json
@YulNaumenko
changed due to comments
86d54d6
@azasypkin
Copy link
Member

I'm transferring the review request to the @elastic/kibana-security as I might not be available for a while due to PTO. Context: The Security Solution team might need assistance if they encounter any issues with the login sequence in the Cypress tests (see #165810).

@azasypkin azasypkin requested review from a team and removed request for azasypkin September 14, 2023 12:47
@kc13greiner kc13greiner self-requested a review September 18, 2023 14:37
@kibanamachine kibanamachine requested review from a team as code owners September 20, 2023 18:05
@kibanamachine kibanamachine requested review from a team as code owners September 20, 2023 18:05
@kibanamachine kibanamachine requested a review from a team September 20, 2023 18:05
@kibanamachine kibanamachine requested a review from a team as a code owner September 20, 2023 18:05
@botelastic botelastic bot added Feature:Embedding Embedding content via iFrame Feature:ExpressionLanguage Interpreter expression language (aka canvas pipeline) Team:APM All issues that need APM UI Team support Team:Fleet Team label for Observability Data Collection Fleet team Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability labels Sep 20, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/apm-ui (Team:APM)

@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@elasticmachine
Copy link
Contributor

Pinging @elastic/uptime (Team:uptime)

@MadameSheema MadameSheema force-pushed the cypress/preparing-for-real-environment branch from 97289ee to 322623d Compare September 21, 2023 04:55
@MadameSheema MadameSheema marked this pull request as draft September 21, 2023 04:57
@kibana-ci
Copy link
Collaborator

kibana-ci commented Sep 21, 2023 •
edited
Loading

💔 Build Failed

Failed CI Steps

Metrics [docs]

✅ unchanged

History

  • 💔 Build #160512 failed 97289ee38afc9c05a68add60f066ede6a1f678dd
  • 💔 Build #160499 failed 6a3dd512ed5b962302bba5e94a69f42123768082
  • 💛 Build #160382 was flaky 09fadabd1a1caa575a258c4abb66362a7403c329
  • 💔 Build #160335 failed ad7018d4d4aebca331851ca6ca5257cbb5f09370

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @MadameSheema

@MadameSheema
Copy link
Member Author

Closing in favor of: #166905

@MadameSheema MadameSheema deleted the cypress/preparing-for-real-environment branch October 2, 2023 08:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@azasypkin azasypkin azasypkin left review comments

@dkirchan dkirchan dkirchan approved these changes

@kc13greiner kc13greiner Awaiting requested review from kc13greiner

@tomsonpl tomsonpl Awaiting requested review from tomsonpl tomsonpl was automatically assigned from elastic/security-defend-workflows

@paul-tavares paul-tavares Awaiting requested review from paul-tavares paul-tavares was automatically assigned from elastic/security-defend-workflows

Assignees

@MadameSheema MadameSheema

Labels
Feature:Embedding Embedding content via iFrame Feature:ExpressionLanguage Interpreter expression language (aka canvas pipeline) release_note:skip Skip the PR/issue when compiling release notes Team:APM All issues that need APM UI Team support Team:Fleet Team label for Observability Data Collection Fleet team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability v8.11.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@MadameSheema @elasticmachine @azasypkin @kibana-ci @dkirchan @YulNaumenko