[Security Solution] Support Serverless Cypress tests with different roles #169017

maximpn · 2023-10-16T18:37:23Z

Addresses: #164451

Summary

This PR allows to run role based reused between ESS and Serverless Cypress tests.

Details

The main idea behind is to make environmental differences for tests unnoticeable. As Serverless env already has roles and users but ESS env allows to create any possible role and user we just need to create Serverless roles and corresponding users + specific ESS roles and corresponding users in ESS env before running any ESS tests. This way tests will run in a similar env and don't have to bother by roles/users creation in test suites. This is achieved by using Cypress support file (Cypress includes support/e2e.js by default) e2e.ts executed for corresponding environments.

Only one user created per role and user has the same name as its corresponding role with changeme password.

To have an ability to create roles we need to store their definitions somewhere. It's also convenient to have JSON definitions instead of YAML. Plus Serverless roles should be pulled from project-controller repo but it's not addressed in this PR. I've chosen the following locations

Serverless Security roles in packages/kbn-es/src/serverless_resources/security_roles.json. While @kbn/es is a common package it has serverless_resources folder containing roles.yml with a mix of https://github.com/elastic/project-controller/blob/main/internal/project/observability/config/roles.yml, https://github.com/elastic/project-controller/blob/main/internal/project/esproject/config/roles.yml and https://github.com/elastic/project-controller/blob/main/internal/project/security/config/roles.yml copied from project-controller and used for ES data restore. As there is no automation yet it looks logical to keep Security roles subset next to ES Serverless resources.
ESS Security specific roles in x-pack/plugins/security_solution/common/test/ess_roles.json

On top of that the following has been done

reader role replaced with t1_analyst where possible in tests (besides e2e/explore/cases/attach_alert_to_case.cy.ts but it's purely ESS test so it's fine) as reader is ESS specific and make harder to run the same tests in ESS and Serverless environments but both roles are almost equivalent
login() helper function accepts all known roles (Serverless + ESS) but throws an exception if a custom ESS role is used under Serverless env
x-pack/plugins/security_solution/server/lib/detection_engine/scripts/roles_users isn't necessary anymore as security_roles.json + ess_roles.json contain all the necessary data to create roles and users

Does it enable role support for MKI environments?

No. This PR only enabling role support for Non-MKI Serverless environments. MKI env has predefined roles but not users. This will be addressed in a follow up PR.

Flaky test runner

Two unskiped in this PR Serverless Cypress tests using non default role detection_response/detection_alerts/missing_privileges_callout.cy.ts and detection_response/prebuilt_rules/prebuilt_rules_install_update_authorization.cy.ts 150 runs 🟢 (there is one env related failure but it doesn't look related to the changes in this PR)

MadameSheema · 2023-10-25T09:56:52Z

x-pack/test/security_solution_cypress/cypress/cypress_ci_serverless.config.ts

@@ -16,6 +16,7 @@ export default defineCypressConfig({
  },
  defaultCommandTimeout: 150000,
  env: {
+    IS_SERVERLESS: true,


This value is already set in our test code, more specifically in the parallel script. Having it here as well may lead to misunderstandings or wrong values, we must have it in just one place. Personally, I would prefer having it outside this configuration file.

MadameSheema · 2023-10-25T09:59:02Z

x-pack/test/security_solution_cypress/cypress/cypress_ci_serverless_qa.config.ts

@@ -16,6 +16,7 @@ export default defineCypressConfig({
  },
  defaultCommandTimeout: 150000,
  env: {
+    CLOUD_SERVERLESS: true,


This value is already set in our test code, more specifically in the scripts we call in our package.json. Having it here as well may lead to misunderstandings or wrong values, we must have it in just one place. Personally, I would prefer to have it outside this configuration file.

MadameSheema · 2023-10-25T09:59:22Z

x-pack/test/security_solution_cypress/cypress/cypress_serverless.config.ts

@@ -22,10 +22,12 @@ export default defineCypressConfig({
  viewportWidth: 1680,
  numTestsKeptInMemory: 10,
  env: {
+    IS_SERVERLESS: true,


This value is already set in our test code, more specifically in the parallel script. Having it here as well may lead to misunderstandings or wrong values, we must have it in just one place. Personally, I would prefer to have it outside this configuration file.

…fix'

kibana-ci · 2023-10-31T16:39:48Z

💚 Build Succeeded

Buildkite Build
Commit: ea8a0a5

Metrics [docs]

✅ unchanged

History

💔 Build #172154 failed 71fc147dd9541a3692c32a31c184a0dec0c9f958
💚 Build #172070 succeeded dd97101fa67afdaf91deb44440ab874593fcd658
💛 Build #171817 was flaky d7c4039b57be5bbf7b0cbe7c81aaf39a92533982
💛 Build #171317 was flaky 0228e8044b78fc049ff7d5e3fcc70e4c211bfda8
💔 Build #171310 failed 0c189ac4f3da36ee48c440590dbba386b04c6b1b

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @maximpn

…oles (elastic#169017) **Addresses:** elastic#164451 ## Summary This PR allows to run role based reused between ESS and Serverless Cypress tests. ## Details The main idea behind is to make environmental differences for tests unnoticeable. As Serverless env already has roles and users but ESS env allows to create any possible role and user we just need to create Serverless roles and corresponding users + specific ESS roles and corresponding users in ESS env before running any ESS tests. This way tests will run in a similar env and don't have to bother by roles/users creation in test suites. This is achieved by using separate Cypress support files (Cypress includes `support/e2e.js` by default) `ess_e2e.ts` and `serverless_e2e.ts` executed for corresponding environments. `ess_e2e.ts` contains logic to create mentioned above roles and users while `serverless_e2e.ts` doesn't contain such logic. _Only one user created per role and user has the same name as its corresponding role with `changeme` password._ To have an ability to create roles we need to store their definitions somewhere. It's also convenient to have JSON definitions instead of YAML. Plus Serverless roles should be pulled from `project-controller` repo but it's not addressed in this PR. I've chosen the following locations - Serverless Security roles in `packages/kbn-es/src/serverless_resources/security_roles.json`. While `@kbn/es` is a common package it has `serverless_resources` folder containing `roles.yml` with a mix of `https://github.com/elastic/project-controller/blob/main/internal/project/observability/config/roles.yml`, `https://github.com/elastic/project-controller/blob/main/internal/project/esproject/config/roles.yml` and `https://github.com/elastic/project-controller/blob/main/internal/project/security/config/roles.yml` copied from `project-controller` and used for ES data restore. As there is no automation yet it looks logical to keep Security roles subset next to ES Serverless resources. - ESS Security specific roles in `x-pack/plugins/security_solution/common/test/ess_roles.json` On top of that the following has been done - `reader` role replaced with `t1_analyst` where possible in tests (besides `e2e/explore/cases/attach_alert_to_case.cy.ts` but it's purely ESS test so it's fine) as `reader` is ESS specific and make harder to run the same tests in ESS and Serverless environments but both roles are almost equivalent - `login()` helper function accepts all known roles (Serverless + ESS) but throws an exception if a custom ESS role is used under Serverless env - `x-pack/plugins/security_solution/server/lib/detection_engine/scripts/roles_users` isn't necessary anymore as `security_roles.json` + `ess_roles.json` contain all the necessary data to create roles and users ### Does it enable role support for MKI environments? No. This PR only enabling role support for Non-MKI Serverless environments. MKI env has predefined roles but not users. This will be addressed in a follow up PR. ## Flaky test runner Two unskiped in this PR Serverless Cypress tests using non default role `detection_response/detection_alerts/missing_privileges_callout.cy.ts` and `detection_response/prebuilt_rules/prebuilt_rules_install_update_authorization.cy.ts` [150 runs](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3723) 🟢 (there is one env related failure but it doesn't look related to the changes in this PR)

**Relates to:** #169017 ## Summary This PR re-adds an utility shell script for roles and users creation to replace scripts removed in a previous [PR](#169017).

maximpn self-assigned this Oct 16, 2023

maximpn force-pushed the support_roles_in_serverless_cypress_security_solution branch from 373613e to 51d6847 Compare October 16, 2023 18:44

maximpn requested a review from MadameSheema October 16, 2023 18:45

maximpn force-pushed the support_roles_in_serverless_cypress_security_solution branch 12 times, most recently from 116a374 to 223754f Compare October 19, 2023 21:29

maximpn force-pushed the support_roles_in_serverless_cypress_security_solution branch 4 times, most recently from 71a71fe to 4b8a43c Compare October 25, 2023 08:13

MadameSheema reviewed Oct 25, 2023

View reviewed changes

maximpn force-pushed the support_roles_in_serverless_cypress_security_solution branch 2 times, most recently from ad6a6e2 to 7b75270 Compare October 25, 2023 13:24

maximpn requested a review from jpdjere October 25, 2023 13:24

maximpn added 18 commits October 31, 2023 16:08

remove unused functions

568a355

get rid of "reader" role

31d9de6

add missing @kbn/es dependency

3d9a1f4

fix endpoint roles related tests

2a9d884

remove env type constants

f4be6c2

roll back renaming

a31d611

update readme

19b333b

use only one support file

f5b399d

add back reader role for ESS only tests

3cfcc40

fix a misprint

a29b160

reexport role definitions constants from security solution common folder

b88ad9b

use import type instead of import

b589b10

specify return types

88ef1c4

remove unused package reference

878b6c5

fix role creation functionality

a054d50

fix tsconfig

0dfe598

roll back role type changes

284288a

update readme

416bfc8

maximpn force-pushed the support_roles_in_serverless_cypress_security_solution branch from 71fc147 to 416bfc8 Compare October 31, 2023 15:08

maximpn enabled auto-merge (squash) October 31, 2023 15:08

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

ea8a0a5

…fix'

maximpn merged commit 58adee0 into elastic:main Oct 31, 2023

kibanamachine added the backport:skip This commit does not require backporting label Oct 31, 2023

maximpn mentioned this pull request Oct 31, 2023

[Security Solution] Support roles in Serverless FTR tests #170131

Closed

maximpn mentioned this pull request Nov 11, 2023

[Security Solution] Add a script to create roles and users #171067

Merged

banderror mentioned this pull request Nov 13, 2023

[Security Solution] Support testing with different roles in Serverless Cypress and API integration tests #164451

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solution] Support Serverless Cypress tests with different roles #169017

[Security Solution] Support Serverless Cypress tests with different roles #169017

maximpn commented Oct 16, 2023 •

edited

Loading

MadameSheema Oct 25, 2023 •

edited

Loading

MadameSheema Oct 25, 2023

MadameSheema Oct 25, 2023

kibana-ci commented Oct 31, 2023

[Security Solution] Support Serverless Cypress tests with different roles #169017

[Security Solution] Support Serverless Cypress tests with different roles #169017

Conversation

maximpn commented Oct 16, 2023 • edited Loading

Summary

Details

Does it enable role support for MKI environments?

Flaky test runner

MadameSheema Oct 25, 2023 • edited Loading

Choose a reason for hiding this comment

MadameSheema Oct 25, 2023

Choose a reason for hiding this comment

MadameSheema Oct 25, 2023

Choose a reason for hiding this comment

kibana-ci commented Oct 31, 2023

💚 Build Succeeded

Metrics [docs]

History

maximpn commented Oct 16, 2023 •

edited

Loading

MadameSheema Oct 25, 2023 •

edited

Loading