[EDR workflows] Osquery serverless tests #163795

tomsonpl · 2023-08-14T10:52:00Z

This PR:

adds TIER tests for response actions in serverless
change the way we login users in osquery cypress tests
enable passing role config and built-in roles to login
run osquery tests (reuse osquery cypress test suites) in serverless

…-ref HEAD~1..HEAD --fix'

…fix'

patrykkopycinski · 2023-08-16T09:35:37Z

x-pack/test_serverless/functional/test_suites/security/cypress/package.json

@@ -7,7 +7,8 @@
  "scripts": {
    "cypress": "../../../../../../node_modules/.bin/cypress",
    "cypress:open": "node ../../../../../plugins/security_solution/scripts/start_cypress_parallel open --config-file ../../../x-pack/test_serverless/functional/test_suites/security/cypress/cypress.config.ts --ftr-config-file ../../../../../../x-pack/test_serverless/functional/test_suites/security/cypress/security_config",
-    "cypress:run": "node ../../../../../plugins/security_solution/scripts/start_cypress_parallel run --browser chrome --config-file ../../../x-pack/test_serverless/functional/test_suites/security/cypress/cypress.config.ts --ftr-config-file ../../../../../../x-pack/test_serverless/functional/test_suites/security/cypress/security_config --reporter ../../../../../../node_modules/cypress-multi-reporters --reporter-options configFile=./reporter_config.json; status=$?; yarn junit:merge && exit $status",
+    "cypress:run": "node ../../../../../plugins/security_solution/scripts/start_cypress_parallel run --browser chrome --config-file ../../../x-pack/test_serverless/functional/test_suites/security/cypress/cypress.config.ts --ftr-config-file ../../../../../../x-pack/test_serverless/functional/test_suites/security/cypress/security_config --reporter ../../../../../../node_modules/cypress-multi-reporters --reporter-options configFile=./reporter_config.json --concurrency=1 ; status=$?; yarn junit:merge && exit $status",
+    "osquery:cypress:open": "node ../../../../../plugins/security_solution/scripts/start_cypress_parallel open --config-file ../../../x-pack/test_serverless/functional/test_suites/security/cypress/cypress_osquery.config.ts --ftr-config-file ../../../../../../x-pack/test_serverless/functional/test_suites/security/cypress/osquery_security_config",


I don't think we need that here

moved to osquery plugin 👍

patrykkopycinski · 2023-08-16T09:36:13Z

x-pack/test/osquery_cypress/artifact_manager.ts

@@ -6,5 +6,5 @@
 */

 export async function getLatestVersion(): Promise<string> {
-  return '8.9.0-SNAPSHOT';


…-fix'

…fix'

patrykkopycinski · 2023-08-22T15:10:10Z

x-pack/plugins/osquery/cypress.config.ts

  },
+  numTestsKeptInMemory: 3,


do we need that on CI?

added this because of failing tests (chrome renderer). But might be not needed. I'll remove it and test.

patrykkopycinski · 2023-08-22T15:11:25Z

x-pack/plugins/osquery/cypress/e2e/all/alerts_cases.cy.ts

@@ -72,10 +73,10 @@ describe('Alert Event Details - Cases', () => {
      cy.getBySel('expand-event').first().click({ force: true });
      cy.getBySel('take-action-dropdown-btn').click();
      cy.getBySel('osquery-action-item').click();
+      // here


patrykkopycinski · 2023-08-22T15:11:47Z

x-pack/plugins/osquery/cypress/e2e/all/alerts_cases.cy.ts

@@ -91,7 +92,8 @@ describe('Alert Event Details - Cases', () => {
    });
  });

-  describe('Case', () => {
+  // verify why calling new action doesnt add to response actions list
+  describe.skip('Case', () => {


maybe we could use tags instead of skipping it fully?

this tests fails on both envs, I'll have to take a look into why that functionality broke on new flyout.

jbudz · 2023-08-22T23:26:05Z

.buildkite/scripts/steps/functional/security_serverless_osquery.sh

+
+set -euo pipefail
+
+source .buildkite/scripts/common/util.sh


I think we can replace this with

Suggested change

source .buildkite/scripts/common/util.sh

source .buildkite/scripts/steps/functional/common.sh

and remove lines 8 and 9. Sorry about the earlier suggestion, I got the abstraction layers mixed up.

jbudz

Small comment above, overall LGTM

This reverts commit d39e031.

kibana-ci · 2023-08-23T09:24:16Z

💔 Build Failed

Failed CI Steps

Test Failures

[job] [logs] FTR Configs #5 / Agents "before all" hook in "Agents"

Metrics [docs]

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
`osquery`	110	114	+4

Total ESLint disabled count

id	before	after	diff
`osquery`	111	115	+4

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @tomsonpl

kibanamachine · 2023-08-23T10:01:22Z

💚 All backports created successfully

Status	Branch	Result
✅	8.10

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

(cherry picked from commit 054cdba)

# Backport This will backport the following commits from `main` to `8.10`: - [[EDR workflows] Osquery serverless tests (#163795)](#163795)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Tomasz Ciecierski <[email protected]>

* main: (150 commits) Fixes unnecessary autocompletes on HTTP methods (elastic#163233) [Defend Workflows] Convert filterQuery to kql (elastic#161806) [Fleet] copy `inactivity_timeout` when duplicating agent policy (elastic#164544) Fix 7.17 forward compatibility with 8.2+ (elastic#164274) [ML] Fixes dark mode in flyouts and modals (elastic#164399) [Defend Workflows]Changes to policy settings are not persistent until a refresh (elastic#164403) [Security Solution][Endpoint] Fixes kibana crash when going back to policy details page (elastic#164329) Prepare the Security domain HTTP APIs for Serverless (elastic#162087) skip failing test suite (elastic#160986) [Security Solution] Fix flaky Event Filters test (elastic#164473) [EDR workflows] Osquery serverless tests (elastic#163795) [Fleet] Only show agent dashboard links if there is more than one non-server agent and if the dashboards exist (elastic#164469) [Chrome UI] Fix background color in serverless (elastic#164419) [DOCS] Saved objects - resolve import errors API (elastic#162825) Remove 'Create Rule' button from Rule Group page (elastic#164167) [Security Solution] expandable flyout - fix infinite loop in correlations (elastic#163450) [Remote Clusters] Update copy about port help text (elastic#164442) [api-docs] 2023-08-23 Daily api_docs build (elastic#164524) [data views] Disable scripted fields in serverless environment (elastic#163228) [Reporting] Fix - show diagnostic only when image reporting is enabled (elastic#164336) ...

tomsonpl and others added 11 commits August 14, 2023 12:49

wip

da74227

[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…

e96b274

…-ref HEAD~1..HEAD --fix'

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

ee821ed

…fix'

test

c5348d0

Merge remote-tracking branch 'origin/cypress-login' into cypress-login

a3f7c8f

Merge branch 'main' into cypress-login

10a72e1

test

bbc4605

Merge branch 'main' into cypress-login

429ca68

fix

14d26f8

fix

6e9c3d8

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

3bd9fd2

…fix'

patrykkopycinski reviewed Aug 16, 2023

View reviewed changes

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

e1dc322

…-fix'

tomsonpl changed the title ~~[EDR workflows] osquery tests (approach 2)~~ [EDR workflows] Osquery serverless tests Aug 16, 2023

tomsonpl and others added 15 commits August 16, 2023 13:31

test- add buildkite

054b9b0

Merge remote-tracking branch 'origin/cypress-login' into cypress-login

e67fb56

Merge branch 'main' into cypress-login

cbdb767

fix

453850e

fix

47a18c6

Merge remote-tracking branch 'origin/cypress-login' into cypress-login

41d6384

fix

a0db043

fix

177bc0e

fix

d330bef

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

7191f8d

…fix'

fix

e2da32c

Merge remote-tracking branch 'origin/cypress-login' into cypress-login

be09e33

fix

585ce18

Merge branch 'main' into cypress-login

c3770df

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

af5a765

…fix'

tomsonpl added 7 commits August 22, 2023 11:01

change string into enum

541a229

fix roles

48f51d2

move roles

2671d97

fix export

c39e693

rename roles to types

13d5c91

fix packs issue

3107e3f

alert rule issue

fd8bd66

tomsonpl requested review from jbudz and patrykkopycinski August 22, 2023 15:06

patrykkopycinski reviewed Aug 22, 2023

View reviewed changes

tomsonpl added 4 commits August 22, 2023 17:49

get ruleName differently

f5588ed

change way to select rule

8233637

remove wait

717ba85

fix flyout

30d1203

jbudz reviewed Aug 22, 2023

View reviewed changes

jbudz approved these changes Aug 22, 2023

View reviewed changes

tomsonpl and others added 5 commits August 23, 2023 08:09

change ci job

d39e031

Revert "change ci job"

9fadd64

This reverts commit d39e031.

Merge branch 'main' into cypress-login

75d81c1

fix

b157854

Merge remote-tracking branch 'origin/cypress-login' into cypress-login

c8cf993

tomsonpl merged commit 054cdba into elastic:main Aug 23, 2023

kibanamachine mentioned this pull request Aug 23, 2023

[8.10] [EDR workflows] Osquery serverless tests (#163795) #164545

Merged

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Aug 23, 2023

[EDR workflows] Osquery serverless tests (elastic#163795)

e347562

(cherry picked from commit 054cdba)

This was referenced Oct 1, 2023

[8.10] fix(slo): Handle partial indicator url state (#167247) #167722

Merged

[8.10] backport SLO fixes #168231

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[EDR workflows] Osquery serverless tests #163795

[EDR workflows] Osquery serverless tests #163795

tomsonpl commented Aug 14, 2023 •

edited

Loading

patrykkopycinski Aug 16, 2023

tomsonpl Aug 16, 2023

patrykkopycinski Aug 16, 2023

patrykkopycinski Aug 22, 2023

tomsonpl Aug 22, 2023

patrykkopycinski Aug 22, 2023

patrykkopycinski Aug 22, 2023

tomsonpl Aug 22, 2023

jbudz Aug 22, 2023

jbudz left a comment

kibana-ci commented Aug 23, 2023 •

edited

Loading

ESLint disabled line counts

Total ESLint disabled count

kibanamachine commented Aug 23, 2023

	source .buildkite/scripts/common/util.sh
	source .buildkite/scripts/steps/functional/common.sh

[EDR workflows] Osquery serverless tests #163795

[EDR workflows] Osquery serverless tests #163795

Conversation

tomsonpl commented Aug 14, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbudz left a comment

Choose a reason for hiding this comment

kibana-ci commented Aug 23, 2023 • edited Loading

💔 Build Failed

Failed CI Steps

Test Failures

Metrics [docs]

ESLint disabled line counts

Total ESLint disabled count

History

kibanamachine commented Aug 23, 2023

💚 All backports created successfully

Questions ?

tomsonpl commented Aug 14, 2023 •

edited

Loading

kibana-ci commented Aug 23, 2023 •

edited

Loading