Failing test: X-Pack Endpoint Functional Tests.x-pack/test/security_solution_endpoint/apps/endpoint/endpoint_solution_integrations·ts - endpoint App level Endpoint functionality from Timeline "before all" hook for "should show Isolation action in alert details"

[kibanamachine]

A test failed on a tracked branch

Error: Timeout of 360000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/var/lib/buildkite-agent/builds/kb-n2-4-spot-cd1f6217d1bd1b3e/elastic/kibana-on-merge/kibana/x-pack/test/security_solution_endpoint/apps/endpoint/endpoint_solution_integrations.ts)
    at listOnTimeout (node:internal/timers:559:17)
    at processTimers (node:internal/timers:502:7) {
  code: 'ERR_MOCHA_TIMEOUT',
  timeout: 360000,
  file: '/var/lib/buildkite-agent/builds/kb-n2-4-spot-cd1f6217d1bd1b3e/elastic/kibana-on-merge/kibana/x-pack/test/security_solution_endpoint/apps/endpoint/endpoint_solution_integrations.ts'
}

First failure: CI Build - main

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-onboarding-and-lifecycle-mgt (Team:Onboarding and Lifecycle Mgt)

[paul-tavares]

Will take a look. This (I think) has something to do with the Endpoint rule perhaps not processing the events when stopped/started.

[paul-tavares]

The failure seems to have occurred while waiting for Alerts to be created by the Endpoint Rule in security solution. We keep pinging the API until the alerts for the endpoint data we loaded for the test case shows up, but for whatever reason, it never shows up.

I would like to see the Kibana server log to see if any errors were logged there while the test was running. The fact that we timed out waiting for the alerts might indicate that the Endpoint rule never ran or might have encountered an error while it was running.

Not sure what might be causing this to fail. Will continue to monitor

[kibanamachine]

New failure: CI Build - 8.5

[kibanamachine]

New failure: CI Build - 8.5

[kibanamachine]

New failure: CI Build - main

[spalger]

6 failures in the last week across 8.5 and main, skipped

main/8.6: 15e4a3f
8.5: d9247c8

[kevinlog]

@paul-tavares

I would like to see the Kibana server log to see if any errors were logged there while the test was running. The fact that we timed out waiting for the alerts might indicate that the Endpoint rule never ran or might have encountered an error while it was running.

Can we populate alerts directly into the alerts index and not wait for Endpoint rule promotion? My guess is that the Detections team does something like this with their integration tests in Alerts.

[paul-tavares]

@kevinlog ,

re: "...Can we populate alerts directly into the alerts index..."

Maybe. I'll look into it to see if it's possible.

[paul-tavares]

PR opened: #144133

@kevinlog ,
I seem ( 🤞 ) to have been successful at loading Endpoint (rule) Alerts directly to the index instead of waiting for the rule to run. Let's hope that does it.