Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/details·ts - Actions and Triggers app Alert Details Alert Instances renders the active alert instances #83478

Merged
merged 9 commits into from
Nov 20, 2020

Conversation

ymao1
Copy link
Contributor

@ymao1 ymao1 commented Nov 16, 2020

Resolves #57426

Summary

The latest flakey test failures were related to showing the action group id as part of the status column on the alert details instance list. Updated the test to explicitly retrieve the action group id from the instance summary and map to action group name to determine what the displayed text should be.

Ran through the flaky test runner 42 times here
I tried to run through it 100 times but the job gets cut off if it runs for too long

@ymao1 ymao1 self-assigned this Nov 17, 2020
@ymao1 ymao1 added Feature:Alerting release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v7.11.0 v8.0.0 labels Nov 17, 2020
@ymao1 ymao1 marked this pull request as ready for review November 17, 2020 18:17
@ymao1 ymao1 requested a review from a team as a code owner November 17, 2020 18:17
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

Copy link
Contributor

@YulNaumenko YulNaumenko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@pmuellr pmuellr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but my understanding from the issue is that there was some kind of date rounding issue - but that report was back from Feb, and the last build that showed the error is no longer available, so ??? hehe.

In general, I'd say this test still feels a little flaky in that it's comparing the instance summary API to the UI, and in theory there could be differences, given when they are run. But I expect the always firing alert to produce pretty stable results :-), so seems fine, especially after running through the flaky test runner. And we're using that instance summary API elsewhere in these tests, so ... it must be working out ok, in practice.

@ymao1
Copy link
Contributor Author

ymao1 commented Nov 18, 2020

LGTM, but my understanding from the issue is that there was some kind of date rounding issue - but that report was back from Feb, and the last build that showed the error is no longer available, so ??? hehe.

The original flaky test failures were due to date issues and those were fixed by previous PRs. This was a failure of the same test, but for a different reason (but because it was the same test, it attached itself to the same original issue). Here is the console output that I copied from Jenkins for posterity.

[00:05:55]                 │       Error: expected [ { instance: 'us-central',
[00:05:55]                 │     status: 'Active (Default)',
[00:05:55]                 │     start: '9 Nov 2020 @ 07:18:34' },
[00:05:55]                 │   { instance: 'us-east',
[00:05:55]                 │     status: 'Active',
[00:05:55]                 │     start: '9 Nov 2020 @ 07:18:34' },
[00:05:55]                 │   { instance: 'us-west',
[00:05:55]                 │     status: 'Active (Default)',
[00:05:55]                 │     start: '9 Nov 2020 @ 07:18:34' } ] to sort of equal [ { instance: 'us-central',
[00:05:55]                 │     status: 'Active (Default)',
[00:05:55]                 │     start: '9 Nov 2020 @ 07:18:34' },
[00:05:55]                 │   { instance: 'us-east',
[00:05:55]                 │     status: 'Active (Default)',
[00:05:55]                 │     start: '9 Nov 2020 @ 07:18:34' },
[00:05:55]                 │   { instance: 'us-west',
[00:05:55]                 │     status: 'Active (Default)',
[00:05:55]                 │     start: '9 Nov 2020 @ 07:18:34' } ]
[00:05:55]                 │       + expected - actual
[00:05:55]                 │ 
[00:05:55]                 │          }
[00:05:55]                 │          {
[00:05:55]                 │            "instance": "us-east"
[00:05:55]                 │            "start": "9 Nov 2020 @ 07:18:34"
[00:05:55]                 │       -    "status": "Active"
[00:05:55]                 │       +    "status": "Active (Default)"
[00:05:55]                 │          }
[00:05:55]                 │          {
[00:05:55]                 │            "instance": "us-west"
[00:05:55]                 │            "start": "9 Nov 2020 @ 07:18:34"

@ymao1
Copy link
Contributor Author

ymao1 commented Nov 18, 2020

@elasticmachine merge upstream

@ymao1
Copy link
Contributor Author

ymao1 commented Nov 20, 2020

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@ymao1 ymao1 merged commit a2598ea into elastic:master Nov 20, 2020
ymao1 added a commit to ymao1/kibana that referenced this pull request Nov 20, 2020
…ctional_with_es_ssl/apps/triggers_actions_ui/details·ts - Actions and Triggers app Alert Details Alert Instances renders the active alert instances (elastic#83478)

* Unskipping test

* Unskipping test

* Making action group name lookup dynamic

* Removing comment

Co-authored-by: Kibana Machine <[email protected]>
ymao1 added a commit that referenced this pull request Nov 20, 2020
…ctional_with_es_ssl/apps/triggers_actions_ui/details·ts - Actions and Triggers app Alert Details Alert Instances renders the active alert instances (#83478) (#83912)

* Unskipping test

* Unskipping test

* Making action group name lookup dynamic

* Removing comment

Co-authored-by: Kibana Machine <[email protected]>

Co-authored-by: Kibana Machine <[email protected]>
gmmorris added a commit to gmmorris/kibana that referenced this pull request Nov 20, 2020
* master: (38 commits)
  [ML] Data frame analytics: Adds functionality to map view (elastic#83710)
  Add usage collection for savedObject tagging (elastic#83160)
  [SECURITY_SOLUTION] 145: Advanced Policy Tests (elastic#82898)
  [APM] Service overview transactions table (elastic#83429)
  [ML] Fix Single Metric Viewer not loading if job is metric with no partition (elastic#83880)
  do not export types from 3rd party modules as 'type' (elastic#83803)
  [Fleet] Allow to send SETTINGS action (elastic#83707)
  Fixes Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/details·ts - Actions and Triggers app Alert Details Alert Instances renders the active alert instances (elastic#83478)
  [Uptime]Reduce chart height on monitor detail page (elastic#83777)
  [APM] Prefer `APIReturnType` over `PromiseReturnType` (elastic#83843)
  [Observability] Fix telemetry for Observability Overview (elastic#83847)
  [Alerting] Adds generic UI for the definition of conditions for Action Groups (elastic#83278)
  ensure workload agg doesnt run until next interval when it fails (elastic#83632)
  [ILM] Policy form should not throw away data (elastic#83077)
  [Monitoring] Stop collecting Kibana Usage in bulkUploader (elastic#83546)
  [TSVB] fix wrong imports (elastic#83798)
  [APM] Correlations UI POC (elastic#82256)
  list all the refs in  tsconfig.json (elastic#83678)
  Bump jest (and related packages) to v26.6.3 (elastic#83724)
  Functional tests - stabilize reporting tests for cloud execution (elastic#83787)
  ...
@ymao1 ymao1 deleted the flaky-test-alert-instances branch February 4, 2021 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Alerting release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v7.11.0 v8.0.0
Projects
None yet
5 participants