Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[APM] E2E tests for rules & alerts #79284

Open
dgieselaar opened this issue Oct 2, 2020 · 2 comments
Open

[APM] E2E tests for rules & alerts #79284

dgieselaar opened this issue Oct 2, 2020 · 2 comments
Assignees
Labels
apm:alerting apm:needs-test Team:APM All issues that need APM UI Team support technical debt Improvement of the software architecture and operational architecture

Comments

@dgieselaar
Copy link
Member

dgieselaar commented Oct 2, 2020

We currently don't have integration tests for our rules and alerts. We should increase coverage by adding API tests and E2E tests.

We should cover the following scenarios with E2E tests:

  • For each rule type, open the create flyout in the APM app, and create a rule. Use the response from the API to test whether the rule was created with the correct parameters.
  • Assert that the form is pre-populated with the correct parameters (e.g. whether the service name was used from the URL).
  • For each rule type, open the create flyout in the stack management UI, and verify that a rule can be created with the correct parameters.
  • For each created rule from the stack management UI, verify that the created rule can be edited and that the parameters can be updated and are passed to the API correctly.
@dgieselaar dgieselaar added the Team:APM All issues that need APM UI Team support label Oct 2, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/apm-ui (Team:apm)

@sorenlouv sorenlouv added [zube]: Inbox technical debt Improvement of the software architecture and operational architecture and removed [zube]: Inbox labels Oct 5, 2020
@dgieselaar dgieselaar changed the title [APM] Integration tests for alerts [APM] Integration tests for rules & alerts Sep 2, 2021
@smith smith self-assigned this Sep 10, 2021
@smith smith removed their assignment Oct 6, 2021
smith added a commit to smith/kibana that referenced this issue Oct 13, 2021
* Run previously disabled APM E2E tests on all PRs (we were previously only running them when APM files had changes.)
* Remove `precise: true` from `getComparisonTypes` call which caused intermittent failures in date comparison tests.
* Simplify error count alert tests to test the "happy path" (elastic#79284 exists in order to expand to more tests for rule editing and creation)
* Wait for alert list API request to complete before clicking "Create rule" button when running the test to create a rule from the Stack Management UI.

I ran the e2e tests 100 times locally with no failures so I'm confident the flakiness has been addressed.

Fixes elastic#114419.
Fixes elastic#109205.
smith added a commit that referenced this issue Oct 15, 2021
* Re-enable previously disabled APM E2E tests.
* Round to the nearest second in `getComparisonTypes` to avoid cases where a millisecond difference can change which results get shown.
* Simplify error count alert tests to test the "happy path" (#79284 exists in order to expand to more tests for rule editing and creation.)
* Wait for alert list API request to complete before clicking "Create rule" button when running the test to create a rule from the Stack Management UI.

I ran the e2e tests 100 times locally with no failures so I'm confident the flakiness has been addressed.

Fixes #114419.
Fixes #109205.
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Oct 15, 2021
* Re-enable previously disabled APM E2E tests.
* Round to the nearest second in `getComparisonTypes` to avoid cases where a millisecond difference can change which results get shown.
* Simplify error count alert tests to test the "happy path" (elastic#79284 exists in order to expand to more tests for rule editing and creation.)
* Wait for alert list API request to complete before clicking "Create rule" button when running the test to create a rule from the Stack Management UI.

I ran the e2e tests 100 times locally with no failures so I'm confident the flakiness has been addressed.

Fixes elastic#114419.
Fixes elastic#109205.
kibanamachine added a commit that referenced this issue Oct 15, 2021
* Re-enable previously disabled APM E2E tests.
* Round to the nearest second in `getComparisonTypes` to avoid cases where a millisecond difference can change which results get shown.
* Simplify error count alert tests to test the "happy path" (#79284 exists in order to expand to more tests for rule editing and creation.)
* Wait for alert list API request to complete before clicking "Create rule" button when running the test to create a rule from the Stack Management UI.

I ran the e2e tests 100 times locally with no failures so I'm confident the flakiness has been addressed.

Fixes #114419.
Fixes #109205.

Co-authored-by: Nathan L Smith <[email protected]>
@kpatticha
Copy link
Contributor

Moving the api test to #155690

We should cover the following scenarios with API tests:- For each rule type, create a rule, and verify whether it fires and recovers appropriately, and fires the correct alerts, e.g. whether the grouping/thresholds etc are interpreted correctly. Query the alerts-as-data APIs/indices to verify whether that is the case.

@sorenlouv sorenlouv changed the title [APM] Integration tests for rules & alerts [APM] E2E tests for rules & alerts Apr 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
apm:alerting apm:needs-test Team:APM All issues that need APM UI Team support technical debt Improvement of the software architecture and operational architecture
Projects
None yet
Development

No branches or pull requests

8 participants