[Stack Monitoring] Implement baseline API tests for packages #148303
Labels
Feature:Stack Monitoring
Team:Infra Monitoring UI - DEPRECATED
DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services
Comments
klacabane
added
Team:Infra Monitoring UI - DEPRECATED
|
Pinging @elastic/infra-monitoring-ui (Team:Infra Monitoring UI) |
This was referenced Jan 3, 2023
klacabane
added a commit
that referenced
this issue
Jan 30, 2023
…kage and metricbeat data (#149161) ### Summary Part of #148303 Closes #146067 Closes #146068 I investigated the failing tests but couldn't get to the root cause quickly enough so I took the opportunity to port the tests to the [monitoring_api_integration](https://github.com/elastic/kibana/tree/main/x-pack/test/monitoring_api_integration) suite. This reimplements all the existing tests (plus missing ones for the `ml_jobs` api) with fresh data. By using the new testing approach we can remove the archived mappings which greatly reduce execution times. ### Testing - The suite was executed against by the [flaky test runner](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1804) multiple times with no failures - Test data is bundled in two archives (one for metricbeat and one for package data) and can be loaded to verify their content: `node scripts/es_archiver.js load x-pack/test/monitoring_api_integration/archives/elasticsearch/single_node/(package|metricbeat) --es-url=http://elastic:changeme@localhost:9200 --kibana-url=http://elastic:changeme@localhost:5601` --------- Co-authored-by: Kibana Machine <[email protected]>
kqualters-elastic
pushed a commit
to kqualters-elastic/kibana
that referenced
this issue
Feb 6, 2023
…kage and metricbeat data (elastic#149161) ### Summary Part of elastic#148303 Closes elastic#146067 Closes elastic#146068 I investigated the failing tests but couldn't get to the root cause quickly enough so I took the opportunity to port the tests to the [monitoring_api_integration](https://github.com/elastic/kibana/tree/main/x-pack/test/monitoring_api_integration) suite. This reimplements all the existing tests (plus missing ones for the `ml_jobs` api) with fresh data. By using the new testing approach we can remove the archived mappings which greatly reduce execution times. ### Testing - The suite was executed against by the [flaky test runner](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1804) multiple times with no failures - Test data is bundled in two archives (one for metricbeat and one for package data) and can be loaded to verify their content: `node scripts/es_archiver.js load x-pack/test/monitoring_api_integration/archives/elasticsearch/single_node/(package|metricbeat) --es-url=http://elastic:changeme@localhost:9200 --kibana-url=http://elastic:changeme@localhost:5601` --------- Co-authored-by: Kibana Machine <[email protected]>
klacabane
added a commit
that referenced
this issue
Mar 8, 2023
…beat data (#152575) ## Summary Closes #151158 Part of #148303 Port logstash tests to the [monitoring_api_integration](https://github.com/elastic/kibana/tree/main/x-pack/test/monitoring_api_integration) suite. This reimplements all the existing tests (plus missing ones for the pipeline api) with fresh data. See [flaky test run](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1982)
bmorelli25
pushed a commit
to bmorelli25/kibana
that referenced
this issue
Mar 10, 2023
…beat data (elastic#152575) ## Summary Closes elastic#151158 Part of elastic#148303 Port logstash tests to the [monitoring_api_integration](https://github.com/elastic/kibana/tree/main/x-pack/test/monitoring_api_integration) suite. This reimplements all the existing tests (plus missing ones for the pipeline api) with fresh data. See [flaky test run](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1982)
klacabane
added a commit
that referenced
this issue
Mar 20, 2023
…at data (#153184) ## Summary Part of #148303 Port kibana api tests to the [monitoring_api_integration](https://github.com/elastic/kibana/tree/main/x-pack/test/monitoring_api_integration) suite. This reimplements all the existing tests with fresh data. See [flaky test run](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2002)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Summary
Foundations for a process allowing validation of the latest version of the Stack Monitoring packages was implemented in #146000. Right now only Beats/APM tests are implemented with this workflow and we should have coverage for all the supported services.
We currently have api tests for legacy, metricbeat, and package data under the
x-pack/test/api_integration/apis/monitoringdirectory. Taking the kibana test suite as example, and more specifically theinstancetests:instance.jsinstance_mb.jsThis effort aims at replacing the metricbeat and package tests with the approach introduced in #146000. While the main driver is the continuous testing of the mappings, it will also create a new baseline with recent and accurate datasets.
Workflow
At the very least each stack monitoring API should have a corresponding test. Variants of these APIs (ie test with single node, multiple nodes..) can be added when relevant. Use the existing test suites at
x-pack/test/api_integration/apis/monitoringto assess the need for multiple test cases.Refer to the documentation to generate new data and for additional context.
Refer to the implementation of the Beat package tests for the test suite blueprint: #147755.
Tasks
The text was updated successfully, but these errors were encountered: